使用 cephadm bootstrap 初始化集群的第一个节点，会安装集群中的第一个 monitor daemon，必须要指定集群通信的 IP 地址。执行以下命令 [3]
$ cephadm bootstrap --mon-ip 10.111.30.100 Verifying podman|docker is present... Verifying lvm2 is present... Verifying time synchronization is in place... Unit chronyd.service is enabled and running Repeating the final host check... podman|docker (/usr/bin/docker) is present systemctl is present lvcreate is present Unit chronyd.service is enabled and running Host looks OK Cluster fsid: e2b9a77e-0c23-11ee-9e9d-000c29687fa4 Verifying IP 10.111.30.100 port 3300 ... Verifying IP 10.111.30.100 port 6789 ... Mon IP 10.111.30.100 is in CIDR network 10.111.30.0/24 Pulling container image quay.io/ceph/ceph:v15... Extracting ceph user uid/gid from container image... Creating initial keys... Creating initial monmap... Creating mon... Waiting for mon to start... Waiting for mon... mon is available Assimilating anything we can from ceph.conf... Generating new minimal ceph.conf... Restarting the monitor... Setting mon public_network... Creating mgr... Verifying port 9283 ... Wrote keyring to /etc/ceph/ceph.client.admin.keyring Wrote config to /etc/ceph/ceph.conf Waiting for mgr to start... Waiting for mgr... mgr not available, waiting (1/10)... mgr not available, waiting (2/10)... mgr not available, waiting (3/10)... mgr not available, waiting (4/10)... mgr is available Enabling cephadm module... Waiting for the mgr to restart... Waiting for Mgr epoch 5... Mgr epoch 5 is available Setting orchestrator backend to cephadm... Generating ssh key... Wrote public SSH key to to /etc/ceph/ceph.pub Adding key to root@localhost's authorized_keys... Adding host ceph-node-1... Deploying mon service with default placement... Deploying mgr service with default placement... Deploying crash service with default placement... Enabling mgr prometheus module... Deploying prometheus service with default placement... Deploying grafana service with default placement... Deploying node-exporter service with default placement... Deploying alertmanager service with default placement... Enabling the dashboard module... Waiting for the mgr to restart... Waiting for Mgr epoch 13... Mgr epoch 13 is available Generating a dashboard self-signed certificate... Creating initial admin user... Fetching dashboard port number... Ceph Dashboard is now available at:
Add the _admin label to the bootstrap host. By default, any host with this label will (also) get a copy of /etc/ceph/ceph.conf and /etc/ceph/ceph.client.admin.keyring
$ docker ps -a CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 48657ea7b022 quay.io/ceph/ceph-grafana:6.7.4 "/bin/sh -c 'grafana…" 22 minutes ago Up 22 minutes ceph-e2b9a77e-0c23-11ee-9e9d-000c29687fa4-grafana.ceph-node-1 cc704589a17f quay.io/prometheus/alertmanager:v0.20.0 "/bin/alertmanager -…" 22 minutes ago Up 22 minutes ceph-e2b9a77e-0c23-11ee-9e9d-000c29687fa4-alertmanager.ceph-node-1 86f95aba68c0 quay.io/prometheus/prometheus:v2.18.1 "/bin/prometheus --c…" 22 minutes ago Up 22 minutes ceph-e2b9a77e-0c23-11ee-9e9d-000c29687fa4-prometheus.ceph-node-1 9b6801c15353 quay.io/prometheus/node-exporter:v0.18.1 "/bin/node_exporter …" 23 minutes ago Up 23 minutes ceph-e2b9a77e-0c23-11ee-9e9d-000c29687fa4-node-exporter.ceph-node-1 459bf96f7646 quay.io/ceph/ceph:v15 "/usr/bin/ceph-crash…" 29 minutes ago Up 29 minutes ceph-e2b9a77e-0c23-11ee-9e9d-000c29687fa4-crash.ceph-node-1 0399340209dc quay.io/ceph/ceph:v15 "/usr/bin/ceph-mgr -…" 31 minutes ago Up 31 minutes ceph-e2b9a77e-0c23-11ee-9e9d-000c29687fa4-mgr.ceph-node-1.bswaqn 93b3f483d33b quay.io/ceph/ceph:v15 "/usr/bin/ceph-mon -…" 31 minutes ago Up 31 minutes ceph-e2b9a77e-0c23-11ee-9e9d-000c29687fa4-mon.ceph-node-1
[ceph: root@ceph-node-1 /]# ceph orch host ls HOST ADDR LABELS STATUS ceph-node-1 ceph-node-1 ceph-node-2 10.111.30.110 ceph-node-3 10.111.30.120
查看集群内运行的组件
查看所有组件 (daemons) 的状态
[ceph: root@ceph-node-1 /]# ceph orch ps NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID alertmanager.ceph-node-1 ceph-node-1 running (42m) 2m ago 2d 0.20.0 quay.io/prometheus/alertmanager:v0.20.0 0881eb8f169f d14854792fd0 crash.ceph-node-1 ceph-node-1 running (2d) 2m ago 2d 15.2.17 quay.io/ceph/ceph:v15 93146564743f 459bf96f7646 crash.ceph-node-2 ceph-node-2 running (43m) 2m ago 43m 15.2.17 quay.io/ceph/ceph:v15 93146564743f a6fcd737a77e crash.ceph-node-3 ceph-node-3 running (5h) 2m ago 5h 15.2.17 quay.io/ceph/ceph:v15 93146564743f 100dfe6be1c6 grafana.ceph-node-1 ceph-node-1 running (2d) 2m ago 2d 6.7.4 quay.io/ceph/ceph-grafana:6.7.4 557c83e11646 48657ea7b022 mgr.ceph-node-1.bswaqn ceph-node-1 running (2d) 2m ago 2d 15.2.17 quay.io/ceph/ceph:v15 93146564743f 0399340209dc mgr.ceph-node-2.jmyqrh ceph-node-2 running (42m) 2m ago 42m 15.2.17 quay.io/ceph/ceph:v15 93146564743f 52bf4fd6dc85 mon.ceph-node-1 ceph-node-1 running (2d) 2m ago 2d 15.2.17 quay.io/ceph/ceph:v15 93146564743f 93b3f483d33b mon.ceph-node-2 ceph-node-2 running (42m) 2m ago 42m 15.2.17 quay.io/ceph/ceph:v15 93146564743f 429a884250d0 mon.ceph-node-3 ceph-node-3 running (5h) 2m ago 5h 15.2.17 quay.io/ceph/ceph:v15 93146564743f c1a83b532057 node-exporter.ceph-node-1 ceph-node-1 running (2d) 2m ago 2d 0.18.1 quay.io/prometheus/node-exporter:v0.18.1 e5a616e4b9cf 9b6801c15353 node-exporter.ceph-node-2 ceph-node-2 running (42m) 2m ago 42m 0.18.1 quay.io/prometheus/node-exporter:v0.18.1 e5a616e4b9cf a4a1551d8512 node-exporter.ceph-node-3 ceph-node-3 running (5h) 2m ago 5h 0.18.1 quay.io/prometheus/node-exporter:v0.18.1 e5a616e4b9cf 807e73c74634 prometheus.ceph-node-1 ceph-node-1 running (42m) 2m ago 2d 2.18.1 quay.io/prometheus/prometheus:v2.18.1 de242295e225 51f52e64da3c quay.io/prometheus/prometheus:v2.18.1 de242295e225
查看某一类型的组件的状态
[ceph: root@ceph-node-1 /]# ceph orch ps --daemon-type mon NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID mon.ceph-node-1 ceph-node-1 running (2d) 7m ago 2d 15.2.17 quay.io/ceph/ceph:v15 93146564743f 93b3f483d33b mon.ceph-node-3 ceph-node-3 running (3h) 7m ago 3h 15.2.17 quay.io/ceph/ceph:v15 93146564743f c1a83b532057
列出服务状态
列出所有服务的状态
[ceph: root@ceph-node-1 /]# ceph orch ls NAME RUNNING REFRESHED AGE PLACEMENT IMAGE NAME IMAGE ID alertmanager 1/1 116s ago 2d count:1 quay.io/prometheus/alertmanager:v0.20.0 0881eb8f169f crash 3/3 117s ago 2d * quay.io/ceph/ceph:v15 93146564743f grafana 1/1 116s ago 2d count:1 quay.io/ceph/ceph-grafana:6.7.4 557c83e11646 mgr 2/2 117s ago 2d count:2 quay.io/ceph/ceph:v15 93146564743f mon 3/3 117s ago 97m ceph-node-1;ceph-node-2;ceph-node-3;count:3 quay.io/ceph/ceph:v15 93146564743f node-exporter 3/3 117s ago 2d * quay.io/prometheus/node-exporter:v0.18.1 e5a616e4b9cf prometheus 1/1 116s ago 2d count:1 quay.io/prometheus/prometheus:v2.18.1 de242295e225
列出某一个服务的状态
[ceph: root@ceph-node-1 /]# ceph orch ls mon NAME RUNNING REFRESHED AGE PLACEMENT IMAGE NAME IMAGE ID mon 3/3 4m ago 99m ceph-node-1;ceph-node-2;ceph-node-3;count:3 quay.io/ceph/ceph:v15 93146564743f
[ceph: root@ceph-node-1 /]# ceph orch ls mgr NAME RUNNING REFRESHED AGE PLACEMENT IMAGE NAME IMAGE ID mgr 2/2 5m ago 2d count:2 quay.io/ceph/ceph:v15 93146564743f
$ ./cephadm Traceback (most recent call last): File "/root/./cephadm", line 27, in <module> import ssl File "/usr/local/python3/lib/python3.10/ssl.py", line 99, in <module> import _ssl # if we can't import it, let the error propagate ModuleNotFoundError: No module named '_ssl'
$ ./cephadm add-repo --release quincy ERROR: Ceph does not support pacific or later for this version of this linux distro and therefore cannot add a repo for it
$ ./cephadm install yum: stderr http://repo.ugm.ac.id/epel/7/x86_64/repodata/c1f20191b9c9dd3a8605e007aec862f995ac567d71ea1adabee495f14d4ea699-primary.sqlite.bz2: [Errno 14] HTTP Error 404 - Not Found yum: stderr Trying other mirror. yum: stderr To address this issue please refer to the below wiki article yum: stderr yum: stderr https://wiki.centos.org/yum-errors yum: stderr yum: stderr If above article doesn't help to resolve this issue please use https://bugs.centos.org/. yum: stderr yum: stderr warning: /var/cache/yum/x86_64/7/Ceph-noarch/packages/cephadm-15.2.17-0.el7.noarch.rpm: Header V4 RSA/SHA256 Signature, key ID 460f3994: NOKEY yum: stderr yum: stderr yum: stderr Invalid GPG Key from https://download.ceph.com/keys/release.gpg: No key found in given key data Traceback (most recent call last): File "/root/./cephadm", line 9667, in <module> main() File "/root/./cephadm", line 9655, in main r = ctx.func(ctx) File "/root/./cephadm", line 8129, in command_install pkg.install(ctx.packages) File "/root/./cephadm", line 7975, in install call_throws(self.ctx, [self.tool, 'install', '-y'] + ls) File "/root/./cephadm", line 1795, in call_throws raise RuntimeError('Failed command: %s' % ' '.join(command)) RuntimeError: Failed command: yum install -y cephadm
$ cephadm bootstrap --mon-ip 10.111.30.100 ERROR: hostname is a fully qualified domain name (localhost.localdomain); either fix (e.g., "sudo hostname localhost" or similar) or pass --allow-fqdn-hostname
