8

I configured a simple Ceph cluster with cephadm, like this:

cephadm bootstrap --mon-ip 192.168.0.4 ssh-copy-id -f -i /etc/ceph/ceph.pub root@host2 ssh-copy-id -f -i /etc/ceph/ceph.pub root@host3 ceph orch host add host2 ceph orch host add host3 ceph orch apply osd --all-available-devices

And it works fine. Now I would like to set up a cluster network on 192.168.1.0/24.

I tried ceph config set global cluster_network 192.168.1.0/24 but it doesn't have any effect. I am not seeing any traffic on the secondary NICs.

How can I troubleshoot this? What is the recommended way to make cephadm notice the change?

Improve this question

2 Answers 2

Reset to default
8

It turns out this is easy. You set the new configs:

ceph config set global cluster_network 192.168.1.0/24

Then have the orchestrator reconfigure the daemons:

ceph orch daemon reconfig mon.host1 ceph orch daemon reconfig mon.host2 ceph orch daemon reconfig mon.host3 ceph orch daemon reconfig osd.1 ceph orch daemon reconfig osd.2 ceph orch daemon reconfig osd.3 ...
Improve this answer
Sign up to request clarification or add additional context in comments.

2 Comments

how do you ensure that network_cluster is really running?
It didn't work for me, i am running octopus release.
4

Here's a link

you can specify a customized ceph.conf before the 'cephadm bootstrap' command [1] to add a dedicated cluster network (if you really need that, it has been discussed extensively on the list):

octo1:~ # cat <<EOF > /root/ceph.conf [global] public network = 192.168.124.0/24 cluster network = 192.168.127.0/24 EOF octo1:~ # cephadm bootstrap -c /root/ceph.conf --mon-ip 192.168.124.5

After you deployed the OSDs they should be attached to the cluster network:

octo1:~ # ceph osd metadata 0 | grep addr "back_addr": "[v2:192.168.127.11:6800/4272275002,v1:192.168.127.11:6801/4272275002]", "front_addr": "[v2:192.168.124.5:6802/4272275002,v1:192.168.124.5:6803/4272275002]", "hb_back_addr": "[v2:192.168.127.11:6802/4272275002,v1:192.168.127.11:6803/4272275002]", "hb_front_addr": "[v2:192.168.124.5:6804/4272275002,v1:192.168.124.5:6805/4272275002]",

You should see the replication traffic on the respective interfaces.

Improve this answer

Comments

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.