Ceph由于更换IP地址导致MON异常
引言
由于机器要更换IP地址，更改重启后，各ceph监控进程出现异常，无法启动，本文用于记录并说明该问题。
现象
更换IP地址后，直接启动mon无法启动，总是发现报绑定IP地址的错误。初步猜测为：
进程需要绑定本地的IP:PORT进行监听，地址已改，而绑定的还是修改前的地址，绑定失败，所以报错。
[root@gnop029-ct-zhejiang_wenzhou-16-20 ceph]# ./startmon.sh
monmaptool: monmap file /etc/ceph/monmap
monmaptool: /etc/ceph/monmap exists, --clobber to overwrite
'/var/lib/ceph/mon/ceph-a' already exists and is not empty: monitor may already exist
=== mon.a ===
Starting Ceph mon.a on ceph-10...
2015-11-23 13:23:47.855887 7f4a5a36a7a0 -1 WARNING: 'mon addr' config option 101.71.4.20:6789/0 does not match monmap file
continuing with monmap configuration
2015-11-23 13:23:47.856064 7f4a5a36a7a0 -1 accepter.accepter.bind unable to bind to 101.67.163.20:6789: (99) Cannot assign requested address
2015-11-23 13:23:47.856085 7f4a5a36a7a0 -1 accepter.accepter.bind was unable to bind. Trying again in 5 seconds
2015-11-23 13:23:52.856281 7f4a5a36a7a0 -1 accepter.accepter.bind unable to bind to 101.67.163.20:6789: (99) Cannot assign requested address
2015-11-23 13:23:52.856319 7f4a5a36a7a0 -1 accepter.accepter.bind was unable to bind. Trying again in 5 seconds
2015-11-23 13:23:57.856622 7f4a5a36a7a0 -1 accepter.accepter.bind unable to bind to 101.67.163.20:6789: (99) Cannot assign requested address
2015-11-23 13:23:57.856657 7f4a5a36a7a0 -1 accepter.accepter.bind was unable to bind after 3 attempts: (99) Cannot assign requested address
2015-11-23 13:23:57.856670 7f4a5a36a7a0 -1 unable to bind monitor to 101.67.163.20:6789/0
failed: 'ssh ceph-10 ulimit -n 32768; /usr/bin/ceph-mon -i a --pid-file /var/run/ceph/mon.a.pid -c /etc/ceph/ceph.conf --cluster ceph '
解决办法
修改ceph.conf配置文件，并重新生成monmap文件，即可启动mon进程
配置文件修改：
public_network = 101.71.4.0/24
cluster_network = 101.71.4.0/24
重新生成monmap文件：
sudo monmaptool --create --generate -c /etc/ceph/ceph.conf /etc/ceph/monmap
也可以通过命令得到新的monmap
#Add the new monitor locations
# monmaptool --create --add mon0 192.168.32.2:6789 --add osd1 192.168.32.3:6789 \
--add osd2 192.168.32.4:6789 --fsid 61a520db-317b-41f1-9752-30cedc5ffb9a \
--clobber monmap
#Retrieve the monitor map
# ceph mon getmap -o monmap.bin
#Check new contents
# monmaptool --print monmap.bin
#Inject the monmap
# ceph-mon -i mon0 --inject-monmap monmap.bin
# ceph-mon -i osd1 --inject-monmap monmap.bin
# ceph-mon -i osd2 --inject-monmap monmap.bin
按照上述办法，即可解决IP地址修改导致监控进程无法启动的问题。
引言
由于机器要更换IP地址，更改重启后，各ceph监控进程出现异常，无法启动，本文用于记录并说明该问题。
现象
更换IP地址后，直接启动mon无法启动，总是发现报绑定IP地址的错误。初步猜测为：
进程需要绑定本地的IP:PORT进行监听，地址已改，而绑定的还是修改前的地址，绑定失败，所以报错。
[root@gnop029-ct-zhejiang_wenzhou-16-20 ceph]# ./startmon.sh
monmaptool: monmap file /etc/ceph/monmap
monmaptool: /etc/ceph/monmap exists, --clobber to overwrite
'/var/lib/ceph/mon/ceph-a' already exists and is not empty: monitor may already exist
=== mon.a ===
Starting Ceph mon.a on ceph-10...
2015-11-23 13:23:47.855887 7f4a5a36a7a0 -1 WARNING: 'mon addr' config option 101.71.4.20:6789/0 does not match monmap file
continuing with monmap configuration
2015-11-23 13:23:47.856064 7f4a5a36a7a0 -1 accepter.accepter.bind unable to bind to 101.67.163.20:6789: (99) Cannot assign requested address
2015-11-23 13:23:47.856085 7f4a5a36a7a0 -1 accepter.accepter.bind was unable to bind. Trying again in 5 seconds
2015-11-23 13:23:52.856281 7f4a5a36a7a0 -1 accepter.accepter.bind unable to bind to 101.67.163.20:6789: (99) Cannot assign requested address
2015-11-23 13:23:52.856319 7f4a5a36a7a0 -1 accepter.accepter.bind was unable to bind. Trying again in 5 seconds
2015-11-23 13:23:57.856622 7f4a5a36a7a0 -1 accepter.accepter.bind unable to bind to 101.67.163.20:6789: (99) Cannot assign requested address
2015-11-23 13:23:57.856657 7f4a5a36a7a0 -1 accepter.accepter.bind was unable to bind after 3 attempts: (99) Cannot assign requested address
2015-11-23 13:23:57.856670 7f4a5a36a7a0 -1 unable to bind monitor to 101.67.163.20:6789/0
failed: 'ssh ceph-10 ulimit -n 32768; /usr/bin/ceph-mon -i a --pid-file /var/run/ceph/mon.a.pid -c /etc/ceph/ceph.conf --cluster ceph '
解决办法
修改ceph.conf配置文件，并重新生成monmap文件，即可启动mon进程
配置文件修改：
public_network = 101.71.4.0/24
cluster_network = 101.71.4.0/24
重新生成monmap文件：
sudo monmaptool --create --generate -c /etc/ceph/ceph.conf /etc/ceph/monmap
也可以通过命令得到新的monmap
#Add the new monitor locations
# monmaptool --create --add mon0 192.168.32.2:6789 --add osd1 192.168.32.3:6789 \
--add osd2 192.168.32.4:6789 --fsid 61a520db-317b-41f1-9752-30cedc5ffb9a \
--clobber monmap
#Retrieve the monitor map
# ceph mon getmap -o monmap.bin
#Check new contents
# monmaptool --print monmap.bin
#Inject the monmap
# ceph-mon -i mon0 --inject-monmap monmap.bin
# ceph-mon -i osd1 --inject-monmap monmap.bin
# ceph-mon -i osd2 --inject-monmap monmap.bin
按照上述办法，即可解决IP地址修改导致监控进程无法启动的问题。