Redis热迁移实战分享

mr923

　　Redis cluster使用slots来分配集群中的资源，因此官方提供了热迁移slots的方案，以便于迁移redis cluster节点中的信息。此方案不仅可以使用于节点迁移，也可以根据资源的不同，配置不同的slots数量。
　　集群原环境：
　　主机IP： 192.168.170.101
　　集群节点端口：10001-10006
　　集群当前主备关系：
　　迁移需求：现在有节点redis007,需要将redis002替换掉
　　迁移步骤：

• 　　首先验证环境当前的配置：
  

　　由上可以看出节点信息如下：
　　10001M Adding node 192.168.170.101:10007 to cluster 192.168.170.101:10001
　　>>> Performing Cluster Check (using node 192.168.170.101:10001)
　　M: b30fdabd07a4bef611c160828965b91a1cdd462a 192.168.170.101:10001
　　slots:0-5460 (5461 slots) master
　　1 additional replica(s)
　　S: 74805544792cc22f09af941fa117b2974c9c0abf 192.168.170.101:10005
　　slots: (0 slots) slave
　　replicates 6036af6afc6567b74ce2fecc734a2d3908d561d1
　　M: 9c6a98926b23585e2eeb49ae17cc77521041d3ed 192.168.170.101:10003
　　slots:10923-16383 (5461 slots) master
　　1 additional replica(s)
　　S: 57a2e7313b952b38e32d6a1030d4812ea4decb5f 192.168.170.101:10006
　　slots: (0 slots) slave
　　replicates 9c6a98926b23585e2eeb49ae17cc77521041d3ed
　　S: 58b7b25501fdba04aef9b8cb47edef89092f10a7 192.168.170.101:10004
　　slots: (0 slots) slave
　　replicates b30fdabd07a4bef611c160828965b91a1cdd462a
　　M: 6036af6afc6567b74ce2fecc734a2d3908d561d1 192.168.170.101:10002
　　slots:5461-10922 (5462 slots) master
　　1 additional replica(s)
　　[OK] All nodes agree about slots configuration.
　　>>> Check for open slots...
　　>>> Check slots coverage...
　　[OK] All 16384 slots covered.
　　>>> Send CLUSTER MEET to node 192.168.170.101:10007 to make it join the cluster.
　　[OK] New node added correctly.

• 　　查看当前节点状态信息
  

　　[root@lab001 redis]# redis-trib.rb check 192.168.170.101:10001
　　>>> Performing Cluster Check (using node 192.168.170.101:10001)
　　M: b30fdabd07a4bef611c160828965b91a1cdd462a 192.168.170.101:10001
　　slots:0-5460 (5461 slots) master
　　1 additional replica(s)
　　M: 7c1a0acf84b2cf08876036b07518fb8f2ea9b8f8 192.168.170.101:10007
　　slots: (0 slots) master
　　0 additional replica(s)
　　S: 74805544792cc22f09af941fa117b2974c9c0abf 192.168.170.101:10005
　　slots: (0 slots) slave
　　replicates 6036af6afc6567b74ce2fecc734a2d3908d561d1
　　M: 9c6a98926b23585e2eeb49ae17cc77521041d3ed 192.168.170.101:10003
　　slots:10923-16383 (5461 slots) master
　　1 additional replica(s)
　　S: 57a2e7313b952b38e32d6a1030d4812ea4decb5f 192.168.170.101:10006
　　slots: (0 slots) slave
　　replicates 9c6a98926b23585e2eeb49ae17cc77521041d3ed
　　S: 58b7b25501fdba04aef9b8cb47edef89092f10a7 192.168.170.101:10004
　　slots: (0 slots) slave
　　replicates b30fdabd07a4bef611c160828965b91a1cdd462a
　　M: 6036af6afc6567b74ce2fecc734a2d3908d561d1 192.168.170.101:10002
　　slots:5461-10922 (5462 slots) master
　　1 additional replica(s)
　　[OK] All nodes agree about slots configuration.
　　>>> Check for open slots...
　　>>> Check slots coverage...
　　[OK] All 16384 slots covered.

• 　　由上可以发现，redis007添加到节点当中，但当前未分配任何slots，这一步把redis002的slots完全迁移到redis007
  

　　由于过程较长，这里不再输出全部过程
　　[root@lab001 redis]# redis-trib.rb reshard --from 6036af6afc6567b74ce2fecc734a2d3908d561d1 --to 7c1a0acf84b2cf08876036b07518fb8f2ea9b8f8 --slots 5462 --yes 192.168.170.101:10001
　　………………
　　Moving slot 10911 from 192.168.170.101:10002 to 192.168.170.101:10007: .....
　　Moving slot 10912 from 192.168.170.101:10002 to 192.168.170.101:10007: ..........
　　Moving slot 10913 from 192.168.170.101:10002 to 192.168.170.101:10007: ......
　　Moving slot 10914 from 192.168.170.101:10002 to 192.168.170.101:10007: .......
　　Moving slot 10915 from 192.168.170.101:10002 to 192.168.170.101:10007: .....
　　Moving slot 10916 from 192.168.170.101:10002 to 192.168.170.101:10007: .....
　　Moving slot 10917 from 192.168.170.101:10002 to 192.168.170.101:10007: ...
　　Moving slot 10918 from 192.168.170.101:10002 to 192.168.170.101:10007: ........
　　Moving slot 10919 from 192.168.170.101:10002 to 192.168.170.101:10007: .....
　　Moving slot 10920 from 192.168.170.101:10002 to 192.168.170.101:10007: .......
　　Moving slot 10921 from 192.168.170.101:10002 to 192.168.170.101:10007: ..
　　Moving slot 10922 from 192.168.170.101:10002 to 192.168.170.101:10007: ..........

• 　　再次查看当前节点信息
  

　　[root@lab001 redis]# redis-trib.rb check 192.168.170.101:10001
　　>>> Performing Cluster Check (using node 192.168.170.101:10001)
　　M: b30fdabd07a4bef611c160828965b91a1cdd462a 192.168.170.101:10001
　　slots:0-5460 (5461 slots) master
　　1 additional replica(s)
　　M: 7c1a0acf84b2cf08876036b07518fb8f2ea9b8f8 192.168.170.101:10007
　　slots:5461-10922 (5462 slots) master
　　1 additional replica(s)
　　S: 74805544792cc22f09af941fa117b2974c9c0abf 192.168.170.101:10005
　　slots: (0 slots) slave
　　replicates 7c1a0acf84b2cf08876036b07518fb8f2ea9b8f8
　　M: 9c6a98926b23585e2eeb49ae17cc77521041d3ed 192.168.170.101:10003
　　slots:10923-16383 (5461 slots) master
　　1 additional replica(s)
　　S: 57a2e7313b952b38e32d6a1030d4812ea4decb5f 192.168.170.101:10006
　　slots: (0 slots) slave
　　replicates 9c6a98926b23585e2eeb49ae17cc77521041d3ed
　　S: 58b7b25501fdba04aef9b8cb47edef89092f10a7 192.168.170.101:10004
　　slots: (0 slots) slave
　　replicates b30fdabd07a4bef611c160828965b91a1cdd462a
　　M: 6036af6afc6567b74ce2fecc734a2d3908d561d1 192.168.170.101:10002
　　slots: (0 slots) master
　　0 additional replica(s)
　　[OK] All nodes agree about slots configuration.
　　>>> Check for open slots...
　　>>> Check slots coverage...
　　[OK] All 16384 slots covered.
　　由此可以发现，redis002的所有slots迁移到redis007完毕，并且redis005自动将主从关系从002转移到了007

• 　　最后来验证下数据的完整性
  

　　from rediscluster import StrictRedisCluster
　　redis_nodes = [
　　{"host": "192.168.170.101", "port": "10001"},
　　{"host": "192.168.170.101", "port": "10002"},
　　{"host": "192.168.170.101", "port": "10003"},
　　{"host": "192.168.170.101", "port": "10004"},
　　{"host": "192.168.170.101", "port": "10005"},
　　{"host": "192.168.170.101", "port": "10006"}
　　]
　　redis_conn = StrictRedisCluster(startup_nodes=redis_nodes, decode_responses=True)
　　for key in range(0, 100000):
　　#value = key
　　key = 'zhang%s' % key
　　#redis_conn.set(key, value)
　　print redis_conn.get(key)
　　执行完毕，数据较验完成，数据完成完整迁移

• 　　最后，将redis002从集群中移除
  

　　[root@lab001 redis]# redis-trib.rb del-node 192.168.170.101:10001 6036af6afc6567b74ce2fecc734a2d3908d561d1
　　>>> Removing node 6036af6afc6567b74ce2fecc734a2d3908d561d1 from cluster 192.168.170.101:10001
　　>>> Sending CLUSTER FORGET messages to the cluster...
　　>>> SHUTDOWN the node.
　　[2]   Done                    ./redis001/bin/redis-server redis00$i/redis00$i.conf

• 　　再次查看集群状态
  

　　[root@lab001 redis]# redis-trib.rb check 192.168.170.101:10001
　　>>> Performing Cluster Check (using node 192.168.170.101:10001)
　　M: b30fdabd07a4bef611c160828965b91a1cdd462a 192.168.170.101:10001
　　slots:0-5460 (5461 slots) master
　　1 additional replica(s)
　　M: 7c1a0acf84b2cf08876036b07518fb8f2ea9b8f8 192.168.170.101:10007
　　slots:5461-10922 (5462 slots) master
　　1 additional replica(s)
　　S: 74805544792cc22f09af941fa117b2974c9c0abf 192.168.170.101:10005
　　slots: (0 slots) slave
　　replicates 7c1a0acf84b2cf08876036b07518fb8f2ea9b8f8
　　M: 9c6a98926b23585e2eeb49ae17cc77521041d3ed 192.168.170.101:10003
　　slots:10923-16383 (5461 slots) master
　　1 additional replica(s)
　　S: 57a2e7313b952b38e32d6a1030d4812ea4decb5f 192.168.170.101:10006
　　slots: (0 slots) slave
　　replicates 9c6a98926b23585e2eeb49ae17cc77521041d3ed
　　S: 58b7b25501fdba04aef9b8cb47edef89092f10a7 192.168.170.101:10004
　　slots: (0 slots) slave
　　replicates b30fdabd07a4bef611c160828965b91a1cdd462a
　　[OK] All nodes agree about slots configuration.
　　>>> Check for open slots...
　　>>> Check slots coverage...
　　[OK] All 16384 slots covered.
　　[root@lab001 redis]# redis-trib.rb info 192.168.170.101:10001
　　192.168.170.101:10001 (b30fdabd...) -> 33357 keys | 5461 slots | 1 slaves.
　　192.168.170.101:10007 (7c1a0acf...) -> 33283 keys | 5462 slots | 1 slaves.
　　192.168.170.101:10003 (9c6a9892...) -> 33360 keys | 5461 slots | 1 slaves.
　　[OK] 100000 keys in 3 masters.
　　6.10 keys per slot on average.
　　总结：
　　1. slots迁移在日常工作中，通常用于节点迁移、扩展，也可以根据服务器本身的压力将slots迁移至性能较优的服务器
　　2. slots迁移过程当中，不影响数据的读写，这点已经做过实际的测试。
　　3. 迁移过程当中，应用配置应当至少包含一个集群master节点信息，否则有可能会造成数据访问异常的情况。迁移完成并配置同步更新至最新后，方可删除旧的空slots节点。