【Redis三】基于Redis sentinel的自动failover主从复制

yanchuen

　　在第二篇中使用2.8.17搭建了主从复制，但是它存在Master单点问题，为了解决这个问题，Redis从2.6开始引入sentinel，用于监控和管理Redis的主从复制环境，进行自动failover，即Master挂了后，sentinel自动从从服务器选出一个Master使主从复制集群仍然可以工作，如果Master醒来再次加入集群，只能以从服务器的形式工作。


什么是Sentinel

Redis Sentinel is a system designed to help managing Redis instances. It performs the following four tasks

• Monitoring. Sentinel constantly checks if your master and slave instances are working as expected.
  
• Notification. Sentinel can notify the system administrator, or another computer program, via an API, that something is wrong with one of the monitored Redis instances.
  
• Automatic failover. If a master is not working as expected, Sentinel can start a failover process where a slave is promoted to master, the other additional slaves are reconfigured to use the new master, and the applications using the Redis server informed about the new address to use when connecting.
  
• Configuration provider. Sentinel acts as a source of authority for clients service discovery: clients connect to Sentinels in order to ask for the address of the current Redis master responsible for a given service. If a failover occurs, Sentinels will report the new address.
  

Redis主从复制搭建步骤
　　Redis搭建主从复制的步骤十分简单，以下在Windows上操作。
　　1. 解压Redis压缩包至E:\devsoftware\redis-2.8.17
　　2.在Redis目录中复制两份redis.windows.conf，分别命名为redis.windows.conf.slave1,redis.windows.conf.slave2
　　3.修改redis.conf.slave1文件，

• port 6379 改为 port 6380
  
• 打开slaveof指令，改为slaveof 127.0.0.1 6379
  
• 将快照文件名改为dbfilename dump.slave1.rdb
  
• 
  如果使用AOF方式持久化，则需要将appendfilename appendonly.aof改为appendfilename appendonly.slave1.aof
  

　　4. 修改redis.conf.slave2文件

• port 6379 改为 port 6381
  
• 打开slaveof指令，改为slaveof 127.0.0.1 6379
  
• 将快照文件名改为dbfilename dump.slave2.rdb
  
• 
  如果使用AOF方式持久化，则需要将appendfilename appendonly.aof改为appendfilename appendonly.slave2.aof
  

　　5. 启动三个Redis服务器，形成一主两从的主从复制副本集
　　通过上面的启动过程，可见只需要redis-server一个启动脚本即可，它读取不同的配置文件，来完成初始化

• Master启动时的日志输出
  

[21684] 25 Nov 20:54:16 * Slave ask for synchronization
[21684] 25 Nov 20:54:16 * Starting BGSAVE for SYNC
[21684] 25 Nov 20:54:16 * Foregroud saving started by pid 21684
[21684] 25 Nov 20:54:17 * DB saved on disk
[21684] 25 Nov 20:54:17 * Background saving terminated with suc
[21684] 25 Nov 20:54:17 * Synchronization with slave succeeded

•  Slave1启动时的日志输出
  

[21068] 25 Nov 20:54:15 * Server started, Redis version 2.4.5
[21068] 25 Nov 20:54:15 * DB loaded from disk: 0 seconds
[21068] 25 Nov 20:54:15 * The server is now ready to accept connections on port
[21068] 25 Nov 20:54:16 - DB 0: 1 keys (0 volatile) in 4 slots HT.
[21068] 25 Nov 20:54:16 - 0 clients connected (0 slaves), 1180024 bytes in use
[21068] 25 Nov 20:54:16 * Connecting to MASTER...
[21068] 25 Nov 20:54:16 * MASTER <-> SLAVE sync started
[21068] 25 Nov 20:54:16 * Non blocking connect for SYNC fired the event.
[21068] 25 Nov 20:54:17 * MASTER <-> SLAVE sync: receiving 19 bytes from master
[21068] 25 Nov 20:54:17 * MASTER <-> SLAVE sync: Loading DB in memory
[21068] 25 Nov 20:54:17 * MASTER <-> SLAVE sync: Finished with success

Redis sentinel 搭建步骤

• 创建sentinel服务器的配置文件,sentinel.conf文件，内容如下
  
  

port 26389
sentinel monitor mymaster 127.0.0.1 6379 1
sentinel down-after-milliseconds mymaster 10000
sentinel failover-timeout mymaster 180000
sentinel parallel-syncs mymaster 1
 特别注意，在2.6版本还需要额外设置一个属性sentinel can-failover mymaster yes，在2.8版本无需设置这个属性，设置了反而不能启动sentinel服务器　　1. down-after-milliseconds
　　master被当前sentinel实例认定为“失效”(SDOWN)的间隔时间
　　2. failover-timeout
　　failover过期时间，当failover开始后，在此时间内仍然没有触发任何failover操作，当前sentinel  将会认为此次failoer失败
　　3. parallel-syncs
　　当新master产生时，同时进行slaveof到新master并进行同步复制的slave个数,也就是同时几个slave进行同步。因为在salve执行salveof与新master同步时，将会终止客户端请求，因此这个值需要权衡。此值较大，意味着“集群”终止客户端请求的时间总和和较大，此值较小,意味着“集群”在故障转移期间，多个salve向客户端提供服务时仍然使用旧数据

• 启动sentinel服务器
  

　　使用如下命令启动sentinel服务器

redis-server.exe sentinel.conf --sentinel
 

• 经过数据同步后，sentinel监测到当前的系统当前有一主两从环境，使用如下命令可以观察到sentinel的状态
  

redis-cli.exe -h 127.0.0.1 -p 26389 info sentinel
　　得到如下结果，结果表明，当前的master监听端口是6381

E:\devsoftware\redis-2.8.17>redis-cli.exe -h 127.0.0.1 -p 26389 info sentinel
# Sentinel
sentinel_masters:1
sentinel_tilt:0
sentinel_running_scripts:0
sentinel_scripts_queue_length:0
master0:name=mymaster,status=ok,address=127.0.0.1:6381,slaves=2,sentinels=1
 
主服务器写入从服务器读取数据测试

•  往主服务器上写入一个数据，检查从服务器是否同步
  

E:\devsoftware\redis-2.4.5-win32-win64\64bit>redis-cli.exe -h 127.0.0.1 -p 6379
redis 127.0.0.1:6379> set  abc 1
OK
redis 127.0.0.1:6379> get abc
"1"
redis 127.0.0.1:6379>Ctrl + C
 

• 两个从服务器的读取操作：
  

E:\devsoftware\redis-2.4.5-win32-win64\64bit>redis-cli.exe -h 127.0.0.1 -p 6380
redis 127.0.0.1:6380> get abc
"1"
redis 127.0.0.1:6380> ^C
E:\devsoftware\redis-2.4.5-win32-win64\64bit>redis-cli.exe -h 127.0.0.1 -p 6381
redis 127.0.0.1:6381> get abc
"1"
redis 127.0.0.1:6381>
 
从服务器写入，其它服务器读取数据测试


系统提示，


(error) READONLY You can't write against a read only slave.
 

只读slave不能写入数据，有只读的slave，就应该有可读写的slave的存在。这个看看配置文件，配置文件里有如下指令

slave-read-only yes


 配置文件对这个指令的注释是，


Since Redis 2.6 by default slaves are read-only.
Note: read only slaves are not designed to be exposed to untrusted clients on the internet. It's just a protection layer against misuse of the instance. Still a read only slave exports by default all the administrative commands
such as CONFIG, DEBUG, and so forth. To a limited extend you can improve security of read only slaves using 'rename-command' to shadow all the  administrative / dangerous commands.
 

主服务器挂了，是否会自动的从Slave中重新选择Master


　　当Master挂了之后，sentinel服务器会自动的监测到主服务器挂了，然后开始启动新Master选举工作，sentinel的工作如此重要，所以实际上sentinel本身应该是运行在集群环境中。当新的Master选举出来后，Slave重写配置文件sentinel.conf文件

port 26389
sentinel monitor mymaster 127.0.0.1 6381 1
sentinel down-after-milliseconds mymaster 10000
sentinel config-epoch mymaster 1
sentinel leader-epoch mymaster 1
# Generated by CONFIG REWRITE
dir "E:\\devsoftware\\redis-2.8.17"
sentinel known-slave mymaster 127.0.0.1 6379
sentinel known-slave mymaster 127.0.0.1 6380
sentinel current-epoch 1
　　 
　　 从上面可以看到，6381成为Master，尽管6379(原来的Master)已经挂了，但是还是在配置中作为已知的slave配置在sentinel.conf中，当6379启动后，将作为Slave加入到集群中。6379挂了，可以通过如下命令查看当前sentinel管理的从服务器信息

redis-cli -h 192.168.7.46 -p 26379 sentinel slaves mymaster
　　如果6379永远下线，所以将这个信息继续保留在sentinel中，就不合理了，所以sentinel提供了删除无效服务器的信息，通过如下命令：


E:\devsoftware\redis-2.8.17>redis-cli.exe -h 127.0.0.1 -p 26389
127.0.0.1:26389> sentinel reset mymaster
(integer) 1
　　 
　　此时再执行

redis-cli -h 192.168.7.46 -p 26379 sentinel slaves mymaster
　　可以看到之前挂掉的服务器将从sentinel中删除掉了。但是，sentenil.conf配置文件中，依然保存了6379是一个slave，也就是说，Sentinel重启后，又会加载配置文件，把6379作为slave加进来了，此时，需要手动的把6379从配置文件中删除掉。
　　最后一点，基于Sentinel的Redis主从复制不是Quarum based，也就是说，即使系统只剩下最后一台服务器，也会继续工作，而不像Zookeeper和MongoDB是Quarum Based，副本集必须保证超过半数的系统还在运行，否则，系统将不能继续工作。

Sentinel、Master和Slave自动发现

 

Sentinels and Slaves auto discovery
　　While Sentinels stay connected with other Sentinels in order to reciprocally check the availability of each other, and to exchange messages, you don't need to configure the other Sentinel addresses in every Sentinel instance you run, as Sentinel uses the Redis master Pub/Sub capabilities in order to discover the other Sentinels that are monitoring the same master.
　　This is obtained by sending Hello Messages into the channel named __sentinel__:hello.
　　Similarly you don't need to configure what is the list of the slaves attached to a master, as Sentinel will auto discover this list querying Redis.

• Every Sentinel publishes a message to every monitored master and slave Pub/Sub channel __sentinel__:hello, every two seconds, announcing its presence with ip, port, runid.
  
• Every Sentinel is subscribed to the Pub/Sub channel __sentinel__:hello of every master and slave, looking for unknown sentinels. When new sentinels are detected, they are added as sentinels of this master.
  
• Hello messages also include the full current configuration of the master. If another Sentinel has a configuration for a given master that is older than the one received, it updates to the new configuration immediately.
  
• Before adding a new sentinel to a master a Sentinel always checks if there is already a sentinel with the same runid or the same address (ip and port pair). In that case all the matching sentinels are removed, and the new added.
  

1. Q: 在Sentinel的配置文件中，我们看到sentinel只监控了Master，那么那些Slave是如何监测到的呢？

A:Sentinel通过监听Master，知道了Master的存在，那么Sentinel可以读取Master关于Slave的信息，然后交由Sentinel进行管理

2.Q:在Sentinel的配置文件中，我们看到sentinel只监控了Master,那么那些同一个集群中的Sentinel是如何发现的呢？


3. Q：当Master挂掉再重启后，将作为Slave加入到集群里，而Master的配置文件中，slaveof指令是没有配置的，也就是说，Master启动时，它自己认为自己将作为Master或者独立的Redis服务器运行，此时，Sentinel如何把它作为Slave加入到集群中的？

A: 老的Master挂掉后，新的Master产生，但是老的Master信息(IP：Port)已然存在于sentinel中，不过它已经不再是Master而是Slave。当Master启动后，Sentinel可以ping到它启动起来了，此时，Sentinel将它的配置文件加了一行，指示它已经成为别人的Slave，如

slaveof 127.0.0.1 6381
 然后通知Master，已经是Slave状态(只要修改用于指示Master/Slave的标示参数即可)