Redis HA+Sentinel

2322312

环境：Centos 6.6
redis version：3.2.5

sentinel是一个管理redis实例的工具，它可以实现对redis的监控、通知、自动故障转移。sentinel不断的检测redis实例是否可以正常工作，通过API向其他程序报告redis的状态，如果redis master不能工作，则会自动启动故障转移进程，将其中的一个slave提升为master，其他的slave重新设置新的master服务器。
配置redis HA+哨兵(sentinel)：
在redis1上配置：

1 2 3 4

[iyunv@redis ~]# cd /usr/local/redis-3.2.5/ [iyunv@redis redis-3.2.5]# make [iyunv@redis redis-3.2.5]# make PREFIX=/usr/local/redis-sentinel install [iyunv@redis redis-3.2.5]# make test

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60

[iyunv@redis etc]# cd /usr/local/redis-3.2.5/ [iyunv@redis redis-3.2.5]# cp redis.conf redis.conf.default [iyunv@redis ~]# mkdir -p /usr/local/redis-3.2.5/{data1,data2} redis1 （6379）配置： [iyunv@redis ~]# grep -v '^#' /usr/local/redis-3.2.5/redis_6379.conf | grep -v '^$'   bind 192.168.31.100 protected-mode yes port 6379 tcp-backlog 511 timeout 0 tcp-keepalive 300 daemonize yes supervised no pidfile "/var/run/redis_6379.pid" loglevel notice logfile "/var/log/redis_6379.log" databases 16 save 900 1 save 300 10 save 60 10000 stop-writes-on-bgsave-error yes rdbcompression yes rdbchecksum yes dbfilename "dump_6379.rdb" dir "/usr/local/redis-3.2.5/data1" repl-diskless-sync no repl-diskless-sync-delay 5 repl-disable-tcp-nodelay no slave-priority 100 requirepass 1234567 maxmemory-policy volatile-lru maxmemory 8gb appendonly yes appendfilename "appendonly.aof" appendfsync everysec no-appendfsync-on-rewrite yes auto-aof-rewrite-percentage 100 auto-aof-rewrite-min-size 64mb aof-load-truncated yes lua-time-limit 5000 slowlog-log-slower-than 10000 slowlog-max-len 128 latency-monitor-threshold 0 notify-keyspace-events "" hash-max-ziplist-entries 512 hash-max-ziplist-value 64 list-max-ziplist-size -2 list-compress-depth 0 set-max-intset-entries 512 zset-max-ziplist-entries 128 zset-max-ziplist-value 64 hll-sparse-max-bytes 3000 activerehashing yes client-output-buffer-limit normal 0 0 0 client-output-buffer-limit slave 256mb 64mb 60 client-output-buffer-limit pubsub 32mb 8mb 60 hz 10 aof-rewrite-incremental-fsync yes [iyunv@redis ~]#

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58

redis2（6380）配置： [iyunv@redis ~]# grep -v '^#' /usr/local/redis-3.2.5/redis_6380.conf | grep -v '^$'   bind 192.168.31.100 protected-mode yes port 6380 tcp-backlog 511 timeout 0 tcp-keepalive 300 daemonize yes supervised no pidfile "/var/run/redis_6380.pid" loglevel notice logfile "/var/log/redis_6380.log" databases 16 save 900 1 save 300 10 save 60 10000 stop-writes-on-bgsave-error yes rdbcompression yes rdbchecksum yes dbfilename "dump_6380.rdb" dir "/usr/local/redis-3.2.5/data2" masterauth "1234567" slave-serve-stale-data yes slave-read-only yes repl-diskless-sync no repl-diskless-sync-delay 5 repl-disable-tcp-nodelay no slave-priority 100 maxmemory-policy volatile-lru maxmemory 8gb appendonly yes appendfilename "appendonly.aof" appendfsync everysec no-appendfsync-on-rewrite no auto-aof-rewrite-percentage 100 auto-aof-rewrite-min-size 64mb aof-load-truncated yes lua-time-limit 5000 slowlog-log-slower-than 10000 slowlog-max-len 128 latency-monitor-threshold 0 notify-keyspace-events "" hash-max-ziplist-entries 512 hash-max-ziplist-value 64 list-max-ziplist-size -2 list-compress-depth 0 set-max-intset-entries 512 zset-max-ziplist-entries 128 zset-max-ziplist-value 64 hll-sparse-max-bytes 3000 activerehashing yes client-output-buffer-limit normal 0 0 0 client-output-buffer-limit slave 256mb 64mb 60 client-output-buffer-limit pubsub 32mb 8mb 60 hz 10 aof-rewrite-incremental-fsync yes [iyunv@redis ~]#

1 2 3 4

[iyunv@redis redis-3.2.5]# cd /usr/local/redis-sentinel/ [iyunv@redis redis-sentinel]# mkdir etc log data [iyunv@redis redis-sentinel]# cp /usr/local/redis-3.2.5/sentinel.conf /usr/local/redis-sentinel/etc/ [iyunv@redis redis-sentinel]# cd etc/

redis Sentinel1的配置：

1 2 3 4 5 6 7 8 9

[iyunv@redis ~]# grep -v '^#' /usr/local/redis-sentinel/etc/sentinel.conf | grep -v '^$' bind 192.168.31.100 port 26379 daemonize yes dir "/usr/local/redis-sentinel/data" logfile "/usr/local/redis-sentinel/log/redis-sentinel.log" pidfile "/usr/local/redis-sentinel/sentinel.pid" sentinel monitor mymaster 192.168.31.100 6379 1 sentinel down-after-milliseconds mymaster 15000

1 2 3

[iyunv@redis ~]# ln -s /usr/local/redis-3.2.5/src/redis-server /bin/ [iyunv@redis ~]# ln -s /usr/local/redis-3.2.5/src/redis-cli /bin/ [iyunv@redis ~]# ln -s /usr/local/redis-sentinel/bin/redis-sentinel /bin/

在redis3配置：

1 2 3 4 5 6 7

[iyunv@redis2 ~]# cd /usr/local/redis-3.2.5/ [iyunv@redis2 redis-3.2.5]# make [iyunv@redis2 redis-3.2.5]# make PREFIX=/usr/local/redis-sentinel install [iyunv@redis2 redis-3.2.5]# make test [iyunv@redis2 etc]# cd /usr/local/redis-3.2.5/ [iyunv@redis2 redis-3.2.5]# cp redis.conf redis.conf.default [iyunv@redis2 ~]# mkdir -p /usr/local/redis-3.2.5/data

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59

redis3（6379）的配置： [iyunv@redis2 ~]# grep -v '^#' /usr/local/redis-3.2.5/redis.conf | grep -v '^$'   bind 192.168.31.110 protected-mode yes port 6379 tcp-backlog 511 timeout 0 tcp-keepalive 300 daemonize yes supervised no pidfile "/var/run/redis_6379.pid" loglevel notice logfile "/var/log/redis_6379.log" databases 16 save 900 1 save 300 10 save 60 10000 stop-writes-on-bgsave-error yes rdbcompression yes rdbchecksum yes dbfilename "dump_6379.rdb" dir "/usr/local/redis-3.2.5/data" slaveof 192.168.31.100 6380 masterauth "1234567" slave-serve-stale-data yes slave-read-only yes repl-diskless-sync no repl-diskless-sync-delay 5 repl-disable-tcp-nodelay no slave-priority 100 maxmemory-policy volatile-lru maxmemory 8gb appendonly yes appendfilename "appendonly.aof" appendfsync everysec no-appendfsync-on-rewrite no auto-aof-rewrite-percentage 100 auto-aof-rewrite-min-size 64mb aof-load-truncated yes lua-time-limit 5000 slowlog-log-slower-than 10000 slowlog-max-len 128 latency-monitor-threshold 0 notify-keyspace-events "" hash-max-ziplist-entries 512 hash-max-ziplist-value 64 list-max-ziplist-size -2 list-compress-depth 0 set-max-intset-entries 512 zset-max-ziplist-entries 128 zset-max-ziplist-value 64 hll-sparse-max-bytes 3000 activerehashing yes client-output-buffer-limit normal 0 0 0 client-output-buffer-limit slave 256mb 64mb 60 client-output-buffer-limit pubsub 32mb 8mb 60 hz 10 aof-rewrite-incremental-fsync yes [iyunv@redis2 ~]#

1 2 3 4

[iyunv@redis2 redis-3.2.5]# cd /usr/local/redis-sentinel/ [iyunv@redis2 redis-sentinel]# mkdir etc log data [iyunv@redis2 redis-sentinel]# cp /usr/local/redis-3.2.5/sentinel.conf /usr/local/redis-sentinel/etc/ [iyunv@redis2 redis-sentinel]# cd etc/

redis sentinel2配置：

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

[iyunv@redis2 ~]# grep -v '^#' /usr/local/redis-sentinel/etc/sentinel.conf | grep -v '^$' bind 192.168.31.110 port 26379 daemonize yes dir "/usr/local/redis-sentinel/data" logfile "/usr/local/redis-sentinel/log/redis-sentinel.log" pidfile "/usr/local/redis-sentinel/sentinel.pid" sentinel monitor mymaster 192.168.31.100 6379 1 sentinel down-after-milliseconds mymaster 15000 sentinel failover-timeout mymaster 10000 sentinel auth-pass mymaster 1234567     //需要和redis验证的密码一致 sentinel config-epoch mymaster 15 sentinel leader-epoch mymaster 8397 sentinel current-epoch 8397 [iyunv@redis2 ~]#

1 2 3

[iyunv@redis2 ~]# ln -s /usr/local/redis-3.2.5/src/redis-server /bin/ [iyunv@redis2 ~]# ln -s /usr/local/redis-3.2.5/src/redis-cli /bin/ [iyunv@redis2 ~]# ln -s /usr/local/redis-sentinel/bin/redis-sentinel /bin/

注：要先启动所以的sentinel再逐一启动各个redis：

1 2 3 4 5 6 7 8 9 10

[iyunv@redis ~]# redis-sentinel /usr/local/redis-sentinel/etc/sentinel.conf [iyunv@redis2 ~]# redis-sentinel /usr/local/redis-sentinel/etc/sentinel.conf [iyunv@redis ~]# redis-server /usr/local/redis-3.2.5/redis_6379.conf [iyunv@redis ~]# redis-server /usr/local/redis-3.2.5/redis_6380.conf [iyunv@redis2 ~]# redis-server /usr/local/redis-3.2.5/redis.conf [iyunv@redis ~]# echo "redis-sentinel /usr/local/redis-sentinel/etc/sentinel.conf" >> /etc/rc.local [iyunv@redis ~]# echo "redis-server /usr/local/redis-3.2.5/redis_6379.conf" >> /etc/rc.local [iyunv@redis ~]# echo "redis-server /usr/local/redis-3.2.5/redis_6380.conf" >> /etc/rc.local [iyunv@redis2 ~]# echo "redis-sentinel /usr/local/redis-sentinel/etc/sentinel.conf" >> /etc/rc.local [iyunv@redis2 ~]# echo "redis-server /usr/local/redis-3.2.5/redis.conf" >> /etc/rc.local

日志报错：

解决办法：

1 2 3 4 5 6 7 8

[iyunv@redis ~]# tail -4 /etc/sysctl.conf #redis net.core.somaxconn = 20480 net.ipv4.tcp_max_syn_backlog = 20480 vm.overcommit_memory =1 [iyunv@redis ~]# sysctl -p [iyunv@redis ~]# echo never > /sys/kernel/mm/transparent_hugepage/enabled [iyunv@redis ~]# echo "echo never > /sys/kernel/mm/transparent_hugepage/enabled" >> /etc/rc.local

主节点测试写入数据：


在redis slave上查看数据同步情况，slave是不能写入数据的：




成功。

测试redis Master宕机后，哨兵是否能正常切换：

1 2	[iyunv@redis ~]# /usr/local/redis-sentinel/bin/redis-cli -h 127.0.0.1 -p 26379 127.0.0.1:26379> info

OK，切换成功，之前的Master变成slave，其中一个slave变成Master

Sentinel日志：



重新启动之前宕机的redis1，日志显示其启动切换角色为slave