Redis3 集群功能配置初尝

hdfdtr

又很久很久没有更新博客了，也很久很久没有做云计算了，前期做了一段时间的游戏，现在又回归到常规运维的岗位，最近接触到redis这玩意，也就简单配置了下，至少要熟悉下常规的配置，才能做到运维支撑嘛。现在也没有很多时间完善博客格式什么的，就把一些笔记贴进来，方便自己后期查看，如果可以，希望对新学者有所帮助吧。

操作环境说明
10.117.41.242 yunwei_dev01 Centos6u5
10.117.41.36  yunwei_dev02  Centos6u5
版本路径：/data/src/redis-3.0.3.tar.gz
安装路径：/data/install/redis
依赖环境:gcc
一.安装redis并配置主从
1. 安装redis
[iyunv@yunwei_dev01 src]#  tar zxvf redis-3.0.3.tar.gz
[iyunv@yunwei_dev01 src]# cd redis-3.0.3
[iyunv@yunwei_dev01 redis-3.0.3]# make PREFIX=/data/install/redis install
2. 查看安装后目录/文件
[iyunv@yunwei_dev01 ~]# ls -R /data/install/redis/
/data/install/redis/:
bin
/data/install/redis/bin:
redis-benchmark  redis-check-aof  redis-check-dump  redis-cli  redis-sentinel  redis-server
3. 默认没有配置文件，创建目录，导入配置
mkdir /data/install/redis/conf

mkdir /data/install/redis/data

mkdir /data/log/redis/

cp /data/src/redis-3.0.3/redis.conf    /data/install/redis/conf/

=====redis.conf==模板====
daemonize no
pidfile /var/run/redis.pid
port 6379
tcp-backlog 511
timeout 0
tcp-keepalive 0
loglevel notice
logfile ""
databases 16
save 900 1
save 300 10
save 60 10000
stop-writes-on-bgsave-error yes
rdbcompression yes
rdbchecksum yes
dbfilename dump.rdb
dir ./
slave-serve-stale-data yes
slave-read-only yes
repl-diskless-sync no
repl-diskless-sync-delay 5
repl-disable-tcp-nodelay no
slave-priority 100
appendonly no
appendfilename "appendonly.aof"
appendfsync everysec
no-appendfsync-on-rewrite no
auto-aof-rewrite-percentage 100
auto-aof-rewrite-min-size 64mb
aof-load-truncated yes
lua-time-limit 5000
slowlog-log-slower-than 10000
slowlog-max-len 128
latency-monitor-threshold 0
notify-keyspace-events ""
hash-max-ziplist-entries 512
hash-max-ziplist-value 64
list-max-ziplist-entries 512
list-max-ziplist-value 64
set-max-intset-entries 512
zset-max-ziplist-entries 128
zset-max-ziplist-value 64
hll-sparse-max-bytes 3000
activerehashing yes
client-output-buffer-limit normal 0 0 0
client-output-buffer-limit slave 256mb 64mb 60
client-output-buffer-limit pubsub 32mb 8mb 60
hz 10
aof-rewrite-incremental-fsync yes
===========End===========
4.修改部分参数后如下显示
daemonize yes
pidfile "/data/install/redis/redis.pid"
port 6379
tcp-backlog 511
timeout 0
tcp-keepalive 0
loglevel notice
logfile "/data/log/redis/redis.log"
databases 16
save 900 1
save 300 10
save 60 10000
stop-writes-on-bgsave-error yes
rdbcompression yes
rdbchecksum yes
dbfilename dump.rdb
dir "/data/install/redis/data"
slave-serve-stale-data yes
slave-read-only yes
maxmemory 128M

masterauth  "test123"

requirepass "test123"

repl-diskless-sync no
repl-diskless-sync-delay 5
repl-disable-tcp-nodelay no
slave-priority 100
appendonly no
appendfilename "appendonly.aof"
appendfsync everysec
no-appendfsync-on-rewrite no
auto-aof-rewrite-percentage 100
auto-aof-rewrite-min-size 64mb
aof-load-truncated yes
lua-time-limit 5000
slowlog-log-slower-than 10000
slowlog-max-len 128
latency-monitor-threshold 0
notify-keyspace-events ""
hash-max-ziplist-entries 512
hash-max-ziplist-value 64
list-max-ziplist-entries 512
list-max-ziplist-value 64
set-max-intset-entries 512
zset-max-ziplist-entries 128
zset-max-ziplist-value 64
hll-sparse-max-bytes 3000
activerehashing yes
client-output-buffer-limit normal 0 0 0
client-output-buffer-limit slave 256mb 64mb 60
client-output-buffer-limit pubsub 32mb 8mb 60
hz 10
aof-rewrite-incremental-fsync yes
5. 启动redis
生产环境通过脚本启停：
/data/src/redis-3.0.3/utils/redis_init_script

测试环境直接通过命令行启动即可
[iyunv@yunwei_dev01 ~]# /data/install/redis/bin/redis-server     /data/install/redis/conf/redis.conf

[iyunv@yunwei_dev01 ~]# ps -ef |grep redis |grep -v grep
root      4179     1  0 09:14 ?        00:00:00 /data/install/redis/bin/redis-server *:6379                             
[iyunv@yunwei_dev01 ~]# netstat -nltup |grep 6379
tcp        0      0 0.0.0.0:6379                0.0.0.0:*                   LISTEN      4179/redis-server *
6. 登陆测试
[iyunv@yunwei_dev01 ~]#
127.0.0.1:6379> set myname “brucefeng”
OK
127.0.0.1:6379> get myname
“brucefeng"
7.配置从服务器  yunwei_dev02
区别：只需要将redis.conf配置中加上slaveof yunwei_dev02 port即可
slaveof 10.117.41.242 6379

统一看下master跟slave的日志吧。
=========master==========
4848:M 06 Aug 09:20:10.814 # Server started, Redis version 3.0.3
4848:M 06 Aug 09:20:10.814 # WARNING overcommit_memory is set to 0! Background save may fail under low memory condition. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect.
4848:M 06 Aug 09:20:10.814 # WARNING you have Transparent Huge Pages (THP) support enabled in your kernel. This will create latency and memory usage issues with Redis. To fix this issue run the command 'echo never > /sys/kernel/mm/transparent_hugepage/enabled' as root, and add it to your /etc/rc.local in order to retain the setting after a reboot. Redis must be restarted after THP is disabled.
4848:M 06 Aug 09:20:10.814 # WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128.
4848:M 06 Aug 09:20:10.814 * DB loaded from disk: 0.000 seconds
4848:M 06 Aug 09:20:10.814 * The server is now ready to accept connections on port 6379
4848:M 06 Aug 09:27:49.448 * Slave 10.117.41.36:6379 asks for synchronization
4848:M 06 Aug 09:27:49.449 * Full resync requested by slave 10.117.41.36:6379
4848:M 06 Aug 09:27:49.449 * Starting BGSAVE for SYNC with target: disk
4848:M 06 Aug 09:27:49.449 * Background saving started by pid 4856
4856:C 06 Aug 09:27:49.460 * DB saved on disk
4856:C 06 Aug 09:27:49.460 * RDB: 6 MB of memory used by copy-on-write
4848:M 06 Aug 09:27:49.548 * Background saving terminated with success
4848:M 06 Aug 09:27:49.548 * Synchronization with slave 10.117.41.36:6379 succeeded
=========end==========

========slave==========

28060:S 06 Aug 09:27:49.444 # Server started, Redis version 3.0.3
28060:S 06 Aug 09:27:49.444 # WARNING overcommit_memory is set to 0! Background save may fail under low memory condition. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect.
28060:S 06 Aug 09:27:49.444 # WARNING you have Transparent Huge Pages (THP) support enabled in your kernel. This will create latency and memory usage issues with Redis. To fix this issue run the command 'echo never > /sys/kernel/mm/transparent_hugepage/enabled' as root, and add it to your /etc/rc.local in order to retain the setting after a reboot. Redis must be restarted after THP is disabled.
28060:S 06 Aug 09:27:49.444 # WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128.
28060:S 06 Aug 09:27:49.444 * The server is now ready to accept connections on port 6379
28060:S 06 Aug 09:27:49.444 * Connecting to MASTER 10.117.41.242:6379
28060:S 06 Aug 09:27:49.444 * MASTER <-> SLAVE sync started
28060:S 06 Aug 09:27:49.447 * Non blocking connect for SYNC fired the event.
28060:S 06 Aug 09:27:49.448 * Master replied to PING, replication can continue...
28060:S 06 Aug 09:27:49.448 * Partial resynchronization not possible (no cached master)
28060:S 06 Aug 09:27:49.449 * Full resync from master: 88f015eff5a360a7432f5ed9f9e92556dadd0eb5:1
28060:S 06 Aug 09:27:49.549 * MASTER <-> SLAVE sync: receiving 38 bytes from master
28060:S 06 Aug 09:27:49.549 * MASTER <-> SLAVE sync: Flushing old data
28060:S 06 Aug 09:27:49.549 * MASTER <-> SLAVE sync: Loading DB in memory
28060:S 06 Aug 09:27:49.549 * MASTER <-> SLAVE sync: Finished with success
=============end================
8.查看主从同步状态：
[iyunv@yunwei_dev01 ~]# /data/install/redis/bin/redis-cli   -h 127.0.0.1 -p 6379 -a test123 info Replication
# Replication
role:master
connected_slaves:1
slave0:ip=10.117.41.36,port=6379,state=online,offset=969,lag=1
master_repl_offset:969
repl_backlog_active:1
repl_backlog_size:1048576
repl_backlog_first_byte_offset:2
repl_backlog_histlen:968
[iyunv@yunwei_dev02 ~]# /data/install/redis/bin/redis-cli   -h 127.0.0.1 -p 6379 -a test123 info Replication
# Replication
role:slave
master_host:10.117.41.242
master_port:6379
master_link_status:up
master_last_io_seconds_ago:4
master_sync_in_progress:0
slave_repl_offset:955
slave_priority:100
slave_read_only:1
connected_slaves:0
master_repl_offset:0
repl_backlog_active:0
repl_backlog_size:1048576
repl_backlog_first_byte_offset:0
repl_backlog_histlen:0
9.查看主从数据
[iyunv@yunwei_dev02 redis]# /data/install/redis/bin/redis-cli   -h 127.0.0.1 -p 6379 -a test123
127.0.0.1:6379> get myname
“brucefeng"
[iyunv@yunwei_dev01 ~]# /data/install/redis/bin/redis-cli   -h 127.0.0.1 -p 6379 -a test123
127.0.0.1:6379> set name_a "jack"
OK
[iyunv@yunwei_dev02 ~]# /data/install/redis/bin/redis-cli   -h 127.0.0.1 -p 6379 -a test123
127.0.0.1:6379> get name_a
“jack"
二.配置Redis Sentinel
Sentinel(哨兵)是用于监控redis集群中Master状态的工具，其已经被集成在redis2.4+的版本中
Sentinel作用：
1)：Master状态检测
2)：如果Master异常，则会进行Master-Slave切换，将其中一个Slave作为Master，将之前的Master作为Slave
3)：Master-Slave切换后，master_redis.conf、slave_redis.conf和sentinel.conf的内容都会发生改变，即master_redis.conf中会多一行slaveof的配置，sentinel.conf的监控目标会随之调换
Sentinel工作方式：
1)：每个Sentinel以每秒钟一次的频率向它所知的Master，Slave以及其他 Sentinel 实例发送一个 PING 命令
2)：如果一个实例（instance）距离最后一次有效回复 PING 命令的时间超过 down-after-milliseconds 选项所指定的值， 则这个实例会被 Sentinel 标记为主观下线。
3)：如果一个Master被标记为主观下线，则正在监视这个Master的所有 Sentinel 要以每秒一次的频率确认Master的确进入了主观下线状态。
4)：当有足够数量的 Sentinel（大于等于配置文件指定的值）在指定的时间范围内确认Master的确进入了主观下线状态， 则Master会被标记为客观下线
5)：在一般情况下， 每个 Sentinel 会以每 10 秒一次的频率向它已知的所有Master，Slave发送 INFO 命令
6)：当Master被 Sentinel 标记为客观下线时，Sentinel 向下线的 Master 的所有 Slave 发送 INFO 命令的频率会从 10 秒一次改为每秒一次
7)：若没有足够数量的 Sentinel 同意 Master 已经下线， Master 的客观下线状态就会被移除。
若 Master 重新向 Sentinel 的 PING 命令返回有效回复， Master 的主观下线状态就会被移除。
1. 配置文件sentinel.conf
port 26379
daemonize yes
sentinel monitor yunwei_redis_master 10.117.41.242 6379 1
sentinel auth-pass yunwei_redis_master test123

sentinel down-after-milliseconds yunwei_redis_master 30000
sentinel parallel-syncs yunwei_redis_master 1
sentinel failover-timeout yunwei_redis_master 900000
logfile "/data/log/redis/sentinel.log"
#上面配置文件说明如下：
#第一行指定sentinel端口号
#第二行指定sentinel为后台启动
#第三行指定Sentinel去监视一个名为 mymaster 的Master，Master的IP地址为192.168.100.211，端口号为6379，最后的2表示当有2个Sentinel检测到Master异常时才会判定其失效，即只有当2个Sentinel都判定Master失效了才会自动迁移，如果Sentinel的数量不达标，则不会执行自动故障迁移。
#第四行指定Sentinel判定Master断线的时间。（单位为毫秒，判定为主观下线SDOWN）
#第五行指定在执行故障转移时，最多可以有多少个Slave同时对新的Master进行同步。这个数字设置为1，虽然完成故障转移所需的时间会变长，但是可以保证每次只有1个Slave处于不能处理命令请求的状态
2.启动sentinel
/data/install/redis/bin/redis-sentinel   /data/install/redis/conf/sentinel.conf

日志如下：
4969:X 06 Aug 10:22:18.038 # Sentinel runid is bbad79a0f754321cb10cd88cd84637eca2003126
4969:X 06 Aug 10:22:18.038 # +monitor master yunwei_redis_master 10.117.41.242 6379 quorum 1
4969:X 06 Aug 10:22:18.040 * +slave slave 10.117.41.36:6379 10.117.41.36 6379 @ yunwei_redis_master 10.117.41.242 6379
[iyunv@yunwei_dev02 redis]# /data/install/redis/bin/redis-cli   -h 10.117.41.242 -p 26379 info Sentinel
# Sentinel
sentinel_masters:1
sentinel_tilt:0
sentinel_running_scripts:0
sentinel_scripts_queue_length:0
master0:name=yunwei_redis_master,status=ok,address=10.117.41.242:6379,slaves=1,sentinels=1
[iyunv@yunwei_dev01 conf]#  /data/install/redis/bin/redis-cli   -h 127.0.0.1 -p 6379 -a test123 shutdown

3.关闭master
日志如下
==> redis.log <==
4954:M 06 Aug 10:29:33.041 # User requested shutdown...
4954:M 06 Aug 10:29:33.041 * Saving the final RDB snapshot before exiting.
4954:M 06 Aug 10:29:33.049 * DB saved on disk
4954:M 06 Aug 10:29:33.049 * Removing the pid file.
4954:M 06 Aug 10:29:33.049 # Redis is now ready to exit, bye bye...
==> sentinel.log <==
4969:X 06 Aug 10:30:03.075 # +sdown master yunwei_redis_master 10.117.41.242 6379
4969:X 06 Aug 10:30:03.075 # +odown master yunwei_redis_master 10.117.41.242 6379 #quorum 1/1
4969:X 06 Aug 10:30:03.076 # +new-epoch 2
4969:X 06 Aug 10:30:03.076 # +try-failover master yunwei_redis_master 10.117.41.242 6379
4969:X 06 Aug 10:30:03.088 # +vote-for-leader bbad79a0f754321cb10cd88cd84637eca2003126 2
4969:X 06 Aug 10:30:03.088 # +elected-leader master yunwei_redis_master 10.117.41.242 6379
4969:X 06 Aug 10:30:03.089 # +failover-state-select-slave master yunwei_redis_master 10.117.41.242 6379
4969:X 06 Aug 10:30:03.145 # +selected-slave slave 10.117.41.36:6379 10.117.41.36 6379 @ yunwei_redis_master 10.117.41.242 6379
4969:X 06 Aug 10:30:03.145 * +failover-state-send-slaveof-noone slave 10.117.41.36:6379 10.117.41.36 6379 @ yunwei_redis_master 10.117.41.242 6379
4969:X 06 Aug 10:30:03.235 * +failover-state-wait-promotion slave 10.117.41.36:6379 10.117.41.36 6379 @ yunwei_redis_master 10.117.41.242 6379
4969:X 06 Aug 10:30:04.116 # +promoted-slave slave 10.117.41.36:6379 10.117.41.36 6379 @ yunwei_redis_master 10.117.41.242 6379
4969:X 06 Aug 10:30:04.116 # +failover-state-reconf-slaves master yunwei_redis_master 10.117.41.242 6379
4969:X 06 Aug 10:30:04.167 # +failover-end master yunwei_redis_master 10.117.41.242 6379
4969:X 06 Aug 10:30:04.167 # +switch-master yunwei_redis_master 10.117.41.242 6379 10.117.41.36 6379
4969:X 06 Aug 10:30:04.167 * +slave slave 10.117.41.242:6379 10.117.41.242 6379 @ yunwei_redis_master 10.117.41.36 6379
[iyunv@yunwei_dev02 redis]# /data/install/redis/bin/redis-cli   -h 10.117.41.242 -p 26379 info Sentinel
# Sentinel
sentinel_masters:1
sentinel_tilt:0
sentinel_running_scripts:0
sentinel_scripts_queue_length:0
master0:name=yunwei_redis_master,status=ok,address=10.117.41.36:6379,slaves=1,sentinels=1
4.现在将master重新启动，然后停止原来的slave
4981:S 06 Aug 10:36:55.164 * MASTER <-> SLAVE sync started
4981:S 06 Aug 10:36:55.164 # Error condition on socket for SYNC: Connection refused
4981:S 06 Aug 10:36:56.166 * Connecting to MASTER 10.117.41.36:6379
4981:S 06 Aug 10:36:56.166 * MASTER <-> SLAVE sync started
4981:S 06 Aug 10:36:56.166 # Error condition on socket for SYNC: Connection refused
4981:S 06 Aug 10:36:57.169 * Connecting to MASTER 10.117.41.36:6379
4981:S 06 Aug 10:36:57.169 * MASTER <-> SLAVE sync started
4981:S 06 Aug 10:36:57.169 # Error condition on socket for SYNC: Connection refused
4981:S 06 Aug 10:36:58.170 * Connecting to MASTER 10.117.41.36:6379
4981:S 06 Aug 10:36:58.171 * MASTER <-> SLAVE sync started
4981:S 06 Aug 10:36:58.171 # Error condition on socket for SYNC: Connection refused
4981:S 06 Aug 10:36:59.173 * Connecting to MASTER 10.117.41.36:6379
4981:S 06 Aug 10:36:59.173 * MASTER <-> SLAVE sync started
4981:S 06 Aug 10:36:59.174 # Error condition on socket for SYNC: Connection refused
4981:S 06 Aug 10:37:00.175 * Connecting to MASTER 10.117.41.36:6379
4981:S 06 Aug 10:37:00.175 * MASTER <-> SLAVE sync started
4981:S 06 Aug 10:37:00.175 # Error condition on socket for SYNC: Connection refused
4981:S 06 Aug 10:37:01.177 * Connecting to MASTER 10.117.41.36:6379
4981:S 06 Aug 10:37:01.177 * MASTER <-> SLAVE sync started
4981:S 06 Aug 10:37:01.177 # Error condition on socket for SYNC: Connection refused
==> sentinel.log <==
4969:X 06 Aug 10:37:01.495 # +sdown master yunwei_redis_master 10.117.41.36 6379
4969:X 06 Aug 10:37:01.495 # +odown master yunwei_redis_master 10.117.41.36 6379 #quorum 1/1
4969:X 06 Aug 10:37:01.496 # +new-epoch 3
4969:X 06 Aug 10:37:01.496 # +try-failover master yunwei_redis_master 10.117.41.36 6379
4969:X 06 Aug 10:37:01.501 # +vote-for-leader bbad79a0f754321cb10cd88cd84637eca2003126 3
4969:X 06 Aug 10:37:01.501 # +elected-leader master yunwei_redis_master 10.117.41.36 6379
4969:X 06 Aug 10:37:01.501 # +failover-state-select-slave master yunwei_redis_master 10.117.41.36 6379
4969:X 06 Aug 10:37:01.592 # +selected-slave slave 10.117.41.242:6379 10.117.41.242 6379 @ yunwei_redis_master 10.117.41.36 6379
4969:X 06 Aug 10:37:01.592 * +failover-state-send-slaveof-noone slave 10.117.41.242:6379 10.117.41.242 6379 @ yunwei_redis_master 10.117.41.36 6379
4969:X 06 Aug 10:37:01.676 * +failover-state-wait-promotion slave 10.117.41.242:6379 10.117.41.242 6379 @ yunwei_redis_master 10.117.41.36 6379
==> redis.log <==
4981:M 06 Aug 10:37:01.676 * Discarding previously cached master state.
4981:M 06 Aug 10:37:01.676 * MASTER MODE enabled (user request)
4981:M 06 Aug 10:37:01.676 # CONFIG REWRITE executed with success.
==> sentinel.log <==
4969:X 06 Aug 10:37:02.558 # +promoted-slave slave 10.117.41.242:6379 10.117.41.242 6379 @ yunwei_redis_master 10.117.41.36 6379
4969:X 06 Aug 10:37:02.558 # +failover-state-reconf-slaves master yunwei_redis_master 10.117.41.36 6379
4969:X 06 Aug 10:37:02.628 # +failover-end master yunwei_redis_master 10.117.41.36 6379
4969:X 06 Aug 10:37:02.628 # +switch-master yunwei_redis_master 10.117.41.36 6379 10.117.41.242 6379
4969:X 06 Aug 10:37:02.628 * +slave slave 10.117.41.36:6379 10.117.41.36 6379 @ yunwei_redis_master 10.117.41.242 6379
继续切换成功。