基于keepalived、redis sentinel的高可用redis集群【修改版】
原方案地址原方案硬件
机器名
IP
作用
master
192.168.0.2
redis的master服务器
slave1
192.168.0.3
redis的slave服务器
slave2
192.168.0.4
redis的slave服务器
route1
192.168.0.5【虚拟IP：192.168.0.7】
keepalived和redis sentinel服务器，承载写redis的VIP【虚拟ip】，做写的双机热备的主master指定
route2
192.168.0.6【虚拟IP：192.168.0.8】
keepalived和redis sentinel服务器，承载读redis的VIP，做读的负载均衡和写的双机热备的master备份路由指定
详细的keepalived配置，route1
! Configuration File for keepalived
global_defs {
notification_email {
邮箱
}
notification_email_from 邮箱@bitauto.com
smtp_server 邮箱服务器地址
smtp_connect_timeout 30
router_id LVS_DEVEL
}
vrrp_instance VI_1 {
state MASTER
interface eth1
virtual_router_id 100
priority 150
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
192.168.0.7
}
}
vrrp_instance VI_2 {
state BACKUP
interface eth1
virtual_router_id 101
priority 101
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
192.168.0.8
}
}
virtual_server 192.168.0.7 6379 {
delay_loop 3
lb_algo rr
lb_kind DR
persistence_timeout 15
protocol TCP
real_server 192.168.0.4 6379 {
weight 4
notify_up /home/wind/redis_up.sh
MISC_CHECK {
misc_path "/home/wind/redischeck.py 192.168.0.4 6379"
misc_timeout 5
}
}
real_server 192.168.0.2 6379 {
weight 3
notify_up /home/wind/redis_up.sh
MISC_CHECK {
misc_path "/home/wind/redischeck.py 192.168.0.2 6379"
misc_timeout 5
}
}
real_server 192.168.0.3 6379 {
weight 3
notify_up /home/wind/redis_up.sh
MISC_CHECK {
misc_path "/home/wind/redischeck.py 192.168.0.2 6379"
misc_timeout 5
}
}
}
virtual_server 192.168.0.8 6379 {
delay_loop 3
lb_algo wrr
lb_kind DR
persistence_timeout 30
protocol TCP
real_server 192.168.0.2 6379 {
weight 8
TCP_CHECK {
connect_timeout 10
nb_get_retry 3
delay_before_retry 3
connect_port 6379
}
}
real_server 192.168.0.3 6379 {
weight 2
TCP_CHECK {
connect_timeout 10
nb_get_retry 3
delay_before_retry 3
connect_port 6379
}
}
real_server 192.168.0.4 6379 {
weight 2
TCP_CHECK {
connect_timeout 10
nb_get_retry 3
delay_before_retry 3
connect_port 6379
}
}
}
route2的配置文件
! Configuration File for keepalived
global_defs {
notification_email {
# xieqj@bitauto.com
#shanghq@bitauto.com
}
notification_email_from xieqj@bitauto.com
smtp_server mail.bitauto.com
smtp_connect_timeout 30
router_id LVS_DEVEL
}
vrrp_instance VI_1 {
state BACKUP
interface eth1
virtual_router_id 100
priority 100
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
192.168.87.89
}
#notify_master "/home/wind/redis.sh"
}
vrrp_instance VI_2 {
state MASTER
interface eth1
virtual_router_id 101
priority 151
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
192.168.87.90
}
}
virtual_server 192.168.87.89 6379 {
delay_loop 3
lb_algo rr
lb_kind DR
persistence_timeout 15
protocol TCP
real_server 192.168.87.103 6379 {
weight 4
notify_up /home/wind/redis_up.sh
MISC_CHECK {
misc_path "/home/wind/redischeck.py 192.168.87.103 6379"
misc_timeout 5
}
}
real_server 192.168.87.104 6379 {
weight 4
notify_up /home/wind/redis_up.sh
MISC_CHECK {
misc_path "/home/wind/redischeck.py 192.168.87.103 6379"
misc_timeout 5
}
}
real_server 192.168.87.105 6379 {
weight 4
notify_up /home/wind/redis_up.sh
MISC_CHECK {
misc_path "/home/wind/redischeck.py 192.168.87.103 6379"
misc_timeout 5
}
}
}
virtual_server 192.168.87.90 6379 {
delay_loop 3
lb_algo wrr
lb_kind DR
persistence_timeout 30
protocol TCP
real_server 192.168.87.104 6379 {
weight 8
TCP_CHECK {
connect_timeout 10
nb_get_retry 3
delay_before_retry 3
connect_port 6379
}
}
real_server 192.168.87.105 6379 {
weight 2
TCP_CHECK {
connect_timeout 10
nb_get_retry 3
delay_before_retry 3
connect_port 6379
}
}
}
keepalived的master的MISCH_CHECK监测脚本
#!/usr/bin/python
import sys,commands
cmd="/usr/local/bin/redis-cli -h "+sys.argv+" -p "+sys.argv+" info"#sys.argv是输入的参数,sys.argv是需要执行的命令，以后才是参数。1是ip，2是端口
str=commands.getoutput(cmd)
ismaster=-100
ismaster=str.count("role:master")#原来是使用index方法，但是找不到字符串时会报错
zero=0
if ismaster>zero:
sys.exit(0) #返回0在keepalived表示健康
else:
sys.exit(1) #返回1表示keepalived检测端口不健康
更多MISC_CHECK参考见http://bbs.ywlm.net/thread-845-1-1.html
redis维护脚本redis_up.sh用在检测到服务启动时
#!/usr/bin/env bash
/usr/local/bin/redis-cli -h $argv -p 6379 config set appendonly no
redis设置与sentinel设置无变化
需要特别注意的配置，如果没有这个配置就会发生keepalived不转发的的问题，而且是如果redis和keepalive是同一台机器，会转发，但是如果相互间独立，则realserver收不到转发包
配置master
vim /etc/sysctl.conf,添加内容如下
net.ipv4.conf.lo.arp_ignore = 1
net.ipv4.conf.lo.arp_announce = 2
net.ipv4.conf.all.arp_ignore = 1
net.ipv4.conf.all.arp_announce = 2
执行命令
sysctl -p
ip addr add 192.168.0.7/32 dev lo
ip addr add 192.168.0.8/32 dev loip add list
lo环上出现了指定的ip即可 配置slave1
vim /etc/sysctl.conf,添加内容如下
net.ipv4.conf.lo.arp_ignore = 1
net.ipv4.conf.lo.arp_announce = 2
net.ipv4.conf.all.arp_ignore = 1
net.ipv4.conf.all.arp_announce = 2
执行命令
sysctl -p
ip addr add 192.168.0.7/32 dev lo
ip addr add 192.168.0.8/32 dev loip add list 配置slave2
vim /etc/sysctl.conf,添加内容如下
net.ipv4.conf.lo.arp_ignore = 1
net.ipv4.conf.lo.arp_announce = 2
net.ipv4.conf.all.arp_ignore = 1
net.ipv4.conf.all.arp_announce = 2
执行命令
sysctl -p
ip addr add 192.168.0.7/32 dev lo
ip addr add 192.168.0.8/32 dev loip add list
页:
[1]