设为首页 收藏本站

运维网

 找回密码
 立即注册
查看: 1301|回复: 0

[经验分享] 基于keepalived、redis sentinel的多实例redis集群【1】---路由以及哨兵配置

[复制链接]
QQ叫紫珊

尚未签到
发表于 2015-9-4 12:08:43 | 显示全部楼层 |阅读模式
　　硬件



机器名IP作用
master192.168.0.2redis的master服务器,两个主实例
slave1192.168.0.3redis的slave服务器，两个从实例
slave2192.168.0.4redis的slave服务器，两个从实例
route1192.168.0.5【虚拟IP：192.168.0.7】keepalived和redis sentinel服务器，承载写redis的VIP【虚拟ip】，做写的双机热备的主master指定，redis哨兵的安装节点1
route2192.168.0.6【虚拟IP：192.168.0.8】keepalived和redis sentinel服务器，承载读redis的VIP，做读的负载均衡和写的双机热备的master备份路由指定，redis哨兵的安装节点2
　　route1
　　1.安装redis在route1上，安装路径/usr/local/redis/
　　2.在redis安装路径下创建scripts目录，将需要的脚本复制到此处：
　　
1RunCmd.py基础功能模块，提供redis服务超时检查
2master_config_set.py将master的save参数配置为空
3redischeck.py检查master的redis服务状态
4slave_config_set.py将slave的save参数配置为特定值
5weightchange.py调整读的redis服务在keepalived的权重




　　详细的keepalived配置，
　　

! Configuration File for keepalived
global_defs {
notification_email {
接收邮箱   
}
notification_email_from 发送邮箱
smtp_server 邮件服务器
smtp_connect_timeout 30
router_id LVS_DEVEL
}
vrrp_instance VI_1 {
state MASTER
interface eth1
lvs_sync_daemon_inteface eth1
virtual_router_id 100
priority 160
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}   
virtual_ipaddress {
192.168.0.7
}         
}
vrrp_instance VI_2 {
state BACKUP
interface eth1
lvs_sync_daemon_inteface eth1
virtual_router_id 101
priority 100
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
192.168.0.
}
}
virtual_server 192.168.0.7 6379 {
delay_loop 3
lb_algo rr
lb_kind DR
#nat_mask 255.255.255.0
persistence_timeout 15
protocol TCP
real_server 192.168.0.2 6379 {
weight 8
notify_up "/usr/local/redis/scripts/master_config_set.py 192.168.0.2 6379"
MISC_CHECK {
misc_path "/usr/local/redis/scripts/redischeck.py 192.168.0.2 6379"
misc_timeout 5
misc_dynamic
}
}
real_server 192.168.0.3 6379 {
weight 3
notify_up "/usr/local/redis/scripts/master_config_set.py 192.168.0.3 6379"
MISC_CHECK {
misc_path "/usr/local/redis/scripts/redischeck.py 192.168.0.3 6379"
misc_timeout 5
misc_dynamic
}
}
real_server 192.168.0.4 6379 {
weight 3
notify_up "/usr/local/redis/scripts/master_config_set.py 192.168.0.4 6379"
MISC_CHECK {
misc_path "/usr/local/redis/scripts/redischeck.py 192.168.0.4 6379"
misc_timeout 5
msic_dynamic
}
}
}
virtual_server 192.168.0.7 6380 {
delay_loop 3
lb_algo rr
lb_kind DR
#nat_mask 255.255.255.0
persistence_timeout 15
protocol TCP
real_server 192.168.0.2 6380 {
weight 8
notify_up "/usr/local/redis/scripts/master_config_set.py 192.168.0.2 6380"
MISC_CHECK {
misc_path "/usr/local/redis/scripts/redischeck.py 192.168.0.2 6380"
misc_timeout 5
misc_dynamic
}
}
real_server 192.168.0.3 6380 {
weight 3
notify_up "/usr/local/redis/scripts/master_config_set.py 192.168.0.3 6380"
MISC_CHECK {
misc_path "/usr/local/redis/scripts/redischeck.py 192.168.0.3 6380"
misc_timeout 5
misc_dynamic
}
}
real_server 192.168.0.4 6380 {
weight 3
notify_up "/usr/local/redis/scripts/master_config_set.py 192.168.0.4 6380"
MISC_CHECK {
misc_path "/usr/local/redis/scripts/redischeck.py 192.168.0.4 6380"
misc_timeout 5
msic_dynamic
}
}
}
virtual_server 192.168.0.8 6379 {
delay_loop 3
lb_algo wrr
lb_kind DR
persistence_timeout 30
protocol TCP
real_server 192.168.0.2 6379 {
weight 6
notify_up "/usr/local/redis/scripts/slave_config_set.py 192.168.0.2 6379"
MISC_CHECK {
misc_path "/usr/local/redis/scripts/weightchange.py 192.168.0.2 6379"
misc_timeout 5
misc_dynamic
}
}
real_server 192.168.0.3 6379 {
weight 2
notify_up "/usr/local/redis/scripts/slave_config_set.py 192.168.0.3 6379"
MISC_CHECK {
misc_path "/usr/local/redis/scripts/weightchange.py 192.168.0.3 6379"
misc_timeout 5
misc_dynamic
}
}
real_server 192.168.0.4 6379 {
weight 2
notify_up "/usr/local/redis/scripts/slave_config_set.py 192.168.0.4 6379"
MISC_CHECK {
misc_path "/usr/local/redis/scripts/weightchange.py 192.168.0.4 6379"
misc_timeout 5
misc_dynamic
}
}
}
virtual_server 192.168.0.8 6380 {
delay_loop 3
lb_algo wrr
lb_kind DR
persistence_timeout 30
protocol TCP
real_server 192.168.0.2 6380 {
weight 6
notify_up "/usr/local/redis/scripts/slave_config_set.py 192.168.0.2 6380"
MISC_CHECK {
misc_path "/usr/local/redis/scripts/weightchange.py 192.168.0.2 6380"
misc_timeout 5
misc_dynamic
}
}
real_server 192.168.0.3 6380 {
weight 2
notify_up "/usr/local/redis/scripts/slave_config_set.py 192.168.0.3 6380"
MISC_CHECK {
misc_path "/usr/local/redis/scripts/weightchange.py 192.168.0.3 6380"
misc_timeout 5
misc_dynamic
}
}
real_server 192.168.0.4 6380 {
weight 2
notify_up "/usr/local/redis/scripts/slave_config_set.py 192.168.0.4 6380"
MISC_CHECK {
misc_path "/usr/local/redis/scripts/weightchange.py 192.168.0.4 6380"
misc_timeout 5
misc_dynamic
}
}
}

route2的keepalived配置文件

! Configuration File for keepalived
global_defs {
notification_email {
接受邮箱
}
notification_email_from 发送邮箱
smtp_server 邮件服务器
smtp_connect_timeout 30
router_id LVS_DEVEL
}
vrrp_instance VI_1 {
state BACKUP
interface eth1
lvs_sync_daemon_inteface eth1
virtual_router_id 100
priority 100
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}   
virtual_ipaddress {
192.168.0.7
}     
}
vrrp_instance VI_2 {
state MASTER
interface eth1
lvs_sync_daemon_inteface eth1
virtual_router_id 101
priority 151
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
192.168.0.8
}
}
virtual_server 192.168.0.7 6379 {
delay_loop 3
lb_algo rr
lb_kind DR
persistence_timeout 15
protocol TCP
real_server 192.168.0.2 6379 {
weight 8
notify_up "/usr/local/redis/scripts/master_config_set.py 192.168.0.2 6379"
MISC_CHECK {
misc_path "/usr/local/redis/scripts/redischeck.py 192.168.0.2 6379"
misc_timeout 5
misc_dynamic
}
}
real_server 192.168.0.3 6379 {
weight 3
notify_up "/usr/local/redis/scripts/master_config_set.py 192.168.0.3 6379"
MISC_CHECK {
misc_path "/usr/local/redis/scripts/redischeck.py 192.168.0.3 6379"
misc_timeout 5
misc_dynamic
}
}
real_server 192.168.0.4 6379 {
weight 3
notify_up "/usr/local/redis/scripts/master_config_set.py 192.168.0.4 6379"
MISC_CHECK {
misc_path "/usr/local/redis/scripts/redischeck.py 192.168.0.4 6379"
misc_timeout 5
misc_dynamic
}
}
}
virtual_server 192.168.0.7 6380 {
delay_loop 3
lb_algo rr
lb_kind DR
persistence_timeout 15
protocol TCP
real_server 192.168.0.2 6380 {
weight 8
notify_up "/usr/local/redis/scripts/master_config_set.py 192.168.0.2 6380"
MISC_CHECK {
misc_path "/usr/local/redis/scripts/redischeck.py 192.168.0.2 6380"
misc_timeout 5
misc_dynamic
}
}
real_server 192.168.0.3 6380 {
weight 3
notify_up "/usr/local/redis/scripts/master_config_set.py 192.168.0.3 6380"
MISC_CHECK {
misc_path "/usr/local/redis/scripts/redischeck.py 192.168.0.3 6380"
misc_timeout 5
misc_dynamic
}
}
real_server 192.168.0.4 6380 {
weight 3
notify_up "/usr/local/redis/scripts/master_config_set.py 192.168.0.4 6380"
MISC_CHECK {
misc_path "/usr/local/redis/scripts/redischeck.py 192.168.0.4 6380"
misc_timeout 5
misc_dynamic
}
}
}
virtual_server 192.168.0.8 6379 {
delay_loop 3
lb_algo wrr
lb_kind DR
persistence_timeout 30
protocol TCP
real_server 192.168.0.2 6379 {
weight 1
notify_up "/usr/local/redis/scripts/slave_config_set.py 192.168.0.2 6379"
MISC_CHECK {
misc_path "/usr/local/redis/scripts/weightchange.py 192.168.0.2 6379"
misc_timeout 5
misc_dynamic
}
}
real_server 192.168.0.3 6379 {
weight 2
notify_up "/usr/local/redis/scripts/slave_config_set.py 192.168.0.3 6379"
MISC_CHECK {
misc_path "/usr/local/redis/scripts/weightchange.py 192.168.0.3 6379"
misc_timeout 5
misc_dynamic
}
}
real_server 192.168.0.4 6379 {
weight 2
notify_up "/usr/local/redis/scripts/slave_config_set.py 192.168.0.4 6379"
MISC_CHECK {
misc_path "/usr/local/redis/scripts/weightchange.py 192.168.0.4 6379"
misc_timeout 5
misc_dynamic
}
}
}
virtual_server 192.168.0.8 6380 {
delay_loop 3
lb_algo wrr
lb_kind DR
persistence_timeout 30
protocol TCP
real_server 192.168.0.2 6380 {
weight 1
notify_up "/usr/local/redis/scripts/slave_config_set.py 192.168.0.2 6380"
MISC_CHECK {
misc_path "/usr/local/redis/scripts/weightchange.py 192.168.0.2 6380"
misc_timeout 5
misc_dynamic
}
}
real_server 192.168.0.3 6380 {
weight 2
notify_up "/usr/local/redis/scripts/slave_config_set.py 192.168.0.3 6380"
MISC_CHECK {
misc_path "/usr/local/redis/scripts/weightchange.py 192.168.0.3 6380"
misc_timeout 5
misc_dynamic
}
}
real_server 192.168.0.4 6380 {
weight 2
notify_up "/usr/local/redis/scripts/slave_config_set.py 192.168.0.4 6380"
MISC_CHECK {
misc_path "/usr/local/redis/scripts/weightchange.py 192.168.0.4 6380"
misc_timeout 5
misc_dynamic
}
}
}
在keepalived使用的脚本　　
　　RunCmd.py
　　

#!/usr/bin/python
import os;  
import sys;  
import time;  
import fcntl;  
import select;  
import signal;  
import commands;  
import subprocess;
class RunCmd:  
def __init__(self):  
pass;   
def Run(self,ip,port, nTimeOut = 0, nIntervalTime = 0.1):  
lsCmd=['/usr/local/redis/bin/redis-cli','-h',ip,'-p',port,'ping']
oProc = subprocess.Popen(lsCmd, stdout =subprocess.PIPE, stderr = subprocess.PIPE)  
istimeout=False
nStartTime = time.time()         
while True:  
time.sleep(nIntervalTime)   
print("1:")
print(oProc.poll())
if None != oProc.poll():  
break;            
if (nTimeOut > 0) and (time.time() - nStartTime) > nTimeOut:
istimeout=True                 
break;
print("2:")
print(istimeout)
if istimeout:
print(oProc.poll())
if None == oProc.poll():  
self.KillAll(oProc.pid)  
print("3:")
print(istimeout)
return istimeout
def KillAll(self, nKillPid, nKillSignal = signal.SIGKILL):  
nRet, strOutput = commands.getstatusoutput("kill "+str(nKillPid));#as root run         
return (True, strOutput)  


master_config_set.py脚本　　
　　

#!/usr/bin/python
from RunCmd import RunCmd
import sys,commands
oCmd = RunCmd();  
istimeout = oCmd.Run(sys.argv[1],sys.argv[2], 0.1)
if not istimeout:
cmd="/usr/local/redis/bin/redis-cli -h "+sys.argv[1]+" -p "+sys.argv[2]+" info"
str=commands.getoutput(cmd)
ismaster=str.count("role:master")
zero=0
if ismaster>zero:
t=commands.getoutput("/usr/local/redis/bin/redis-cli -h "+sys.argv[1]+" -p "+sys.argv[2]+" config set save \"\"")
print t



slave_config_set.py　　
　　

#!/usr/bin/python
from RunCmd import RunCmd
import sys,commands
oCmd = RunCmd();  
istimeout = oCmd.Run(sys.argv[1],sys.argv[2], 0.1)
if not istimeout:
cmd="/usr/local/redis/bin/redis-cli -h "+sys.argv[1]+" -p "+sys.argv[2]+" info"
str=commands.getoutput(cmd)
isslave=str.count("role:slave")
zero=0
if isslave>zero:
t=commands.getoutput("/usr/local/redis/bin/redis-cli -h "+sys.argv[1]+" -p "+sys.argv[2]+" config set save \"90 1 300 10 60 1000\"")
print t



redischeck.py　　
　　

#!/usr/bin/python
from RunCmd import RunCmd
import sys,commands
oCmd = RunCmd();  
istimeout = oCmd.Run(sys.argv[1],sys.argv[2], 0.1)
if not istimeout:
cmd="/usr/local/redis/bin/redis-cli -h "+sys.argv[1]+" -p "+sys.argv[2]+" info"
str=commands.getoutput(cmd)
ismaster=str.count("role:master")
zero=0
if ismaster>zero:
sys.exit(0)
else:
sys.exit(1)
else:
sys.exit(1)



weightchange.py　　
　　

#!/usr/bin/python
from RunCmd import RunCmd
import sys,commands
oCmd = RunCmd();  
istimeout = oCmd.Run(sys.argv[1],sys.argv[2], 0.1)
if not istimeout:
result=1
cmd="/usr/local/redis/bin/redis-cli -h "+sys.argv[1]+" -p "+sys.argv[2]+" ping"
strping=commands.getoutput(cmd)
zero=0
ispong=-100
ispong=strping.count("PONG")
if ispong>zero:
result=0
if result>zero:
sys.exit(1)
else:
cmdmaster="/usr/local/redis/bin/redis-cli -h "+sys.argv[1]+" -p "+sys.argv[2]+" info"
str=commands.getoutput(cmdmaster)
ismaster=-100
ismaster=str.count("role:master")
if ismaster>zero:
sys.exit(3)
else:
sys.exit(10)
else:
sys.exit(1)



redis的哨兵的配置文件sentinel.conf　　
　　

# Example sentinel.conf
# port <sentinel-port>
# The port that this sentinel instance will run on
port 26379
# sentinel monitor <master-name> <ip> <redis-port> <quorum>
#
# Tells Sentinel to monitor this slave, and to consider it in O_DOWN
# (Objectively Down) state only if at least <quorum> sentinels agree.
#
# Note: master name should not include special characters or spaces.
# The valid charset is A-z 0-9 and the three characters &quot;.-_&quot;.
sentinel monitor mymaster 192.168.0.2 6379 2
sentinel monitor mymaster6380 192.168.0.2 6380 2
# sentinel auth-pass <master-name> <password>
#
# Set the password to use to authenticate with the master and slaves.
# Useful if there is a password set in the Redis instances to monitor.
#
# Note that the master password is also used for slaves, so it is not
# possible to set a different password in masters and slaves instances
# if you want to be able to monitor these instances with Sentinel.
#
# However you can have Redis instances without the authentication enabled
# mixed with Redis instances requiring the authentication (as long as the
# password set is the same for all the instances requiring the password) as
# the AUTH command will have no effect in Redis instances with authentication
# switched off.
#
# Example:
#
# sentinel auth-pass mymaster MySUPER--secret-0123passw0rd
# sentinel down-after-milliseconds <master-name> <milliseconds>
#
# Number of milliseconds the master (or any attached slave or sentinel) should
# be unreachable (as in, not acceptable reply to PING, continuously, for the
# specified period) in order to consider it in S_DOWN state (Subjectively
# Down).
#
# Default is 30 seconds.
sentinel down-after-milliseconds mymaster 3800
sentinel down-after-milliseconds mymaster6380 3800
# sentinel can-failover <master-name> <yes|no>
#
# Specify if this Sentinel can start the failover for this master.
sentinel can-failover mymaster yes
sentinel can-failover mymaster6380 yes
# sentinel parallel-syncs <master-name> <numslaves>
#
# How many slaves we can reconfigure to point to the new slave simultaneously
# during the failover. Use a low number if you use the slaves to serve query
# to avoid that all the slaves will be unreachable at about the same
# time while performing the synchronization with the master.
sentinel parallel-syncs mymaster 1
sentinel parallel-syncs mymaster6380 1
# sentinel failover-timeout <master-name> <milliseconds>
#
# Specifies the failover timeout in milliseconds. When this time has elapsed
# without any progress in the failover process, it is considered concluded by
# the sentinel even if not all the attached slaves were correctly configured
# to replicate with the new master (however a &quot;best effort&quot; SLAVEOF command
# is sent to all the slaves before).
#
# Also when 25% of this time has elapsed without any advancement, and there
# is a leader switch (the sentinel did not started the failover but is now
# elected as leader), the sentinel will continue the failover doing a
# &quot;takeover&quot;.
#
# Default is 15 minutes.
sentinel failover-timeout mymaster 90000
sentinel failover-timeout mymaster6380 90000
# SCRIPTS EXECUTION
#
# sentinel notification-script and sentinel reconfig-script are used in order
# to configure scripts that are called to notify the system administrator
# or to reconfigure clients after a failover. The scripts are executed
# with the following rules for error handling:
#
# If script exists with &quot;1&quot; the execution is retried later (up to a maximum
# number of times currently set to 10).
#
# If script exists with &quot;2&quot; (or an higher value) the script execution is
# not retried.
#
# If script terminates because it receives a signal the behavior is the same
# as exit code 1.
#
# A script has a maximum running time of 60 seconds. After this limit is
# reached the script is terminated with a SIGKILL and the execution retried.
# NOTIFICATION SCRIPT
#
# sentinel notification-script <master-name> <script-path>
#
# Call the specified notification script for any sentienl event that is
# generated in the WARNING level (for instance -sdown, -odown, and so forth).
# This script should notify the system administrator via email, SMS, or any
# other messaging system, that there is something wrong with the monitored
# Redis systems.
#
# The script is called with just two arguments: the first is the event type
# and the second the event description.
#
# The script must exist and be executable in order for sentinel to start if
# this option is provided.
#
# Example:
#
# sentinel notification-script mymaster /var/redis/notify.sh
# CLIENTS RECONFIGURATION SCRIPT
#
# sentinel client-reconfig-script <master-name> <script-path>
#
# When the failover starts, ends, or is aborted, a script can be called in
# order to perform application-specific tasks to notify the clients that the
# configuration has changed and the master is at a different address.
#
# The script is called in the following cases:
#
# Failover started (a slave is already promoted)
# Failover finished (all the additional slaves already reconfigured)
# Failover aborted (in that case the script was previously called when the
#                   failover started, and now gets called again with swapped
#                   addresses).
#
# The following arguments are passed to the script:
#
# <master-name> <role> <state> <from-ip> <from-port> <to-ip> <to-port>
#
# <state> is &quot;start&quot;, &quot;end&quot; or &quot;abort&quot;
# <role> is either &quot;leader&quot; or &quot;observer&quot;
#
# The arguments from-ip, from-port, to-ip, to-port are used to communicate
# the old address of the master and the new address of the elected slave
# (now a master) in the case state is &quot;start&quot; or &quot;end&quot;.
#
# For abort instead the &quot;from&quot; is the address of the promoted slave and
# &quot;to&quot; is the address of the original master address, since the failover
# was aborted.
#
# This script should be resistant to multiple invocations.
#
# Example:
#
# sentinel client-reconfig-script mymaster /var/redis/reconfig.sh



在两个route上修改/etc/sysctl.conf文件



net.ipv4.ip_forward=1#转发开启

　　执行sysctl -p让文件起效
　　有防火墙需要设置防火墙转发


　　
　　

运维网声明 1、欢迎大家加入本站运维交流群:群②:261659950 群⑤:202807635 群⑦870801961 群⑧679858003
2、本站所有主题由该帖子作者发表，该帖子作者与运维网享有帖子相关版权
3、所有作品的著作权均归原作者享有，请您和我们一样尊重他人的著作权等合法权益。如果您对作品感到满意，请购买正版
4、禁止制作、复制、发布和传播具有反动、淫秽、色情、暴力、凶杀等内容的信息，一经发现立即删除。若您因此触犯法律，一切后果自负，我们对此不承担任何责任
5、所有资源均系网友上传或者通过网络收集，我们仅提供一个展示、介绍、观摩学习的平台，我们不对其内容的准确性、可靠性、正当性、安全性、合法性等负责，亦不承担任何法律责任
6、所有作品仅供您个人学习、研究或欣赏，不得用于商业或者其他用途，否则，一切后果均由您自己承担，我们对此不承担任何法律责任
7、如涉及侵犯版权等问题，请您及时通知我们，我们将立即采取措施予以解决
8、联系人Email：admin@iyunv.com 网址：www.yunweiku.com

所有资源均系网友上传或者通过网络收集,我们仅提供一个展示、介绍、观摩学习的平台,我们不对其承担任何法律责任,如涉及侵犯版权等问题,请您及时通知我们,我们将立即处理,联系人Email:kefu@iyunv.com,QQ:1061981298 本贴地址：https://www.yunweiku.com/thread-109356-1-1.html 上篇帖子： Nginx+Keepalived主主负载均衡服务器 下篇帖子： Atlas+Keepalived系列一：安装Atlas：
回复

使用道具 举报

返回列表
高级模式
B Color Image Link Quote Code Smilies
您需要登录后才可以回帖 登录 | 立即注册

本版积分规则

扫码加入运维网微信交流群X

扫码加入运维网微信交流群

扫描二维码加入运维网微信交流群，最新一手资源尽在官方微信交流群！快快加入我们吧...

扫描微信二维码查看详情

客服 E-mail:iyunvcom@gmail.com

本站由青云提供云计算服务

运维网--中国最专业的运维工程师交流社区

豫ICP备20007574号-1 Copyright © 2012-2025

关于我们

发展历程

联系我们

关注我们

广告服务

广告案例

隐私条款

免责声明

用户登录

用户注册

版主守则

申请版主

手机版

金币获取

帮助中心

每日签到

Good Good

Study

Day Day

UP


客服E-mail:kefu@iyunv.com 客服QQ：1061981298


QQ群⑦：运维网交流群⑦ QQ群⑧：运维网交流群⑧ k8s群：运维网kubernetes交流群


提醒：禁止发布任何违反国家法律、法规的言论与图片等内容；本站内容均来自个人观点与网络等信息，非本站认同之观点.


本站大部分资源是网友从网上搜集分享而来，其版权均归原作者及其网站所有，我们尊重他人的合法权益，如有内容侵犯您的合法权益，请及时与我们联系进行核实删除！



合作伙伴: 青云cloud

快速回复 返回顶部 返回列表