Memcached集群介绍
现实生产环境中,我们要实现memcached主从复制避免单点故障风险,往往需要部署高可用群集。这里我们使用magent代理memcached实现主从备份,从而来保证缓存数据的完整性。工作原理如下:
1)数据在到达memcached服务器前会经过magent,每次写数据到主memcached服务器,magent会同时写到从 服务器 上,并且主从服务器写的算法一样。
2)当主memcached服务器宕掉之后,magent会向从memcached服务器中读取数据
3)当memcached恢复后,magent将重新从主memcached中读取数据,此时由于主memcached刚恢复正常,所以在宕掉期间的数据是无法获取的,这也是magent的缺点
再通过keepalived设置VIP地址,提供给Memcached API客户端以及magent连接访问。
1)memcached主从复制这种架构,在程序连接时不知道应该连接哪个主服务器,所以需要用keepalived在前端加VIP地址实现高可用;
2)keepalived不断检测memcached主服务器的11211端口,如果检测到memcached服务发生宕机或者死机等情况,就会将VIP从主服务器移至从服务器,从而实现高可用。
Memcached高可用架构图:
实验环境介绍:
服务器主机
IP地址
需要安装的软件包
操作系统
master
192.168.30.55
VIP:192.168.30.16
libevent-2.1.8-stable.tar.gz,memcached-1.5.6.tar.gz,
magent-0.5.tar.gz,keepalived
CentOS7
backup
192.168.30.10
VIP:192.168.30.16
libevent-2.1.8-stable.tar.gz,memcached-1.5.6.tar.gz,
keepalived
CentOS7
client
192.168.30.15
telnet
RedHat6.5
百度网盘免费提供软件包:
libevent-2.1.8-stable.tar.gz:https://pan.baidu.com/s/1TBVEyndXVHcMbxst-kpCuA
memcached-1.5.6.tar.gz:https://pan.baidu.com/s/1oO_mC6J3SV6PLVepjmi2eQ
magent-0.5.tar.gz:https://pan.baidu.com/s/1xoT3Px14RHu6QE-66nLtMQ
实验部署:
首先所有主机关闭防火墙,及selinux
systemctl stop firewalld.service(CentOS7上)或者service iptables stop(RedHat6.5上)
setenforce 0
配置memcached主缓存节点和从缓存节点(主,从基本步骤相同,个别不同步骤下面会单独说明)
tar xvf memcached-1.5.6.tar.gz -C /opt/
tar xvf libevent-2.1.8-stable.tar.gz -C /opt/ #事件通知库
yum install gcc gcc-c++ make -y
编译安装libvent
cd /opt/libevent-2.1.8-stable
./configure --prefix=/usr #指定工作目录
make && make install
cd /opt/memcached-1.5.6
./configure \
--prefix=/usr/local/memcached \ #指定工作目录
--with-libevent=/usr #指定libevent的安装路径
make && make install
ln -s /usr/local/memcached/bin/* /usr/local/bin/ #让系统识别命令
mkdir /opt/magent
cd ~/
tar xvf magent-0.5.tar.gz -C /opt/magent/
ketama.c
magent.c
ketama.h
Makefile
cd /opt/magent/
编辑配置文件
vim ketama.h
#ifndef SSIZE_MAX #将原来的参数修改为SSIZE_MAX
#define SSIZE_MAX 32767 #将原来的参数修改为SSIZE_MAX 32767
修改Makefile
vim Makefile
LIBS = -levent -lm #在后面加上-lm
编译(完成后会生成一个magent的可执行文件)
make
复制mgent命令到系统管理中,便于使用
cp magent /usr/bin/ #将magent可执行文件加入usr可执行命令中
[root@localhost magent]#scp magent 192.168.30.10:/usr/bin
The authenticity of host '192.168.30.10 (192.168.30.10)' can't be established.
ECDSA key fingerprint is SHA256:/5yAWf89joXesnoqczcLtVetMK2DIseKpZOcWkC+7V4.
ECDSA key fingerprint is MD5:a1:a4:ef:ff:84:46:0b:0c:fb:1d:de:56:4e:36:89:15.
Are you sure you want to continue connecting (yes/no)?yes #输入yes,同意复制
Warning: Permanently added '192.168.30.10' (ECDSA) to the list of known hosts.
root@192.168.30.10's password:#输入从服务器的root密码
magent 100% 112KB 4.9MB/s 00:00
memcached -d -m 32m -p 11211 -u root
netstat -antp | grep 11211
tcp 0 0 0.0.0.0:11211 0.0.0.0:* LISTEN 57782/memcached
tcp6 0 0 :::11211 :::* LISTEN 57782/memcached
接着在主从服务器上安装配置keepalived
yum install keepalived -y
vim /etc/keepalived/keepalived.conf
其余设置项删除掉,就留下面的设置
! Configuration File for keepalived
global_defs {
notification_email {
acassen@firewall.loc
failover@firewall.loc
sysadmin@firewall.loc
}
notification_email_from Alexandre.Cassen@firewall.loc
smtp_server 192.168.200.1
smtp_connect_timeout 30
router_id MAGENT_HA #指定router_id, 从服务器为MAGENT_HB
}
vrrp_script magent { #定义函数
script "/opt/shell/magent.sh"
interval 2
}
vrrp_instance VI_1 {
state MASTER #从服务器为BACKUP
interface ens33 #本地网卡名称
virtual_router_id 51 #从服务器id不同
priority 100 #从服务器优先级要小于主的
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
track_script { #调用函数
magent
}
virtual_ipaddress {
192.168.30.16 #设置VIP
}
[root@localhost magent]# vim /etc/keepalived/keepalived.conf
! Configuration File for keepalived
global_defs {
notification_email {
acassen@firewall.loc
failover@firewall.loc
sysadmin@firewall.loc
}
notification_email_from Alexandre.Cassen@firewall.loc
smtp_server 192.168.200.1
smtp_connect_timeout 30
router_id MAGENT_HA
}
vrrp_script magent {
script "/opt/shell/magent.sh"
interval 2
}
vrrp_instance VI_1 {
stateMASTER
interface ens33
virtual_router_id 51
priority100
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
track_script {
magent
}
virtual_ipaddress {
192.168.30.16
}
}
[root@localhost memcached-1.5.6]# vim /etc/keepalived/keepalived.conf
! Configuration File for keepalived
global_defs {
notification_email {
acassen@firewall.loc
failover@firewall.loc
sysadmin@firewall.loc
}
notification_email_from Alexandre.Cassen@firewall.loc
smtp_server 192.168.200.1
smtp_connect_timeout 30
router_idMAGENT_HB
}
vrrp_script magent {
script "/opt/shell/magent.sh"
interval 2
}
vrrp_instance VI_1 {
stateBACKUP
interfaceens33
virtual_router_id 52
priority 99
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
track_script {
magent
}
virtual_ipaddress {
192.168.30.16
}
}
在主服务器上建立/opt/shell目录,创建magent.sh脚本
mkdir /opt/shell
vim /opt/shell/magent.sh
#!/bin/bash
K=`ps -ef | grep keepalived | grep -v grep | wc -l`
if [ $K -gt 0 ]; then
magent -u root -n 51200 -l 192.168.30.16 -p 12000 -s 192.168.30.55:11211 -b 192.168.30.10:11211
else
pkill -9 magent
fi
-n 51200 //定义用户最大连接数
-l 192.168.175.188 //指定虚拟IP
-p 12000 //指定端口号
-s //指定主缓存服务器
-b //指定从缓存服务器
chmod +x magent.sh
启动keepalived服务
systemctl start keepalived.service
查看12000端口的进程(magent)是否运行
netstat -antp | grep 12000
tcp 0 0 192.168.30.16:12000 0.0.0.0:* LISTEN 22180/magent
查看系统公共日志文件/var/log/messages ,检测keepalived功能是否启用
vim /var/log/messages
Sep 26 12:20:42 localhost Keepalived_vrrp[24260]: VRRP_Instance(VI_1) Transition to MASTER STATE
Sep 26 12:20:43 localhost Keepalived_vrrp[24260]: VRRP_Instance(VI_1)Entering MASTER STATE
Sep 26 12:20:43 localhost Keepalived_vrrp[24260]: VRRP_Instance(VI_1) setting protocol VIPs.
Sep 26 12:20:43 localhost Keepalived_vrrp[24260]: Sending gratuitous ARP on ens33 for 192.168.30.16
Sep 26 12:20:43 localhost Keepalived_vrrp[24260]: VRRP_Instance(VI_1) Sending/queueing gratuitous ARPs on ens33 for 192.168.30.16
同样在从服务器上建立/opt/shell目录,创建magent.sh脚本
mkdir /opt/shell
vim /opt/shell/magent.sh
#!/bin/bash
K=`ip addr | grep 192.168.30.16 | grep -v grep | wc -l`
if [ $K -gt 0 ]; then
magent -u root -n 51200 -l 192.168.30.16 -p 12000 -s 192.168.30.55:11211 -b 192.168.30.10:11211
else
pkill -9 magent
fi
chmod +x magent.sh
启动keepalived服务
systemctl start keepalived.service
查看12000端口的进程(magent)是否运行
netstat -antp | grep 12000
tcp 0 0 192.168.30.16:12000 0.0.0.0:* LISTEN 22180/magent
查看系统公共日志文件/var/log/messages ,检测keepalived功能是否启用
vim /var/log/messages
Sep 26 12:32:50 localhost Keepalived_vrrp[22129]: VRRP_Instance(VI_1) Entering BACKUP STATE
Sep 26 12:32:50 localhost Keepalived_vrrp[22129]: VRRP sockpool: [ifindex(2), proto(112), unicast(0), fd(10,11)]
Sep 26 12:32:51 localhost Keepalived_vrrp[22129]: VRRP_Script(magent) succeeded
在client上测试
yum -y install telnet
连接VIP,端口12000进行远程登录memcached群集,测试主从复制同步,添加键值username
[root@localhost ~]# telnet 192.168.30.16 12000
Trying 192.168.30.16...
Connected to 192.168.30.16.
Escape character is '^]'.
add username 0 0 6
abcdef
STORED
quit
Connection closed by foreign host.
连接master主服务器,查看username,同步成功
[root@localhost ~]#telnet 192.168.30.55 11211
Trying 192.168.30.55...
Connected to 192.168.30.55.
Escape character is '^]'.
get username
VALUE username 0 6
abcdef
END
quit
Connection closed by foreign host.
连接backup从服务器,查看username,同步成功
[root@localhost ~]#telnet 192.168.30.10 11211
Trying 192.168.30.10...
Connected to 192.168.30.10.
Escape character is '^]'.
get username
VALUE username 0 6
abcdef
END
quit
Connection closed by foreign host.
测试故障转移
在client端通过VIP连接群集,检测群集仍然正常运行
运维网声明
1、欢迎大家加入本站运维交流群:群②:261659950 群⑤:202807635 群⑦870801961 群⑧679858003
2、本站所有主题由该帖子作者发表,该帖子作者与运维网 享有帖子相关版权
3、所有作品的著作权均归原作者享有,请您和我们一样尊重他人的著作权等合法权益。如果您对作品感到满意,请购买正版
4、禁止制作、复制、发布和传播具有反动、淫秽、色情、暴力、凶杀等内容的信息,一经发现立即删除。若您因此触犯法律,一切后果自负,我们对此不承担任何责任
5、所有资源均系网友上传或者通过网络收集,我们仅提供一个展示、介绍、观摩学习的平台,我们不对其内容的准确性、可靠性、正当性、安全性、合法性等负责,亦不承担任何法律责任
6、所有作品仅供您个人学习、研究或欣赏,不得用于商业或者其他用途,否则,一切后果均由您自己承担,我们对此不承担任何法律责任
7、如涉及侵犯版权等问题,请您及时通知我们,我们将立即采取措施予以解决
8、联系人Email:admin@iyunv.com 网址:www.yunweiku.com