[转] keepalived+nginx 高可用性+负载均衡
nginx+keepalived高可用性负载均衡一、前言
最近因业务扩展，需要将当前的apache 转为nginx(web), 再在web前端放置nginx(负载均衡)。同时结合keepalived 对前端nginx实现HA。
nginx 进程基于于Master+Slave(worker)多进程模型，自身具有非常稳定的子进程管理功能。在Master进程分配模式下，Master进程永 远不进行业务处理，只是进行任务分发，从而达到Master进程的存活高可靠性，Slave(worker)进程所有的业务信号都 由主进程发出，Slave(worker)进程所有的超时任务都会被Master中止，属于非阻塞式任务模型。
Keepalived是Linux下面实现VRRP 备份路由的高可靠性运行件。基于Keepalived设计的服务模式能够真正做到主服务器和备份服务器故障时IP瞬间无缝交接。二者结合，可以构架出比较稳定的软件lb方案。
二、nginx+keepalived架构实战
采用两台服务器做nginx主备，后端采用两个realserver（可以随意扩展）。
1、 结构如下：
nginx master 172.31.1.100
nginx backup 172.31.1.135
vip 172.31.1.110
http://hiphotos.baidu.com/cong_rong520/pic/item/bcc96c523643da59377abe16.jpg
2、 安装配置
(1)、安装keepalived（在nginx的mater和backup都安装）
wgethttp://www.keepalived.org/software/keepalived-1.1.19.tar.gz
tar zxvf keepalived-1.1.19.tar.gz
cd keepalived-1.1.19
./configure --prefix=/usr/local/keepalived
make
make install
cp /usr/local/keepalived/sbin/keepalived /usr/sbin/
cp /usr/local/keepalived/etc/sysconfig/keepalived /etc/sysconfig/
cp /usr/local/keepalived/etc/rc.d/init.d/keepalived /etc/init.d/
mkdir /etc/keepalived
cd /etc/keepalived/
配置nginx master的keepalived配置文件
# cat /etc/keepalived/keepalived.conf
############################################################
vrrp_script chk_http_port {
script "/etc/keepalived/nginx_pid.sh" /* 监控脚本 */
interval 2 /* 监控时间间隔 */
weight 2
}
global_defs {
notification_email {
}
notification_email_from
smtp_server 172.31.1.101
smtp_connect_timeout 30
router_id LVS_Twioo
}
vrrp_instance VI_1 {
state MASTER /* 设置为主 */
interface eth0 /* 监控网卡 */
virtual_router_id 51 /* 这个两台服务器必须一样 */
mcast_src_ip 172.31.1.100 /* 多播的源IP，设置为本机IP，与VIP同一网卡*/
priority 100 /* 权重值 MASTRE 一定要高于 BAUCKUP */
advert_int 1 /* VRRP Multicast广播周期秒数*/
smtp_alert
authentication {
auth_type PASS /* 加密 */
auth_pass 123456 /* 加密的密码，两台服务器一定要一样，不然会出错 */
}
track_script {
chk_http_port /* 执行监控的服务 */
}
virtual_ipaddress {
172.31.1.110
}
}
############################################################
配置nginx backup的keepalived配置文件
# cat /etc/keepalived/keepalived.conf
############################################################
vrrp_script chk_http_port {
script "/etc/keepalived/nginx_pid.sh"
interval 2
weight 2
}
global_defs {
notification_email {
}
notification_email_from
smtp_server 172.31.1.101
smtp_connect_timeout 30
router_id LVS_Twioo
}
vrrp_instance VI_1 {
state BACKUP
interface eth0
virtual_router_id 51
mcast_src_ip 172.31.1.100
priority 80
advert_int 1
smtp_alert
authentication {
auth_type PASS
auth_pass 123456
}
track_script {
chk_http_port
}
virtual_ipaddress {
172.31.1.110
}
}
启动keepalived，查看虚拟IP是否绑定！
# /etc/rc.d/init.d/keepalived start
Starting keepalived: [ OK ]
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast qlen 1000
link/ether 00:26:55:1a:e5:28 brd ff:ff:ff:ff:ff:ff
inet 172.31.1.100/24 brd 172.31.1.255 scope global eth0
inet 172.31.1.110/32 scope global eth0
inet6 fe80::226:55ff:fe1a:e528/64 scope link
valid_lft forever preferred_lft forever
3: eth1: <BROADCAST,MULTICAST> mtu 1500 qdisc pfifo_fast qlen 1000
link/ether 00:26:55:1a:e5:2a brd ff:ff:ff:ff:ff:ff
4: sit0: <NOARP> mtu 1480 qdisc noop
link/sit 0.0.0.0 brd 0.0.0.0
可以看到多了一个虚拟IP
查看master keepalived日志 tailf /var/log/message
Jul 23 14:50:38 nginx Keepalived_vrrp: Registering Kernel netlink command channel
Jul 23 14:50:38 nginx Keepalived_vrrp: Registering gratutious ARP shared channel
Jul 23 14:50:38 nginx Keepalived_vrrp: Opening file '/etc/keepalived/keepalived.conf'.
Jul 23 14:50:38 nginx Keepalived_vrrp: Configuration is using : 38512 Bytes
Jul 23 14:50:38 nginx Keepalived_vrrp: Using LinkWatch kernel netlink reflector...
Jul 23 14:50:38 nginx Keepalived_vrrp: VRRP sockpool:
Jul 23 14:50:39 nginx Keepalived_vrrp: VRRP_Instance(VI_1)Transition to MASTER STATE
Jul 23 14:50:40 nginx Keepalived_vrrp: VRRP_Instance(VI_1) Entering MASTER STATE
Jul 23 14:50:40 nginx Keepalived_vrrp: VRRP_Instance(VI_1) setting protocol VIPs.
Jul 23 14:50:40 nginx Keepalived_vrrp: VRRP_Instance(VI_1) Sending gratuitous ARPs on eth0 for 172.31.1.110
Jul 23 14:50:40 nginx Keepalived_vrrp: Remote SMTP server connected.
Jul 23 14:50:40 nginx Keepalived_vrrp: Remote SMTP server connected.
Jul 23 14:50:40 nginx Keepalived_vrrp: SMTP alert successfully sent.
可以看出状态为MASTER
此时在BACKUP机上启动keepalived，再查看日志
Jul 23 14:58:30 SvrTest Keepalived_vrrp: Opening file '/etc/keepalived/keepalived.conf'.
Jul 23 14:58:30 SvrTest Keepalived_vrrp: Configuration is using : 65052 Bytes
Jul 23 14:58:30 SvrTest Keepalived_vrrp: Using LinkWatch kernel netlink reflector...
Jul 23 14:58:30 SvrTest Keepalived_vrrp: VRRP_Instance(VI_1) Entering BACKUP STATE
Jul 23 14:58:30 SvrTest Keepalived_vrrp: VRRP sockpool:
Jul 23 14:58:30 SvrTest Keepalived_vrrp: Remote SMTP server connected.
Jul 23 14:58:30 SvrTest Keepalived_vrrp: SMTP alert successfully sent.
Jul 23 14:58:30 SvrTest Keepalived_vrrp: SMTP alert successfully sent.
此时如果停止MASTER上的keepalived ,那么在BACKUP的日志会出现
Jul 23 14:59:45 SvrTest Keepalived_vrrp: VRRP_Instance(VI_1)Transition to MASTER STATE
Jul 23 14:59:46 SvrTest Keepalived_vrrp: VRRP_Instance(VI_1)Entering MASTER STATE
Jul 23 14:59:46 SvrTest Keepalived_vrrp: VRRP_Instance(VI_1) setting protocol VIPs.
Jul 23 14:59:46 SvrTest Keepalived_vrrp: VRRP_Instance(VI_1) Sending gratuitous ARPs on eth0 for 172.31.1.110
Jul 23 14:59:46 SvrTest avahi-daemon: Registering new address record for 172.31.1.110 on eth0.
Jul 23 14:59:46 SvrTest Keepalived_vrrp: Remote SMTP server connected.
Jul 23 14:59:46 SvrTest Keepalived_vrrp: SMTP alert successfully sent.
此时BACKUP接管了VIP
若再启动MASTER上的keepalived,BACKUP上的日志会出现：
Jul 23 14:56:37 SvrTest Keepalived_vrrp: VRRP_Instance(VI_1) Received higher prio advert
Jul 23 14:56:37 SvrTest Keepalived_vrrp: VRRP_Instance(VI_1) Entering BACKUP STATE
Jul 23 14:56:37 SvrTest Keepalived_vrrp: VRRP_Instance(VI_1) removing protocol VIPs.
Jul 23 14:56:37 SvrTest Keepalived_vrrp: Remote SMTP server connected.
Jul 23 14:56:37 SvrTest avahi-daemon: Withdrawing address record for 172.31.1.110 on eth0.
Jul 23 14:56:37 SvrTest Keepalived_vrrp: SMTP alert successfully sent.
则是说明有更高优先级的机器即MASTER启动，去除VIP，还给更优先级的机器
这样就实现了如果某一主机宕机，IP自动切换到备机上继续运行
################################### nginx_pid.sh 脚本
# cat /etc/keepalived/nginx_pid.sh
#!/bin/bash
NGINX_PROCESS=`ps -C nginx --no-header | wc -l`
if [ $NGINX_PROCESS -eq 0 ];then
/usr/local/nginx/sbin/nginx
sleep 3
if [ `ps -C nginx --no-header | wc -l` -eq 0 ];then
/etc/init.d/keepalived stop
fi
fi
说明：
当主nginx 宕机后，会自动切换到 从nginx上，这个时候可以配合以个报警的脚不来通告运维人员
当主修复好后，需要手动启动keepalived 程序，以便将服务重新切换回主服务器。
报警脚步我抽个时间补上。
页:
[1]