Keepalived+nginx实现双主高可用负载均衡

qder

Nginx+keepalived高可用有两种配置方案：

1、Nginx+keepalived 主从配置

这种方案，使用一个vip地址，前端使用2台机器，一台做主，一台做备，但同时只有一台机器工作，另一台备份机器在主机器不出现故障的时候，永远处于浪费状态，对于服务器不多的网站，该方案不经济实惠，所以本次不予采用。

2、Nginx+keepalived 双主配置

　　这种方案，使用两个vip地址，前端使用2台机器，互为主备，同时有两台机器工作，当其中一台机器出现故障，两台机器的请求转移到一台机器负担，非常适合于当前架构环境，故本次采用此方案对网站进行高可用架构。
　　下面我们以第二种配置演示实现双主高可用负载均衡。
　　


　　第一组
　　VIP1：192.168.36.100
　　主nginx：192.168.36.99
　　备nginx：192.168.36.86
　　
第二组
VIP2：192.168.36.100
主nginx：192.168.36.86
备nginx：192.168.36.99
web服务：
　　web1：192.168.36.215
web2：192.168.36.80
一、nginx配置（安装省略）


worker_processes  1;
events {
worker_connections  1024;
}
http {
include       mime.types;
default_type  application/octet-stream;
sendfile        on;
upstream web1 {
server 192.168.36.215:80 max_fails=3 fail_timeout=3s;
}
upstream web2 {
server 192.168.36.80:80 max_fails=3 fail_timeout=3s;
}
server {
listen         88;
server_name  localhost;
location / {
root   html;
index  index.html index.htm;
proxy_pass http://web1;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
}
}
server {
listen         99;
server_name  localhost;
location / {
root   html;
index  index.html index.htm;
proxy_pass http://web2;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
}
}
}配置并启动192.168.36.99和192.168.36.86的nginx。检查访问是否正常.　　
web1
http://192.168.36.86:88/


http://192.168.36.99:88/


web2
http://192.168.36.86:99/
　　
http://192.168.36.99:99/
二、安装、配置keepalived（安装省略）
1)192.168.36.86配置
vi /etc/keepalived/keepalived.conf




! Configuration File for keepalived
global_defs {
notification_email {
monitor@3evip.cn
#failover@firewall.loc
}
notification_email_from tianwei7518@163.com
smtp_server smtp.163.com
smtp_connect_timeout 30
router_id LVS_DEVEL
}
vrrp_script chk_nginx {
script "/etc/keepalived/chk_nginx.sh"
interval 2
weight 2
}
vrrp_instance VI_1 {
state BACKUP
interface eth0
virtual_router_id 100
priority 50
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
track_script {
chk_nginx
}
virtual_ipaddress {
192.168.36.100
}
}

vrrp_instance VI_2 {
state MASTER
interface eth0
virtual_router_id 200
priority 100
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
track_script {
chk_nginx
}   
virtual_ipaddress {
192.168.36.200
}
}2)192.168.36.99配置　　
vi /etc/keepalived/keepalived.conf



! Configuration File for keepalived
global_defs {
notification_email {
monitor@3evip.cn
#failover@firewall.loc
}
notification_email_from tianwei7518@163.com
smtp_server smtp.163.com
smtp_connect_timeout 30
router_id LVS_DEVEL
}
vrrp_script chk_nginx {
script "/etc/keepalived/chk_nginx.sh"
interval 2
weight 2
}
vrrp_instance VI_1 {
state MASTER
interface eth0
virtual_router_id 100
priority 100
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
track_script {
chk_nginx
}
virtual_ipaddress {
192.168.36.100
}
}

vrrp_instance VI_2 {
state BACKUP
interface eth0
virtual_router_id 200
priority 50
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
track_script {
chk_nginx
}   
virtual_ipaddress {
192.168.36.200
}
}3）两台机器都需要有nginx进程检查脚本，当检测不到进程的时候重启nginx，无法重启就关闭keepalived，以便让vip切换到备机。　　
vi /etc/keepalived/chk_nginx.sh


#!/bin/sh
#description: # 如果启动失败，则停止keepalived
status=$( ps -C nginx --no-heading| wc -l)
if [ "${status}" = "0" ]; then
/usr/local/nginx/sbin/nginx
status2=$( ps -C nginx --no-heading| wc -l)
if [ "${status2}" = "0" ]; then
service keepalived stop
fi
fi设置执行权限：chmod +x /etc/keepalived/chk_nginx.sh　　
　　三、启动、测试

分别启动nginx和keepalived：　　 /usr/local/nginx/sbin/nginx


　　service keepalived start


　　访问：
　　
　　web1
　　http://192.168.36.100:88/
　　http://192.168.36.200:88/
　　web2
　　http://192.168.36.100:99/
　　http://192.168.36.200:99/
　　可以正常访问。
　　我们可以试一下，关闭一个keepalived。关闭192.168.36.86上的keepalived
　　service keepalived stop


　　查看日志：
　　tail -f /var/log/messages


　　192.168.36.86日志：

Feb  7 00:39:05 slave-d Keepalived[5738]: Stopping Keepalived v1.2.15 (02/07,2015)
Feb  7 00:39:05 slave-d Keepalived_vrrp[5741]: VRRP_Instance(VI_2) sending 0 priority
Feb  7 00:39:05 slave-d Keepalived_vrrp[5741]: VRRP_Instance(VI_2) removing protocol VIPs.
Feb  7 00:39:05 slave-d Keepalived_healthcheckers[5740]: Netlink reflector reports IP 192.168.36.200 removed
Feb  7 00:39:05 slave-d avahi-daemon[1823]: Withdrawing address record for 192.168.36.200 on eth0.192.168.36.99日志：Feb  7 00:39:11 slave-c Keepalived_vrrp[25103]: VRRP_Instance(VI_2) Transition to MASTER STATE
Feb  7 00:39:12 slave-c Keepalived_vrrp[25103]: VRRP_Instance(VI_2) Entering MASTER STATE
Feb  7 00:39:12 slave-c Keepalived_vrrp[25103]: VRRP_Instance(VI_2) setting protocol VIPs.
Feb  7 00:39:12 slave-c Keepalived_healthcheckers[25102]: Netlink reflector reports IP 192.168.36.200 added
Feb  7 00:39:12 slave-c avahi-daemon[1832]: Registering new address record for 192.168.36.200 on eth0.IPv4.
Feb  7 00:39:12 slave-c Keepalived_vrrp[25103]: VRRP_Instance(VI_2) Sending gratuitous ARPs on eth0 for 192.168.36.200
Feb  7 00:39:17 slave-c Keepalived_vrrp[25103]: VRRP_Instance(VI_2) Sending gratuitous ARPs on eth0 for 192.168.36.200再开启192.168.36.86上的keepalived：　　192.168.36.86日志：

Feb  7 00:40:42 slave-d Keepalived[6004]: Starting Keepalived v1.2.15 (02/07,2015)
Feb  7 00:40:42 slave-d Keepalived[6005]: Starting Healthcheck child process, pid=6007
Feb  7 00:40:42 slave-d Keepalived[6005]: Starting VRRP child process, pid=6008
Feb  7 00:40:42 slave-d Keepalived_vrrp[6008]: Netlink reflector reports IP 192.168.36.86 added
Feb  7 00:40:42 slave-d Keepalived_vrrp[6008]: Netlink reflector reports IP fe80::20c:29ff:fe6a:66ff added
Feb  7 00:40:42 slave-d Keepalived_vrrp[6008]: Registering Kernel netlink reflector
Feb  7 00:40:42 slave-d Keepalived_vrrp[6008]: Registering Kernel netlink command channel
Feb  7 00:40:42 slave-d Keepalived_vrrp[6008]: Registering gratuitous ARP shared channel
Feb  7 00:40:42 slave-d Keepalived_vrrp[6008]: Opening file '/etc/keepalived/keepalived.conf'.
Feb  7 00:40:42 slave-d Keepalived_healthcheckers[6007]: Netlink reflector reports IP 192.168.36.86 added
Feb  7 00:40:42 slave-d Keepalived_healthcheckers[6007]: Netlink reflector reports IP fe80::20c:29ff:fe6a:66ff added
Feb  7 00:40:42 slave-d Keepalived_healthcheckers[6007]: Registering Kernel netlink reflector
Feb  7 00:40:42 slave-d Keepalived_healthcheckers[6007]: Registering Kernel netlink command channel
Feb  7 00:40:42 slave-d Keepalived_healthcheckers[6007]: Opening file '/etc/keepalived/keepalived.conf'.
Feb  7 00:40:42 slave-d Keepalived_vrrp[6008]: Configuration is using : 44182 Bytes
Feb  7 00:40:42 slave-d Keepalived_vrrp[6008]: Using LinkWatch kernel netlink reflector...
Feb  7 00:40:42 slave-d Keepalived_vrrp[6008]: VRRP_Instance(VI_1) Entering BACKUP STATE
Feb  7 00:40:42 slave-d Keepalived_vrrp[6008]: VRRP sockpool: [ifindex(2), proto(112), unicast(0), fd(10,11)]
Feb  7 00:40:42 slave-d Keepalived_healthcheckers[6007]: Configuration is using : 7257 Bytes
Feb  7 00:40:42 slave-d Keepalived_healthcheckers[6007]: Using LinkWatch kernel netlink reflector...
Feb  7 00:40:42 slave-d Keepalived_vrrp[6008]: VRRP_Instance(VI_2) Transition to MASTER STATE
Feb  7 00:40:42 slave-d Keepalived_vrrp[6008]: VRRP_Instance(VI_2) Received lower prio advert, forcing new election
Feb  7 00:40:43 slave-d Keepalived_vrrp[6008]: VRRP_Instance(VI_2) Entering MASTER STATE
Feb  7 00:40:43 slave-d Keepalived_vrrp[6008]: VRRP_Instance(VI_2) setting protocol VIPs.
Feb  7 00:40:43 slave-d avahi-daemon[1823]: Registering new address record for 192.168.36.200 on eth0.IPv4.
Feb  7 00:40:43 slave-d Keepalived_vrrp[6008]: VRRP_Instance(VI_2) Sending gratuitous ARPs on eth0 for 192.168.36.200
Feb  7 00:40:43 slave-d Keepalived_healthcheckers[6007]: Netlink reflector reports IP 192.168.36.200 added
Feb  7 00:40:48 slave-d Keepalived_vrrp[6008]: VRRP_Instance(VI_2) Sending gratuitous ARPs on eth0 for 192.168.36.200192.168.36.99日志：　　

Feb  7 00:40:47 slave-c Keepalived_vrrp[25103]: VRRP_Instance(VI_2) Received higher prio advert
Feb  7 00:40:47 slave-c Keepalived_vrrp[25103]: VRRP_Instance(VI_2) Entering BACKUP STATE
Feb  7 00:40:47 slave-c Keepalived_vrrp[25103]: VRRP_Instance(VI_2) removing protocol VIPs.
Feb  7 00:40:47 slave-c Keepalived_healthcheckers[25102]: Netlink reflector reports IP 192.168.36.200 removed
Feb  7 00:40:47 slave-c avahi-daemon[1832]: Withdrawing address record for 192.168.36.200 on eth0.关闭后依旧可以正常访问。　　关闭nginx后，可以看到nginx立即有启动。
　　


　　参考文章：
　　CentOS6.5 keepalived详解及实现Nginx服务的高可用性


　　用NginX+keepalived实现高可用的负载均衡


　　Keepalived+Nginx实现高可用和双主节点负载均衡


　　nginx+keepalived实现nginx双主高可用的负载均衡