Nginx高可用群集架构（Keepalived+nginx+ipvsadm）
案例说明为了解决Nginx的单点故障问题，使用Keepalived实现双机热备，Keepalived 是一种高性能的服务器高可用或热备解决方案， Keepalived 可以用来防止服务器单点故障的发生，通过配合 Nginx 可以实现 web 前端服务的高可用。使用ipvsadm轮询规则实现负载均衡。
案例拓扑
http://i2.运维网.com/images/blog/201810/14/81226f40caea9ea324abef020d4e1987.png
案例环境
主机
操作系统
ip地址
主要软件
keepalivedMaster
Centos7
192.168.100.130
keepalived、ipvsadm
keepalivedBackup
Centos7
192.168.100.140
keepalived、ipvsadm
nginx1
Centos7
192.168.100.100
nginx-1.14.0
nginx2
Centos7
192.168.100.110
nginx-1.14.0
客户机
win7
192.168.100.55
网页浏览器
案例操作
一、安装两台nginx服务器
1、安装环境包
yum install gcc gcc-c++ pcre pcre-devel zlib-devel -y
2、下载nginx包
cd /usr/local/
wget http://nginx.org/download/nginx-1.14.0.tar.gz
3、解压编译
tar zxfnginx-1.14.0.tar.gz
cdnginx-1.14.0
useradd -M -s /sbin/nologin nginx
./configure \
--prefix=/usr/local/nginx \
--user=nginx \
--group=nginx \
--with-http_stub_status_module
make && make install
4、优化
ln -s /usr/local/nginx/sbin/nginx /usr/local/sbin/
5、修改网页站点
# cd /usr/local/nginx/html/
http://i2.运维网.com/images/blog/201810/14/69db76bcefd1b6e83d3ccfe248b7b909.png
http://i2.运维网.com/images/blog/201810/14/35fcb48a36e63cabc58dae2270ea645b.png
6、关闭防火墙、开启Nginx服务
# systemctl stop firewalld.service
# setenforce 0
# nginx
# netstat -ntap | grep nginx#检测nginx端口有没有开启
tcp 0 0 0.0.0.0:80 0.0.0.0:* LISTEN 40686/nginx: master
7、客户机测试访问
http://i2.运维网.com/images/blog/201810/14/30b37359ad21cafb321c5c55cf9b8a82.png
二、配置ipvsamd调度服务器
1、下载ipvsadm和keepaliveed
# yum install keepalived ipvsadm -y
2、关闭防火墙，开启路由转发功能
# systemctl stop firewalld.service
# systemctl disable firewalld.service
# setenforce 0
# vim /etc/sysctl.conf #添加如下四行内容
net.ipv4.ip_forward=1
net.ipv4.conf.all.send_redirects = 0
net.ipv4.conf.default.send_redirects = 0
net.ipv4.conf.ens33.send_redirects =
sysctl -p #刷新即时生效
3、复制ens33网卡，设置虚拟ip
# cd /etc/sysconfig/network-scripts/
# cp ifcfg-ens33 ifcfg-ens33:0
# vim ifcfg-ens33:0 #删除原有内容，添加如下4行
DEVICE=ens33:0
ONBOOT=yes
IPADDR=192.168.100.10
NETMASK=255.255.255.0
# ifup ens33:0#启动ens33:0网卡
4、配置ipvsadm启动脚本
vim /etc/init.d/dr.sh
#!/bin/bash
GW=192.168.100.1
#网关
VIP=192.168.100.10
#虚拟ip
RIP1=192.168.100.100
#nginx1服务器ip
RIP2=192.168.100.110
#nginx2服务器ip
case "$1" in
start)
/sbin/ipvsadm --save > /etc/sysconfig/ipvsadm
systemctl start ipvsadm
/sbin/ifconfig ens33:0 $VIP broadcast $VIP netmask 255.255.255.255 broadcast $VIP up
/sbin/route add -host $VIP dev ens33:0
/sbin/ipvsadm -A -t $VIP:80 -s rr
/sbin/ipvsadm -a -t $VIP:80 -r $RIP1:80 -g
/sbin/ipvsadm -a -t $VIP:80 -r $RIP2:80 -g
echo "ipvsadm starting --------------------"
;;
stop)
/sbin/ipvsadm -C
systemctl stop ipvsadm
ifconfig ens33:0 down
route del $VIP
echo "ipvsamd stoped----------------------"
;;
status)
if [ ! -e /var/lock/subsys/ipvsadm ];then
echo "ipvsadm stoped---------------"
exit 1
else
echo "ipvsamd Runing ---------"
fi
;;
*)
echo "Usage: $0 {start|stop|status}"
exit 1
esac
exit 0
5、添加脚本权限，启动ipvsadm
# chmod +x /etc/init.d/dr.sh
# service dr.sh start
三、回到Nginx节点服务器配置虚拟ip
1、复制ifcfg-lo网卡，进行修改
# cd /etc/sysconfig/network-scripts/
# cp ifcfg-lo ifcfg-lo:0
# vim ifcfg-lo:0 #删除原本内容，添加如下4行
DEVICE=lo:0
IPADDR=192.168.100.10
NETMASK=255.255.255.0
ONBOOT=yes
2、设置ifcfg-lo:0启动脚本
# cd /etc/init.d/
# vim lo.sh
#!/bin/bash
VIP=192.168.100.10
#虚拟ip
case "$1" in
start)
ifconfig lo:0 $VIP netmask 255.255.255.255 broadcast $VIP
/sbin/route add -host $VIP dev lo:0
echo "1" >/proc/sys/net/ipv4/conf/lo/arp_ignore
echo "2" >/proc/sys/net/ipv4/conf/lo/arp_announce
echo "1" >/proc/sys/net/ipv4/conf/all/arp_ignore
echo "2" >/proc/sys/net/ipv4/conf/all/arp_announce
sysctl -p >/dev/null 2>&1
echo "RealServer Start OK "
;;
stop)
ifconfig lo:0 down
route del $VIP /dev/null 2>&1
echo "0" >/proc/sys/net/ipv4/conf/lo/arp_ignore
echo "0" >/proc/sys/net/ipv4/conf/lo/arp_announce
echo "0" >/proc/sys/net/ipv4/conf/all/arp_ignore
echo "0" >/proc/sys/net/ipv4/conf/all/arp_announce
echo "RealServer Stopd"
;;
*)
echo "Usage: $0 {start|stop}"
exit 1
esac
exit 0
3、添加脚本权限，启动lo:0网卡
# chmod +x lo.sh
# service lo.sh start
# ifup lo:0
4、本地再次自测站点
# firefox "http://127.0.0.1/" &
http://i2.运维网.com/images/blog/201810/14/3c28fc5eb9649d5151599dc007ccb9da.png
http://i2.运维网.com/images/blog/201810/14/e901c0c1d1a302e2d267b617726be28d.png
四、配置Keepalived
vim /etc/keepalived/keepalived.conf （主从服务器配置内容如下）
! Configuration File for keepalived
global_defs {
notification_email {
acassen@firewall.loc
failover@firewall.loc
sysadmin@firewall.loc
}
notification_email_from Alexandre.Cassen@firewall.loc
smtp_server 127.0.0.1 #指向本地地址
smtp_connect_timeout 30
router_id LVS_01 #从服务器LVS_02加以区分
vrrp_skip_check_adv_addr
vrrp_strict
vrrp_garp_interval 0
vrrp_gna_interval 0
}
vrrp_instance VI_1 {
state MASTER #从服务器为BACKUP
interface ens33 #绑定真实网卡为ens33
virtual_router_id 51
priority 100
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress { #虚拟ip地址
192.168.100.10
}
}
virtual_server 192.168.100.10 80 { #虚拟ip地址，端口号80
delay_loop 6
lb_algo rr
lb_kind NAT
persistence_timeout 50
protocol TCP
real_server 192.168.100.100 80 { #nginx1ip地址，端口80
weight 1
TCP_CHECK { #改为TCP_CHECK,删除原有的url8行
connect_port 80
connect_timeout 3
nb_get_retry 3
delay_before_retry 3
}
}
real_server 192.168.100.110 443 { #nginx2ip地址，端口80
weight 1
TCP_CHECK { #改为TCP_CHECK，删除原有的url8行
connect_port 80
connect_timeout 3
nb_get_retry 3
delay_before_retry 3
}
}
}
2、开启keepalived服务
# systemctl start keepalived.service
# systemctl status keepalived.service #查看状态
http://i2.运维网.com/images/blog/201810/14/aecbb3e5df0d33f8a852e5331a545e98.png
五、客户机测试
1、通过访问虚拟ip测试nginx网站
http://i2.运维网.com/images/blog/201810/14/b27e71d7aae87f70c8a23dda77b72842.png
2、宕掉Master的keepalived服务器的虚拟ip，再次访问
# systemctl stop keepalived.service
http://i2.运维网.com/images/blog/201810/14/fd6289fbe9171415fc17e6fbebb072c0.png
结尾小结：
1、因本文使用的虚拟机模拟案例，所以先用NAT模式下好了所有软件包，然后修改为仅主机模式。
2、关于keepalived宕机再开的问题，要查看防火墙，重新关闭一次。
3、测试时宕掉了master的keepalved，发现无法访问站点，需要检查虚拟IP，重新启动ipvsadm。
页:
[1]