KeepalivedHA+LVS 部署安装文档

xsw222

　　1．1测试环境
操作系统：CentOS5.5 x86_64

主机名	用途	IP地址	网关
serverA.cyhd.com.cn	LVS主机	IP:eth0:192.168.20.61/24 VIP:eth0:192.168.20.70/24	192.168.20.1
serverB.cyhd.com.cn	LVS备机	IP:eth0:192.168.20.62/24 VIP:eth0:192.168.20.70/24	192.168.20.1
serverE.cyhd.com.cn	Realserver LAMP1	RIP:eth0:192.168.20.65/24 VIP:Lo:0:192.168.20.70/32	192.168.20.1
serverF.cyhd.com.cn	Realserver LAMP2	RIP:eth0:192.168.20.66/24 VIP:Lo:0:192.168.20.70/32	192.168.20.1

 
　　1．2拓扑图

软件安装
2．1 LVS软件安装

Yum –y install ipvs*

 
2．2 Keepalived软件安装

wget http://www.keepalived.org/software/keepalived-1.1.17.tar.gz tar xzvf keepalived-1.1.17.tar.gz cd keepalived-1.1.17 ./configure --enable-lvs-syncd --enable-lvs --with-kernel-dir=/lib/modules/`uname -r`/build make && make install cp /usr/local/etc/rc.d/init.d/keepalived /etc/rc.d/init.d/ chmod a+x /etc/init.d/keepalived cp /usr/local/etc/sysconfig/keepalived /etc/sysconfig/ ln -s /usr/local/sbin/keepalived /usr/sbin/keepalived ln -s /usr/local/etc/keepalived/ /etc/keepalived

 
 
　　LVS-DR模式实现 3．1 DR配置
用脚本配置LVS-DR模式的调度器

#!/bin/bash GW=192.168.20.1 # website director vip. SNS_VIP=192.168.20.70 SNS_RIP1=192.168.20.65 SNS_RIP2=192.168.20.66 . /etc/rc.d/init.d/functions logger $0 called with $1 case "$1" in start) # set squid vip /sbin/ipvsadm --set 30 5 60 /sbin/ifconfig eth0:0 $SNS_VIP broadcast $SNS_VIP netmask 255.255.255.255 broadcast $SNS_VIP up /sbin/route add -host $SNS_VIP dev eth0:0 sysctl -w "net.ipv4.ip_forward=0" sysctl -w "net.ipv4.conf.all.send_redirects=1" sysctl -w "net.ipv4.conf.default.send_redirects=1" sysctl -w "net.ipv4.conf.eth1.send_redirects=1" /sbin/ipvsadm -A -t $SNS_VIP:80 -s wrr /sbin/ipvsadm -a -t $SNS_VIP:80 -r $SNS_RIP1:80 -g -w 1 /sbin/ipvsadm -a -t $SNS_VIP:80 -r $SNS_RIP2:80 -g -w 1 touch /var/lock/subsys/ipvsadm >/dev/null 2>&1 ;; stop) /sbin/ipvsadm -C /sbin/ipvsadm -Z ifconfig eth0:0 down ifconfig eth0:1 down sysctl -w "net.ipv4.ip_forward=0" sysctl -w "net.ipv4.conf.all.send_redirects=0" sysctl -w "net.ipv4.conf.default.send_redirects=0" sysctl -w "net.ipv4.conf.eth1.send_redirects=0" route del $SNS_VIP route del $SS_VIP rm -rf /var/lock/subsys/ipvsadm >/dev/null 2>&1 echo "ipvsadm stoped" ;; status) if [ ! -e /var/lock/subsys/ipvsadm ];then echo "ipvsadm stoped" exit 1 else echo "ipvsadm OK" fi ;; *) echo "Usage: $0 {start|stop|status}" exit 1 esac exit 0

 
　　3．2 RS配置
用脚本配置LVS-DR模式的Realserver

#!/bin/bash SNS_VIP=192.168.20.70 . /etc/rc.d/init.d/functions case "$1" in start) ifconfig lo:0 $SNS_VIP netmask 255.255.255.255 broadcast $SNS_VIP /sbin/route add -host $SNS_VIP dev lo:0 echo "1" >/proc/sys/net/ipv4/conf/lo/arp_ignore echo "2" >/proc/sys/net/ipv4/conf/lo/arp_announce echo "1" >/proc/sys/net/ipv4/conf/all/arp_ignore echo "2" >/proc/sys/net/ipv4/conf/all/arp_announce sysctl -p >/dev/null 2>&1 echo "RealServer Start OK" ;; stop) ifconfig lo:0 down route del $SNS_VIP >/dev/null 2>&1 echo "0" >/proc/sys/net/ipv4/conf/lo/arp_ignore echo "0" >/proc/sys/net/ipv4/conf/lo/arp_announce echo "0" >/proc/sys/net/ipv4/conf/all/arp_ignore echo "0" >/proc/sys/net/ipv4/conf/all/arp_announce echo "RealServer Stoped" ;; *) echo "Usage: $0 {start|stop}" exit 1 esac exit 0

 
　　3．3测试
在LVS主机上运行配置LVS-DR模式调度器脚本，在两台Realserver上运行配置LVS-DR模式Realserver脚本。

[iyunv@serverA ~]# ipvsadm -ln IP Virtual Server version 1.2.1 (size=4096) Prot LocalAddress:Port Scheduler Flags -> RemoteAddress:Port Forward Weight ActiveConn InActConn TCP 192.168.20.70:80 wrr -> 192.168.20.66:80 Route 1 0 0 -> 192.168.20.65:80 Route 1 0 0

 
查看调度器IP信息

[iyunv@serverB ~]# ip addr 2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast qlen 1000 link/ether 00:0c:29:5f:6d:c7 brd ff:ff:ff:ff:ff:ff inet 192.168.20.62/24 brd 192.168.20.255 scope global eth0 inet 192.168.20.70/32 brd 192.168.20.70 scope global eth0:0 inet6 fe80::20c:29ff:fe5f:6dc7/64 scope link valid_lft forever preferred_lft forever

 
测试调度器作用

[iyunv@serverG ~]# for((i=1;i<=90;i++));do curl http://192.168.20.70/test.html;done 2>/dev/null | sort -n | uniq -c 45 192.168.20.65 45 192.168.20.66

 
停掉serverF上的HTTPD服务

[iyunv@serverF ~]# service httpd stop

 
测试调度器作用

[iyunv@serverA ~]# for((i=1;i<=90;i++));do curl http://192.168.20.70/test.html;done 2>/dev/null | sort -n | uniq -c 45 192.168.20.65

 
查看调度器信息

[iyunv@serverB ~]# ipvsadm -ln IP Virtual Server version 1.2.1 (size=4096) Prot LocalAddress:Port Scheduler Flags -> RemoteAddress:Port Forward Weight ActiveConn InActConn TCP 192.168.20.70:80 wrr -> 192.168.20.66:80 Route 1 0 45 -> 192.168.20.65:80 Route 1 0 0

 
后端RS死掉了，LVS并不会自动发现，请求仍然转发过去，致使有些请求没有正确的返回。
 
　　KeepalivedHA+LVS 4．1 LVS主机配置
Keepalived主机配置

! Configuration File for keepalived global_defs { router_id cy2009 } vrrp_instance VI_1 { state BACKUP interface eth0 virtual_router_id 61 priority 200 advert_int 1 nopreempt authentication { auth_type PASS auth_pass 1111 } virtual_ipaddress { 192.168.20.70/24 dev eth0 } } virtual_server 192.168.20.70 80 { delay_loop 6 lb_algo wrr lb_kind DR # persistence_timeout 60 nat_mask 255.255.255.0 protocol TCP real_server 192.168.20.65 80 { weight 20 TCP_CHECK { connect_port 80 connect_timeout 3 nb_get_retry 3 delay_before_retry 3 } } real_server 192.168.20.66 80 { weight 10 TCP_CHECK { connect_port 80 connect_timeout 3 nb_get_retry 3 delay_before_retry 3 } } }

 
　　4．2 LVS备机配置
Keepalived备机配置

! Configuration File for keepalived global_defs { router_id cy2009 } vrrp_instance VI_1 { state BACKUP interface eth0 virtual_router_id 61 priority 100 advert_int 1 # nopreempt authentication { auth_type PASS auth_pass 1111 } virtual_ipaddress { 192.168.20.70/24 dev eth0 } } virtual_server 192.168.20.70 80 { delay_loop 6 lb_algo wrr lb_kind DR # persistence_timeout 60 nat_mask 255.255.255.0 protocol TCP real_server 192.168.20.65 80 { weight 20 TCP_CHECK { connect_port 80 connect_timeout 3 nb_get_retry 3 delay_before_retry 3 } } real_server 192.168.20.66 80 { weight 10 TCP_CHECK { connect_port 80 connect_timeout 3 nb_get_retry 3 delay_before_retry 3 } } }

 
　　4．3 RS配置
和LVS配置RS相同，使用RS配置脚本

#!/bin/bash SNS_VIP=192.168.20.70 . /etc/rc.d/init.d/functions case "$1" in start) ifconfig lo:0 $SNS_VIP netmask 255.255.255.255 broadcast $SNS_VIP /sbin/route add -host $SNS_VIP dev lo:0 echo "1" >/proc/sys/net/ipv4/conf/lo/arp_ignore echo "2" >/proc/sys/net/ipv4/conf/lo/arp_announce echo "1" >/proc/sys/net/ipv4/conf/all/arp_ignore echo "2" >/proc/sys/net/ipv4/conf/all/arp_announce sysctl -p >/dev/null 2>&1 echo "RealServer Start OK" ;; stop) ifconfig lo:0 down route del $SNS_VIP >/dev/null 2>&1 echo "0" >/proc/sys/net/ipv4/conf/lo/arp_ignore echo "0" >/proc/sys/net/ipv4/conf/lo/arp_announce echo "0" >/proc/sys/net/ipv4/conf/all/arp_ignore echo "0" >/proc/sys/net/ipv4/conf/all/arp_announce echo "RealServer Stoped" ;; *) echo "Usage: $0 {start|stop}" exit 1 esac exit 0

 
　　4．4切换测试
先启动LVS主机上的Keepalived

[iyunv@serverA ~]# service keepalived start

 
LVS主机IP信息

[iyunv@serverA ~]# ip addr 2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast qlen 1000 link/ether 00:0c:29:06:d1:2c brd ff:ff:ff:ff:ff:ff inet 192.168.20.61/24 brd 192.168.20.255 scope global eth0 inet 192.168.20.70/24 scope global secondary eth0 inet6 fe80::20c:29ff:fe06:d12c/64 scope link valid_lft forever preferred_lft forever

 
LVS信息

[iyunv@serverA ~]# ipvsadm -ln IP Virtual Server version 1.2.1 (size=4096) Prot LocalAddress:Port Scheduler Flags -> RemoteAddress:Port Forward Weight ActiveConn InActConn TCP 192.168.20.70:80 wrr -> 192.168.20.66:80 Route 10 0 0 -> 192.168.20.65:80 Route 20 0 0

 
测试信息

[iyunv@serverG ~]# for((i=1;i<=90;i++));do curl http://192.168.20.70/test.html;done 2>/dev/null | sort -n | uniq -c 60 192.168.20.65 30 192.168.20.66

 
停止掉serverF的HTTPD服务

[iyunv@serverF ~]# service httpd stop

 
测试信息

[iyunv@serverG ~]# for((i=1;i<=90;i++));do curl http://192.168.20.70/test.html;done 2>/dev/null | sort -n | uniq -c 90 192.168.20.65

 
LVS信息

[iyunv@serverA ~]# ipvsadm -ln IP Virtual Server version 1.2.1 (size=4096) Prot LocalAddress:Port Scheduler Flags -> RemoteAddress:Port Forward Weight ActiveConn InActConn TCP 192.168.20.70:80 wrr -> 192.168.20.65:80 Route 20 0 60

 
重新启动serverF的HTTPD服务

[iyunv@serverF ~]# service httpd start

 
测试信息

[iyunv@serveG ~]# for((i=1;i<=90;i++));do curl http://192.168.20.70/test.html;done 2>/dev/null | sort -n | uniq -c 60 192.168.20.65 30 192.168.20.66

 
LVS信息

[iyunv@serverB ~]# ipvsadm -ln IP Virtual Server version 1.2.1 (size=4096) Prot LocalAddress:Port Scheduler Flags -> RemoteAddress:Port Forward Weight ActiveConn InActConn TCP 192.168.20.70:80 wrr -> 192.168.20.66:80 Route 10 0 0 -> 192.168.20.65:80 Route 20 0 0

 
启动LVS备机上的Keepalived

[iyunv@serverB ~]# service keepalived start

 
关闭LVS主机上的Keepalived

[iyunv@serverA ~]# service keepalived stop

 
查看LVS主机IP信息

[iyunv@serverA ~]# ip addr 2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast qlen 1000 link/ether 00:0c:29:06:d1:2c brd ff:ff:ff:ff:ff:ff inet 192.168.20.61/24 brd 192.168.20.255 scope global eth0 inet6 fe80::20c:29ff:fe06:d12c/64 scope link valid_lft forever preferred_lft forever

 
查看LVS备机IP信息

[iyunv@serverB ~]# ip addr 2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast qlen 1000 link/ether 00:0c:29:5f:6d:c7 brd ff:ff:ff:ff:ff:ff inet 192.168.20.62/24 brd 192.168.20.255 scope global eth0 inet 192.168.20.70/24 scope global secondary eth0 inet6 fe80::20c:29ff:fe5f:6dc7/64 scope link valid_lft forever preferred_lft forever

 
LVS信息

[iyunv@serverB ~]# ipvsadm -ln IP Virtual Server version 1.2.1 (size=4096) Prot LocalAddress:Port Scheduler Flags -> RemoteAddress:Port Forward Weight ActiveConn InActConn TCP 192.168.20.70:80 wrr -> 192.168.20.66:80 Route 10 0 0 -> 192.168.20.65:80 Route 20 0 0

 
重新启动LVS主机Keepalived

[iyunv@serverA ~]# service keepalived start

 
查看LVS主机IP信息

[iyunv@serverA ~]# ip addr 2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast qlen 1000 link/ether 00:0c:29:06:d1:2c brd ff:ff:ff:ff:ff:ff inet 192.168.20.61/24 brd 192.168.20.255 scope global eth0 inet6 fe80::20c:29ff:fe06:d12c/64 scope link valid_lft forever preferred_lft forever

 
LVS主机并没有和备机争VIP
测试

[iyunv@serverG ~]# for((i=1;i<=90;i++));do curl http://192.168.20.70/test.html;done 2>/dev/null | sort -n | uniq -c 60 192.168.20.65 30 192.168.20.66