centos6.2,64位系统搭建LVS+Keepalived（DR模式）

ienki

　　 最近公司网站访问又见瓶颈，为了缓解压力，决定用LVS+Keepalived来实现负载均衡.在前面的文章中已经写过了用NAT模式来实现负载均衡 http://duyunlong.blog.运维网.com/1054716/716138，而今天用的是DR模式，其实NAT跟DR模式，各有优缺点，根据实际情况来选择适合自己的。下面看下具体步骤以及注意事项。
　　拓扑图如下：

首先看下我用的系统，系统为centos6.2，64位系统，用getconf LONG_BIT,和uname -a都可以查看

[root@lvs-master src]# getconf LONG_BIT
64
[root@lvs-master src]# uname -a
Linux lvs-master 2.6.32-279.19.1.el6.x86_64 #1 SMP Wed Dec 19 07:05:20 UTC 2012 x86_64 x86_64 x86_64 GNU/Linux

然后分析下搭建这个LVS+Keepalived（DR）需要的关键步骤及注意事项，第一、DR模式主要是靠虚拟IP来实现，客户端访问服务器首先会找keeaplived服务器，然后由keepalived服务器调度找到真实机，最后真实机通过自己的虚拟IP直接返回给客户端。那么这里有个疑问了，为什么每一台机器上都有虚拟IP，而客户端访问却单单找keepalived？那是因为keepalived服务器在不停发出例如：我就你要找的IP的信号，所以客户端很容易发现自己需要访问的IP而找到keepalived服务器。这个可以通过日志来发现。那么我们就需要在除了keepalived服务器之外的每台真实机上建立一个虚拟IP。这里又有疑问keepalived服务器为什么不需要建立虚拟IP，那是因为keepalived会自己生成虚拟IP，所以没必要自己去建立。第二、要注意的就是popt-static-1.13-7.el6.x86_64.rpm这个包，不装这个包，安装ipvs的时候会报错。第三、keepalived安装的时候一定要指定内核，要不然keepalived启动后只有两进程，正常是三个。如果不指定内核即使keepalived启动了，keepalived也发现不了真实机。第四、就是一些零碎的配置。

[root@lvs-master src]# wget http://www.keepalived.org/software/keepalived-1.2.2.tar.gz
[root@lvs-master src]# wget http://www.linuxvirtualserver.org/software/kernel-2.6/ipvsadm-1.26.tar.gz
[root@lvs-master src]# wget http://mirror.centos.org/centos/6/os/x86_64/Packages/popt-static-1.13-7.el6.x86_64.rpm

然后，在LVS主跟备份上安装

[root@lvs-master src]# rpm -ivh popt-static-1.13-7.el6.x86_64.rpm
[root@lvs-master src]# yum -y install kernel-devel make gcc openssl-devel libnl* popt*
[root@lvs-master src]# ln -s /usr/src/kernels/2.6.32-220.13.1.el6.x86_64/ /usr/src/linux
[root@lvs-master src]# tar -zxvf ipvsadm-1.26.tar.gz
[root@lvs-master src]# cd ipvsadm-1.26/
[root@lvs-master ipvsadm-1.26]# make
[root@lvs-master ipvsadm-1.26]# make install
[root@lvs-master ipvsadm-1.26]# cd ..
[root@lvs-master src]# tar -zxvf keepalived-1.2.2.tar.gz
[root@lvs-master src]# cd keepalived-1.2.2/
[root@lvs-master keepalived-1.2.2]# ./configure --with-kernel-dir=/usr/src/kernels/2.6.32-358.2.1.el6.x86_64/
[root@lvs-master keepalived-1.2.2]# make && make install
[root@lvs-master keepalived-1.2.2]# cp /usr/local/etc/rc.d/init.d/keepalived /etc/rc.d/init.d/
[root@lvs-master keepalived-1.2.2]# cp /usr/local/etc/sysconfig/keepalived /etc/sysconfig/
[root@lvs-master keepalived-1.2.2]# mkdir /etc/keepalived
[root@lvs-master keepalived-1.2.2]# cp /usr/local/etc/keepalived/keepalived.conf /etc/keepalived/
[root@lvs-master keepalived-1.2.2]# cp /usr/local/sbin/keepalived /usr/sbin/

在两台LVS上分别配置keepalived， vim /etc/keepalived/keepalived.conf

[root@lvs-master src]# vim /etc/keepalived/keepalived.conf
#! Configuration File for keepalived
global_defs {
router_id LVS_MASTER #网上资料说这个值也需要修改，具体不详，之前我们线上的主备就一直是一样的 ^ ^还是修改一下吧！
             }
##################第一部分###################
vrrp_instance VI_1 {
     state MASTER #LVS_Backup上修改成BACKUP
     interface eth0
     virtual_router_id 60 #默认51 主从都修改为60
     priority 100 #LVS_Backup上修改成80
     advert_int 1
     authentication {
     auth_type PASS
     auth_pass 1111
     }
virtual_ipaddress {
     192.168.10.100
     }
}
##################第二部分###################
virtual_server 192.168.10.100 80 {
     delay_loop 6
     lb_algo rr
     lb_kind DR
     nat_mask 255.255.255.0
     persistence_timeout 50
     protocol TCP
real_server 192.168.10.3 80 {
     weight 1
     TCP_CHECK {
         connect_timeout 10
         nb_get_retry 3
         connect_port 80
         }
     }
real_server 192.168.10.4 80 {
     weight 1
     TCP_CHECK {
         connect_timeout 10
         nb_get_retry 3
         connect_port 80
         }
     }
}

在LVS_Backup上也进行配置，只要把state MASTER改为BACKUP,priority 100改为90，然后我们在真实机上配置虚拟IP启动脚本

[root@real-1 src]# vim /etc/init.d/realserver.sh
#!/bin/bash
SNS_VIP=192.168.10.100
. /etc/rc.d/init.d/functions
case "$1" in
start)
ifconfig lo:0 $SNS_VIP netmask 255.255.255.255 broadcast $SNS_VIP
/sbin/route add -host $SNS_VIP dev lo:0
echo "1" >/proc/sys/net/ipv4/conf/lo/arp_ignore
echo "2" >/proc/sys/net/ipv4/conf/lo/arp_announce
echo "1" >/proc/sys/net/ipv4/conf/all/arp_ignore
echo "2" >/proc/sys/net/ipv4/conf/all/arp_announce
sysctl -p >/dev/null 2>&1
echo "RealServer Start OK"
;;
stop)
ifconfig lo:0 down
route del $SNS_VIP >/dev/null 2>&1
echo "0" >/proc/sys/net/ipv4/conf/lo/arp_ignore
echo "0" >/proc/sys/net/ipv4/conf/lo/arp_announce
echo "0" >/proc/sys/net/ipv4/conf/all/arp_ignore
echo "0" >/proc/sys/net/ipv4/conf/all/arp_announce
echo "RealServer Stoped"
;;
*)
echo "Usage: $0 {start|stop}"
exit 1
esac
exit 0

在两台真是机器上面配置apache并启动realserver.sh

[root@real-1 src]# yum -y install httpd
[root@real-1 src]# cd /var/www/html/
[root@real-1 html]# cat index.html < h1>RealServer 192.168.10.3
[root@real-1 html]# /etc/init.d/httpd start
[root@real-1 src]# /etc/init.d/realserver.sh start

配置完后我们来测试下：分别启动Keepalived

[root@lvs-master src]# chkconfig keepalived on
[root@lvs-master src]# /etc/init.d/keepalived restart
[root@lvs-master src]# ipvsadm
IP Virtual Server version 1.2.1
(size=4096)Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP 192.168.10.100:http rr persistent 50
-> 192.168.10.3:http Route 1 0 0
-> 192.168.10.4:http Route 1 0 0

关闭LVS_MASTER，然后查看LVS_backup日志

[root@lvs-backup src]# tail -f /var/log/messages
Mar 21 07:22:28 host2 Keepalived_vrrp: VRRP_Instance(VI_1)
Transition to MASTER STATE
Mar 21 07:22:29 host2 Keepalived_vrrp: VRRP_Instance(VI_1)
Entering MASTER STATE
Mar 21 07:22:29 host2 Keepalived_vrrp: VRRP_Instance(VI_1)
setting protocol VIPs.
Mar 21 07:22:29 host2 Keepalived_healthcheckers: Netlink
reflector reports IP 192.168.10.100 added
Mar 21 07:22:29 host2 Keepalived_vrrp: VRRP_Instance(VI_1)
Sending gratuitous ARPs on eth0 for 192.168.10.100

测试访问

[root@test html]# curl http://192.168.10.100
< h1>RealServer 192.168.10.4

然后，启动LVS_MASTER查看LVS_BACKUP日志

[root@lvs-backup src]# tail -f /var/log/messages
Mar 21 07:26:20 host2 Keepalived_vrrp: VRRP_Instance(VI_1)
Received higher prio advert
Mar 21 07:26:20 host2 Keepalived_vrrp: VRRP_Instance(VI_1)
Entering BACKUP STATE
Mar 21 07:26:20 host2 Keepalived_vrrp: VRRP_Instance(VI_1)
removing protocol VIPs.
Mar 21 07:26:20 host2 Keepalived_healthcheckers: Netlink
reflector reports IP 192.168.10.100 removed

非常成功！！！