centos6.5里搭建LVS+Keepalived(DR模式)集群

were

最近闲着，整理下以前搭建的服务。。
介绍还是百度下，
LVS+Keepalived 介绍

LVS

LVS是Linux Virtual Server的简写，意即Linux虚拟服务器，是一个虚拟的服务器集群系统。本项目在1998年5月由章文嵩博士成立，是中国国内最早出现的自由软件项目之一。目前有三种IP负载均衡技术（VS/NAT、VS/TUN和VS/DR）；
十种调度算法（rrr|wrr|lc|wlc|lblc|lblcr|dh|sh|sed|nq）。
Keepalvied
Keepalived在这里主要用作RealServer的健康状态检查以及LoadBalance主机和BackUP主机之间failover的实现

在虚拟机里搭建的环境拓扑

**********************************************
备注：实际生产环境图例如下


开始配置第一台主LVS服务器

准备：

[iyunv@master ~]# hostname
master
[iyunv@master ~]# cat /etc/sysconfig/network-scripts/ifcfg-eth0
DEVICE=eth0
HWADDR=00:0C:29:5F:BF:C2
TYPE=Ethernet
UUID=e9a08caa-f0df-485f-8bbe-1a736e9c2c02
ONBOOT=yes
IPADDR=192.168.1.150
PREFIX=24
GATEWAY=192.168.1.1
DNS1=61.134.1.4
NM_CONTROLLED=yes
BOOTPROTO=none
********
1）安装系统依赖包
yum -y install gcc gcc-c++ makepcre pcre-devel kernel-devel openssl-devel libnl-devel popt-devel
2）安装ipvsadm

wget http://www.linuxvirtualserver.or ... ipvsadm-1.26.tar.gz
tar -zxvf ipvsadm-1.26.tar.gz
cd ipvsadm-1.26
make
报错啦：
ipvsadm.o: In function `parse_options':
/usr/local/ipvsadm-1.26/ipvsadm.c:432: undefined reference to `poptGetContext'
/usr/local/ipvsadm-1.26/ipvsadm.c:435: undefined reference to `poptGetNextOpt'
/usr/local/ipvsadm-1.26/ipvsadm.c:660: undefined reference to `poptBadOption'
/usr/local/ipvsadm-1.26/ipvsadm.c:502: undefined reference to `poptGetNextOpt'
/usr/local/ipvsadm-1.26/ipvsadm.c:667: undefined reference to `poptStrerror'
/usr/local/ipvsadm-1.26/ipvsadm.c:667: undefined reference to `poptBadOption'
/usr/local/ipvsadm-1.26/ipvsadm.c:670: undefined reference to `poptFreeContext'
/usr/local/ipvsadm-1.26/ipvsadm.c:677: undefined reference to `poptGetArg'
/usr/local/ipvsadm-1.26/ipvsadm.c:678: undefined reference to `poptGetArg'
/usr/local/ipvsadm-1.26/ipvsadm.c:679: undefined reference to `poptGetArg'
/usr/local/ipvsadm-1.26/ipvsadm.c:690: undefined reference to `poptGetArg'
/usr/local/ipvsadm-1.26/ipvsadm.c:693: undefined reference to `poptFreeContext'
collect2: ld returned 1 exit status
make: *** [ipvsadm] Error 1

解决办法： yum -y install kernel-headers   popt-static
然后 make clean
接着make
make install
3)安装keepalived
wget http://www.keepalived.org/software/keepalived-1.2.7.tar.gz
tar -zxvf keepalived-1.2.7.tar.gz
cd keepalived-1.2.7
./configure --prefix=/usr/local/keepalived
make

make install
4)配置keepalived 为系统服务

cp /usr/local/keepalived/etc/rc.d/init.d/keepalived /etc/init.d/
cp /usr/local/keepalived/etc/sysconfig/keepalived /etc/sysconfig/
mkdir /etc/keepalived/
cp /usr/local/keepalived/etc/keepalived/keepalived.conf /etc/keepalived
cp /usr/local/keepalived/sbin/keepalived /usr/sbin/
5）修改主的keepalived配置文件

cd /etc/keepalived/
然后对主配置文件做个备份 mv keepalived.conf keepalived.conf.bak

编辑主配置文件
vim /etc/keepalived/keepalived.conf
***
! Configuration File forkeepalived
global_defs {
notification_email {
309937985@qq.com            #发生故障联系人地址
}
notification_email_from 309937985@qq.com   #故障发送人
smtp_server 192.168.1.150    #本机发送邮件
smtp_connect_timeout 30
router_id LVS_MASTER     #BACKUP上修改为LVS_BACKUP
}
vrrp_instance VI_1 {
state MASTER             #BACKUP上修改为BACKUP
interface eth0
virtual_router_id 51     #虚拟路由标识，主从相同
priority 100             #BACKUP上修改为50
advert_int 1
authentication {
auth_type PASS
auth_pass 1111           #主从认证密码必须一致
}
virtual_ipaddress {      #Web虚拟IP（VTP）
192.168.1.200
  }
}
virtual_server 192.168.1.200 80 { #定义虚拟IP和端口
delay_loop 6    #检查真实服务器时间，单位秒
lb_algo rr      #设置负载调度算法，rr为轮训
lb_kind DR      #设置LVS负载均衡DR模式
persistence_timeout 50 #同一IP的连接60秒内被分配到同一台真实服务器
protocol TCP    #使用TCP协议检查realserver状态
real_server 192.168.1.170 80 {  #第一个web服务器
weight 3          #节点权重值
TCP_CHECK {       #健康检查方式
connect_timeout 3 #连接超时
nb_get_retry 3    #重试次数
delay_before_retry 3  #重试间隔/S
  }
}
real_server 192.168.1.180 80 {  #第二个web服务器
weight 3
TCP_CHECK {
connect_timeout 3
nb_get_retry 3
delay_before_retry 3
     }
  }
}
ok**********************************8
启动服务
service keepalived restart
&&&&&&&&&&&&&&&&&
&&&&&&&&&&&

从LVS服务器（192.168.1.160）同理


[iyunv@backup ~]# hostname
backup
[iyunv@backup ~]# cat /etc/sysconfig/network-scripts/ifcfg-eth0
DEVICE=eth0
HWADDR=00:0C:29:37:07:70
TYPE=Ethernet
UUID=058947a0-20c0-4135-a6d0-f4345a951f9f
ONBOOT=yes
IPADDR=192.168.1.160
PREFIX=24
GATEWAY=192.168.1.1
DNS1=61.134.1.4
NM_CONTROLLED=yes
BOOTPROTO=none
1）安装依赖包。安装ipvsadm 和keepalived
2) 编辑keepalived


! Configuration File forkeepalived
! Configuration File forkeepalived
global_defs {
notification_email {
309937985@qq.com            #故障接受联系人
}
notification_email_from 309937985@qq.com   #故障发送人
smtp_server 192.168.1.160    #本机发送邮件
smtp_connect_timeout 30
router_id LVS_BACKUP     #BACKUP上修改为LVS_BACKUP
}
vrrp_instance VI_1 {
state BACKUP             #BACKUP上修改为BACKUP
interface eth0
virtual_router_id 51     #虚拟路由标识，主从相同
priority 50             #BACKUP上修改为50
advert_int 1
authentication {
auth_type PASS
auth_pass 1111           #主从认证密码必须一致
}
virtual_ipaddress {      #Web虚拟IP（VTP）
192.168.1.200
  }
}
virtual_server 192.168.1.200 80 { #定义虚拟IP和端口
delay_loop 6    #检查真实服务器时间，单位秒
lb_algo rr      #设置负载调度算法，rr为轮训
lb_kind DR      #设置LVS负载均衡DR模式
persistence_timeout 50 #同一IP的连接60秒内被分配到同一台真实服务器
protocol TCP    #使用TCP协议检查realserver状态
real_server 192.168.1.170 80 {  #第一个web服务器
weight 3          #节点权重值
TCP_CHECK {       #健康检查方式
connect_timeout 3 #连接超时
nb_get_retry 3    #重试次数
delay_before_retry 3  #重试间隔/S
  }
}
real_server 192.168.1.180 80 {  #第二个web服务器
weight 3
TCP_CHECK {
connect_timeout 3
nb_get_retry 3
delay_before_retry 3
     }
  }
}

然后重新启动服务 /etc/init.d/keepalived restart

$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$
最后的部分：
在两台web服务器上安装好httpd,测试用
在两台web服务器上分别绑定Vip

1))   
[iyunv@rs1 ~]# hostname
rs1
[iyunv@rs1 ~]# cat /etc/sysconfig/network-scripts/ifcfg-eth0
DEVICE=eth0
HWADDR=00:0C:29:47:C9:73
TYPE=Ethernet
UUID=c26a2e9c-bdee-437b-aff8-c460f92c3ca9
ONBOOT=yes
IPADDR=192.168.1.170
PREFIX=24
GATEWAY=192.168.1.1
DNS1=61.134.1.4
NM_CONTROLLED=yes
BOOTPROTO=none
2)) 安装apache
   yum install httpd -y
cd /var/www/html/
touch index.html
echo "170" > index.html
第2台WEB服务器同样这么： echo "180" > index.html

然后重新启动apache

分别在两台web服务器上手工绑定VIP
1》[iyunv@rs1 network-scripts]# cat /etc/sysconfig/network-scripts/ifcfg-lo:200
DEVICE=lo:200
IPADDR=192.168.1.200
NETMASK=255.255.255.255
NETWORK=127.0.0.0
# If you're having problems with gated making 127.0.0.0/8 a martian,
# you can change this to something else (255.255.255.255, for example)
BROADCAST=127.255.255.255
ONBOOT=yes
NAME=loopback
2》》抑制真实服务器的ARP广播

echo "1" > /proc/sys/net/ipv4/conf/lo/arp_ignore
echo "2" > /proc/sys/net/ipv4/conf/lo/arp_announce
echo "1" > /proc/sys/net/ipv4/conf/all/arp_ignore
echo "2" > /proc/sys/net/ipv4/conf/all/arp_announce

=====================================================================
当然两个WEB服务器端也可以直接用脚本实现ARP抑制和绑定VIP
vi /etc/init.d/real.sh
#description : start realserver

VIP=192.168.1.200
. /etc/init.d/functions
case "$1" in
start)
/sbin/ifconfig lo:0 $VIP broadcast $VIP netmask 255.255.255.255 up
echo "1" >/proc/sys/net/ipv4/conf/lo/arp_ignore
echo "2" >/proc/sys/net/ipv4/conf/lo/arp_announce
echo "1" >/proc/sys/net/ipv4/conf/all/arp_ignore
echo "2" >/proc/sys/net/ipv4/conf/all/arp_announce
echo "LVS RealServer Start OK"
;;
stop)
/sbin/ifconfig lo:0 down
echo "0" >/proc/sys/net/ipv4/conf/lo/arp_ignore
echo "0" >/proc/sys/net/ipv4/conf/lo/arp_announce
echo "0" >/proc/sys/net/ipv4/conf/all/arp_ignore
echo "0" >/proc/sys/net/ipv4/conf/all/arp_announce
echo "LVS RealServer Stoped OK"
;;
*)
echo "Usage: $0 {start|stop}"
exit 1
esac

===============================================
然后启动脚本
chmod +x /etc/init.d/real.sh

/etc/init.d/real.sh start

echo "/etc/init.d/real.sh start" >> /etc/rc.local





所有配置结束。最后验证集群。。
[iyunv@master ~]# ipvsadm -L -n --stats
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port               Conns   InPkts  OutPkts  InBytes OutBytes
  -> RemoteAddress:Port
TCP  192.168.1.200:80                   10       50        0     4016        0
  -> 192.168.1.170:80                    5       25        0     2008        0
  -> 192.168.1.180:80                    5       25        0     2008        0

当挂掉rs2服务器，访问VIP自动切换到rs1服务器上
当挂掉rs1服务器，访问VIP自动切换到rs2服务器上
而且当LVS的master和backup任何一台服务器挂掉，访问vip地址，网站都正常。