实例配置keepalived+lvs+nginx

124q31 · 发表于 2015-6-9 09:07:17

Lvs+Keepalived高可用负载均衡配置,近期需要使用，所以自己搞了个虚拟机进行了简单的安装测试，希望可以帮助大家解决小问题。
mater    192.168.78.128                            //keepalived+lvs
slave    192.168.78.129                            //keepalived+lvs
real server1 192.168.78.130                   //web1,nginx
real server2    192.168.78.131                   //web1,nginx
vip    192.168.78.132

1.首先给服务器安装依赖包

1	yum -y install gcc gcc-c++ make pcre pcre-devel kernel-devel openssl-devel

2.安装keepalived，在master和slave上

1
2
3
4
5
6
7
8
9
10
11
12
13

cd /root/software
wget http://www.keepalived.org/software/keepalived-1.2.15.tar.gz
tar xzf keepalived-1.2.15.tar.gz
cd keepalived-1.2.15
./configure --prefix=/usr/local/keepalived --with-kernel-dir=/usr/src/kernels/2.6.32-504.16.2.el6.i686/
make
make install

cp -a /usr/local/keepalived/etc/rc.d/init.d/keepalived /etc/init.d/
cp -a /usr/local/keepalived/etc/sysconfig/keepalived /etc/sysconfig/
mkdir /etc/keepalived
cp -a /usr/local/keepalived/etc/keepalived/keepalived.conf /etc/keepalived/
cp -a /usr/local/keepalived/sbin/keepalived /usr/bin/

注意：--with-kernel-dir=/usr/src/kernels/2.6.32-504.16.2.el6.i686/ 这个选项一定要加；这个选项并不是把keepalived编译进内核，而是指定使用内核源码中的头文件，即include目录（只有在配置lvs时才用此选项，其他时候不需要）

3.配置keepalived.conf文件，先将原来的模板配置文件拷贝一份备份，然后在修改
cp /etc/keepalived/keepalived.conf /etc/keepalived/keepalived.conf.old
vim /etc/keepalived/keepalived.conf

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47

! Configuration File for keepalived
global_defs {
notification_email {
1015099137@qq.com                      #故障联系人
}
notification_email_from root@localhost #故障发发送人
smtp_server 127.0.0.1
smtp_connect_timeout 30
router_id lvs
}
vrrp_instance VI_1 {
state MASTER
interface eth0
virtual_router_id 51    #虚拟路由标识，主从相同
priority 150
advert_int 1
authentication {
      auth_type PASS
      auth_pass 1111       #主从认证密码，必须保持一致
}
virtual_ipaddress {
      192.168.78.132       #Web虚拟IP（VTP）
}
}
virtual_server 192.168.78.132 80 { #定义虚拟IP和端口
delay_loop 6                   #检查真实服务器时间，单位秒
lb_algo rr                      #设置负载调度算法
lb_kind DR                      #设置LVS负载均衡DR模式
persistence_timeout 50          #同一IP的连接50秒内被分配到同一台真实服务器
protocol TCP                   #使用TCP协议检查realserver状态
real_server 192.168.78.130 80 {  #第一个web服务器
      weight 3                   #节点权重值
      TCP_CHECK {                #健康检查方式
         connect_timeout 10    #连接超时
         nb_get_retry 3          #重试次数
         delay_before_retry 3    #重试间隔/S
         }
}
real_server 192.168.78.131 80 {  #第二个web服务器
      weight 3
      TCP_CHECK {
         connect_timeout 10
         nb_get_retry 3
         delay_before_retry 3
         }
}
}

在（备）192.168.186.129，对配置文件做如下修改：
将state master改为state backup
将priority 150改为priority 100
/etc/init.d/keepalived start

查看是否启动：
tail -n 30 /var/log/messages

注意：/etc/sysconfig/keepalived 和 /etc/keepalived/keepalived.conf 的路径一定要正确，因为在执行/etc/init.d/keepalived这个启动脚本时，会读取/etc/sysconfig/keepalived 和 /etc/keepalived/keepalived.conf 这两个文件

4.安装IPVS：在master和slave进行如下操作
(1)安装依赖包

1	yum -y install popt popt-devel libnl libnl-devel popt-static

(2)安装IPVS

1
2
3
4
5
6

cd /root/software
wget http://www.linuxvirtualserver.or ... ipvsadm-1.26.tar.gz
tar xzf ipvsadm-1.26.tar.gz
cd ipvsadm-1.26
make && make insatll
ipvsadm --help //查看帮助，如果出现就说明安装成功

5.配置real server (master1,slave1)
注意：每台real server都要进行以下操作：
我们采用的是DR负载方式，用户的请求到达real server后，real server处理完数据后是直接返回给用户的，不再经过LB负载机，因此，需要在每台real server上都绑定一个vip,vip绑定在lo接口
vim /etc/rc.d/init.d/rscreatevip.sh

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25

#!/bin/bash
#real server create vip
VIP=192.168.78.132
#./etc/rc.d/init.d/functions
case $1 in
   start)
   echo "real server create vip"
   /sbin/ifconfig lo:0 $VIP broadcast $VIP netmask 255.255.255.255 up
   echo 1 > /proc/sys/net/ipv4/conf/lo/arp_ignore
   echo 2 > /proc/sys/net/ipv4/conf/lo/arp_announce
   echo 1 > /proc/sys/net/ipv4/conf/all/arp_ignore
   echo 2 > /proc/sys/net/ipv4/conf/all/arp_announce
   ;;
   stop)
   /sbin/ifconfig lo:0 down
   echo "real server remove vip"
   echo 0 > /proc/sys/net/ipv4/conf/lo/arp_ignore
   echo 0 > /proc/sys/net/ipv4/conf/lo/arp_announce
   echo 0 > /proc/sys/net/ipv4/conf/all/arp_ignore
   echo 0 > /proc/sys/net/ipv4/conf/all/arp_announce
   ;;
   *)
   echo "usage: (start|stop)"
   exit 1
esac

chmod 755 /etc/init.d/rscreatevip.sh #加执行权限
/etc/init.d/rscreatevip.sh start
echo "/etc/init.d/rscreatevip.sh start" >> /etc/rc.d/rc.local #设置开机自动启动

6.把real server 2台web开启
/usr/local/nginx/sbin/nginx
/usr/local/php/sbin/php-fpm
echo "welcome to 130" > /data/web/welcome.com/index.html
echo "welcome to 131" > /data/web/welcome.com/index.html

7.把master和slave的keepalived启动，并把所以机器的防火墙和selinux全部关闭进行测试
/etc/init.d/keepalived start
service iptables stop
setenforce 0

8.在master上执行如下命令
ipvsadm -ln #查看监控的real server 有哪些
QQ截图20150609090647.png