高可用集群技术之keepalived实现lvs高可用并负载均衡web服务

56341

Keepalived概述:
  Keepalived的作用是检测服务器的状态，如果有一台web服务器死机，或工作出现故障，Keepalived将检测到，并将有故障的服务器从系统中剔除，同时使用其他服务器代替该服务器的工作，当服务器工作正常后Keepalived自动将服务器加入到服务器群中，这些工作全部自动完成，不需要人工干涉，需要人工做的只是修复故障的服务器.


keepalived实现lvs高可用并负载均衡web服务：
准备工作:
2台keepalived节点为: node1.samlee.com node2.samlee.com
2台realserver服务为: node3.samlee.com node4.samlee.com

1、配置RSserver过程如下(node3.samlee.com\node4.samlee.com):
(1)部署web服务(使用nginx配置)

1	# yum -y install nginx-1.10.0-1.el6.ngx.x86_64.rpm

(2)禁用防火墙及主机名修改

1 2	# service iptables stop # chkconfig iptables off

(3)配置路由转发及转发模块配置

1 2 3 4 5 6 7 8

# echo 2 > /proc/sys/net/ipv4/conf/all/arp_announce # echo 2 > /proc/sys/net/ipv4/conf/eth0/arp_announce # echo 1 > /proc/sys/net/ipv4/conf/eth0/arp_ignore # echo 1 > /proc/sys/net/ipv4/conf/all/arp_ignore --添加路由条目 # ifconfig lo:0 172.16.100.99 netmask 255.255.255.255 broadcast 172.16.100.99 up # route add -host 172.16.100.99 dev lo:0

2、配置keepalived服务器
(1)安装ipvsadm负载均衡管理器

1	# yum -y install ipvsadm

(2)安装keepalived

1 2	# yum -y install keepalived # chkconfig keepalived on

(3)配置keepalived默认初始状态如下:(主节点:master状态)

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25

# vim /etc/keepalived/keepalived.conf ! Configuration File for keepalived global_defs {    notification_email {      root@localhost    }    notification_email_from root@localhost    smtp_server 127.0.0.1    smtp_connect_timeout 30    router_id LVS_DEVEL             #物理路由id，一般指定为本机的hostname } vrrp_instance VI_1 {                #vrrp实例配置段     state MASTER                    #指定keepalived服务运行状态     interface eth0                  #指定keepalived的心跳口     virtual_router_id 51            #指定虚拟路由id（1~255），同一vrrp实例的主备keepalived必须配置为一样     priority 101                    ##指定起始优先级，优先级高的会成为master     advert_int 1                    #vrrp通告的发送间隔     authentication {                #配置通过密码认证         auth_type PASS         auth_pass keepalivedpass     }     virtual_ipaddress {     172.16.100.99 dev eth0 label eth0:0    #定义VIP     } }

(4)配置keepalived默认初始状态如下:(备节点:backuo状态)

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27

# vim /etc/keepalived/keepalived.conf ! Configuration File for keepalived global_defs {    notification_email {      root@localhost    }    notification_email_from root@localhost    smtp_server 127.0.0.1    smtp_connect_timeout 30    router_id LVS_DEVEL } vrrp_instance VI_1 {     state BACKUP     interface eth0     virtual_router_id 51     priority 100     advert_int 1     authentication {         auth_type PASS         auth_pass keepalivedpass     }     virtual_ipaddress {     172.16.100.99 dev eth0 label eth0:0     } }

(5)调用周期性检测down文件是否存在来控制keepalived服务的主备切换
在keepalived.conf添加如下内容:

1 2 3 4 5 6 7 8 9 10 11

全局配置的段下添加如下: vrrp_script chk_downfile {         #服务状态检测脚本配置段     script "[[ -f /etc/keepalived/down ]] && exit 1 || exit 0"         #通过周期性检测down文件是否存在来控制keepalived服务的主备切换     interval 1     #指定检测间隔为1秒     weight -2     #指定检测失败时，优先级减2；检测的成功或失败是由script后面指定的命令或脚本执行返回的状态码决定的，0表示成功，非0表示失败 } 在vrrp_instance VI_1 ##vrrp实例配置段添加如下: track_script {     chk_downfile          #通过指定上面定义监控脚本来监控服务状态，以完成主备切换 }

重启keepalived服务创建down文件测试:

1 2 3 4

# service keepalived restart # touch /etc/keepalived/down ---在/etc/keepalived/下建立down文件可实现主备模式切换

(6)调用外部脚本检测web(nginx)健康状态
建立/etc/keepalived/chkngx.sh脚本

1 2 3 4 5 6 7 8 9

#/bin/bash #/etc/keepalived/chkngx.sh # if pkill -0 nginx>/dev/null; then         [ `curl -sL -w %{http_code} http://localhost -o /dev/null` -eq 200 ] && exit 0 else         service nginx start fi exit $?

授予脚本执行权限:

1	# chmod +x /etc/keepalived/chkngx.sh

配置keepalived.conf调用外部脚本

1 2 3 4 5 6 7 8 9 10 11 12 13

全局配置的段下添加如下: vrrp_script chkngx {     script "/etc/keepalived/chkngx.sh"    #指定用于检测nginx服务的执行脚本路径     interval 1        #监测间隔     weight -2         #失败时，优先级减2     fall 3          # 指定nginx检测脚本连续执行失败次数为3，才进行Failover     rise 3          # 指定nginx检测脚本连续执行成功次数为3，才进行Failback } 在vrrp_instance VI_1 ##vrrp实例配置段添加如下: track_script {     chkngx          #通过指定上面定义监控脚本来监控服务状态，监控nginx监控状态 }

(7)使用通知脚本实现如何在vrrp事务发生时，发送警告邮件给指定的管理员？ 比如：主从节点故障时
建立脚本监控脚本如下:

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42

# vim /etc/keepalived/notify.sh #!/bin/bash # declare -a vips=(172.16.100.98 172.16.100.99) contact='root@localhost' Usage() {         echo "Usage:`basename $0`{master|backup|fault}" } Notify() {     if [ "$2" = "VI_1" ]; then         vip=${vips[0]}     echo $vip     elif [ "$2" = "VI_2" ]; then         vip=${vips[1]}     echo $vip     fi     suject="`hostname`'s state chaged to $1"     mailbody="`date  +%F--%H:%M:%S`: `hostname`'s state chage to $1,vip floating..."     echo $mailbody | mail -s  "$subject" $contact     echo $mailbody >> /tmp/keepalived.log } case $1 in     master)     Notify master         exit 0     ;;     backup)     Notify backup         exit 0     ;;     fault)     Notify fault         exit 0     ;;     *)         Usage         exit 1     ;; esac

授予脚本执行权限:

1	# chmod +x /etc/keepalived/notify.sh

配置keepalived.conf调用外部脚本

1 2 3 4 5

在vrrp_instance VI_1 ##vrrp实例配置段最后添加如下: notify_master "/etc/keepalived/notify.sh master"        #状态变为master时，触发的通知脚本 notify_backup "/etc/keepalived/notify.sh backup"        #状态变为backup时，触发的通知脚本 notify_fault "/etc/keepalived/notify.sh fault"          #状态变为fault时，触发的通知脚本 notify "/etc/keepalived/notify.sh"    #当发生所有的状态改变时，会先触发对应的状态通知脚本后，再触发该脚本

(8)使用脚本根据down文件检测实现主从状态转换

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23

# vim /etc/keepalived/convert_state.sh #!/bin/bash # usage() {     echo "./`basename $0` {master|backup}"     exit 1 } if [ $# -eq 0 ]; then     usage fi if [[ "$1" = "master" ]]; then     [ -f /etc/keepalived/down ] && rm -f /etc/keepalived/down elif [[ "$1" = "backup" ]]; then     [ ! -f /etc/keepalived/down ] && touch /etc/keepalived/down else     usage fi sleep 1 # chmod +x /etc/keepalived/convert_state.sh

执行脚本convert_state进行备节点转换:

1 2 3

# /etc/keepalived/convert_state.sh backup # ls /etc/keepalived/ | grep down down

执行脚本convert_state进行主节点转换:

1	# /etc/keepalived/convert_state.sh master

(9)LVS+keepalived配置ipvs规则实现负载均衡高可用
配置如下:

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34

virtual_server 172.16.100.99 80 {     delay_loop 6     lb_algo rr     lb_kind DR     nat_mask 255.255.0.0     # persistence_timeout 50     protocol TCP     sorry_server 127.0.0.1 80        #如果所有节点都down机，将转向本地web服务     real_server 172.16.100.8 80 {         weight 1         HTTP_GET {                    #web健康状态检测             url {               path /               status_code 200             }             connect_timeout 3             nb_get_retry 3             delay_before_retry 3         }     }     real_server 172.16.100.9 80 {         weight 1         HTTP_GET {                    #web健康状态检测             url {               path /               status_code 200             }             connect_timeout 3             nb_get_retry 3             delay_before_retry 3         }     } }

最后查询ipvs规则，如下所示:

1 2 3 4 5 6 7

# ipvsadm -L -n --stats IP Virtual Server version 1.2.1 (size=4096) Prot LocalAddress:Port               Conns   InPkts  OutPkts  InBytes OutBytes   -> RemoteAddress:Port TCP  172.16.100.99:80                    3       22        0     4657        0   -> 172.16.100.8:80                     0        0        0        0        0   -> 172.16.100.9:80                     0        0        0        0        0

以上为keepalived+lvs+web服务实现所有内容。

keepalived双主模型配置案例如下:
master1为:

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48

vrrp_instance VI_1 {     state MASTER     interface eth0     virtual_router_id 51     priority 101     advert_int 1     authentication {         auth_type PASS         auth_pass keepalivedpass     }     virtual_ipaddress {     172.16.100.99 dev eth0 label eth0:0     }     track_script {     chk_downfile     }     track_script {     chkngx     }     notify_master "/etc/keepalived/notify.sh master"     notify_backup "/etc/keepalived/notify.sh backup"     notify_fault "/etc/keepalived/notify.sh fault"     notify "/etc/keepalived/notify.sh" } vrrp_instance VI_2 {     state BACKUP     interface eth0     virtual_router_id 52     priority 100     advert_int 1     authentication {         auth_type PASS         auth_pass keepalivedpass     }     virtual_ipaddress {         172.16.100.98 dev eth0 label eth0:1     }     track_script {         chk_downfile     }     track_script {         chkngx     }         notify_master "/etc/keepalived/notify.sh master"         notify_backup "/etc/keepalived/notify.sh backup"         notify_fault "/etc/keepalived/notify.sh fault"         notify "/etc/keepalived/notify.sh" }

master2为:

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49

vrrp_instance VI_1 {     state BACKUP     interface eth0     virtual_router_id 51     priority 100     advert_int 1     authentication {         auth_type PASS         auth_pass keepalivedpass     }     virtual_ipaddress {     172.16.100.99 dev eth0 label eth0:0     }     track_script {     chk_downfile     chkngx     }     track_script {     chkngx     }     notify_master "/etc/keepalived/notify.sh master"     notify_backup "/etc/keepalived/notify.sh backup"     notify_fault "/etc/keepalived/notify.sh fault"     notify "/etc/keepalived/notify.sh" } vrrp_instance VI_2 {     state MASTER     interface eth0     virtual_router_id 52     priority 101     advert_int 1     authentication {         auth_type PASS         auth_pass keepalivedpass     }     virtual_ipaddress {         172.16.100.98 dev eth0 label eth0:1     }     track_script {         chk_downfile     }     track_script {         chkngx     }         notify_master "/etc/keepalived/notify.sh master"         notify_backup "/etc/keepalived/notify.sh backup"         notify_fault "/etc/keepalived/notify.sh fault"         notify "/etc/keepalived/notify.sh" }