keepalived入门之keepalive+nginx实例部署

3refee

在实际环境中，keepalive常常与lvs，nginx，haproxy，Mysql等等应用组成高可用计算集群服务，比如web前端应用等等场景，接下来一起来讲讲关于keepalive+lvs实例部署

Keepalive+Lvs（lvs/dr模式）实例部署

如图所示为整体的拓扑图：

一.部署前说明：

(1)系统版本: centos 6.6（64位）

(2)角色及ip相关信息:

角色名称	网络ip信息
客户端(CIP)	10.58.137.203/24
Lvs_Master_DR	eth0:172.51.96.105/24 &&  eth1:192.168.0.105/24
Lvs_Backup_DR	eth0:172.51.96.119/24 &&  eth1:192.168.0.119/24
LVS_VIP	192.168.0.88/32

(3)相关中间件信息

keepalive版本信息: keepalived-1.2.15

httpd版本信息:   nginx1.6.0  （提供http服务）

二.部署操作：

负载均衡器上配置操作

(1)分别在Lvs_Master_DR和Lvs_backup_DR上安装Keepalive所需要的相关依赖包:

# yum install openssl-devel popt-devel libnl-devel kernel-devel  -y
2. 编译安装keepalive

1.1 keepalived的源码获取

keepalived源码包我们可以到keepalived的官网:http://www.keepalived.org/去下载，相关说明文档亦可在其官网查看，比如keepalived的使用，相关配置说明，这里演示的版本为:1.2.15

# cd ~
# wget http://www.keepalived.org/software/keepalived-1.2.15.tar.gz

1.2 编译安装keepalived

<--编译安装keepalived-->

# ln -s /usr/src/kernels/2.6.32-573.18.1.el6.x86_64/ /usr/src/linux
# tar zxvf keepalived-1.2.15.tar.gz -C /usr/local/src
# cd /usr/local/src/keepalived-1.2.15/
# ./configure \
  --prefix=/usr/local/keepalived \
  --with-kernel-dir=/usr/src/kernels/2.6.32-573.18.1.el6.x86_64
# make  make install

<--对keepalived进行相关路径优化调整-->

<---拷贝keepalived相关启动命令--->
# cp /usr/local/keepalived/sbin/keepalived /usr/sbin/
# cp /usr/local/keepalived/etc/sysconfig/keepalived /etc/sysconfig/

<---将keepalived启动脚本添加到系统服务--->
# cp /usr/local/keepalived/etc/rc.d/init.d/keepalived /etc/init.d/
# chkconfig --add keepalived
# chkconfig --level 2345 keepalived on

<---创建keepalived相关配置文件--->
# mkdir -p /etc/keepalived
# cp /usr/local/keepalived/etc/keepalived/keepalived.conf /etc/keepalived

备注说明：
1.keepalived安装完成后，相关路径如下:
安装目录为:/usr/lccal/keepalived, 配置文件路径为:/etc/keepalive/
2.安装完成后，需要将启动脚本复制到/etc/ini.d/下
3.注意一定要执行述上的相关操作，不然有可能导致keepalived服务起不来

1.3  启动keepalived服务

# service keepalived start

(3)分别配置Lvs_Master_DR以及Lvs_Backup_DR上的keepalive实例，如下所示:

1. Lvs_master_dr配置代码示例（主调度器）

vim /etc/keepalived/keepalived.conf

内容如下

! Configuration File for keepalived

global_defs {
   notification_email {
       admin@bluemobi.cn
   }
   notification_email_from  lvs_admin@bluemobi.cn
   smtp_server 127.0.0.1
   smtp_connect_timeout 30
   router_id DR_MASTER
}

vrrp_script check_nginx {
   script "/etc/keepalived/scripts/check_nginx.sh"
   interval 3
   weight -5
}

############################################################################################
# vrrp_script check_nginx {                         #表示创建一个脚本check_nginx
   script "/etc/keepalived/scripts/check_nginx.sh"  #引用的脚本路径
   interval 3  #表示检测时间的间隔为3s
   weight -5   #当脚本执行结果失败，则优先级降低5级
}
############################################################################################

vrrp_instance http {
    state BACKUP
    interface eth0
    lvs sync daemon interface eth0
############################################################################################
# lvs sync daemon interface eth0:类似HA心跳检测的端口
############################################################################################
    dont_track_primary
    nopreempt
############################################################################################
# nopreempt: 不抢占master
############################################################################################
    track_interface {
    eth0
    eth1
    }
############################################################################################
# track_interface {  }:表示需要检测的网卡
############################################################################################
    mcast_src_ip 172.51.96.105
############################################################################################
# track_interface :表示设置组播的源地址
############################################################################################
    garp_master_delay 6
    virtual_router_id 60
    priority 110
    advert_int 1

    authentication {
    auth_type PASS
    autp_pass 1234
    }

    virtual_ipaddress {
    172.51.96.175/32 brd 192.168.0.88 dev eth0 label eth0:1
    }

    virtual_routes {
    172.51.96.175/32 dev eth0
    }

    track_script {
    check_nginx
    }

    notify_master /etc/keepalived/scripts/state_master.sh
    notify_backup /etc/keepalived/scripts/state_backup.sh
    notify_fault  /etc/keepalived/scripts/state_fault.sh
}
############################################################################################
# notify_master :表示当前调度器竞选为master server时需要调用的脚本
# notify_backup :表示当前调度器竞选为backup server时需要调用的脚本
# notify_fault  :表示当前调度器竞选为出现问题时（比如网卡down）需要调用的脚本
# notify_stop   :表示当前调度器竞服务停止时（比如keepalive服务down）需要调用的脚本
############################################################################################

2 Lvs_backup_dr配置示例（备调度器）

vim /etc/keepalived/keepalived.conf

内容如下

! Configuration File for keepalived

global_defs {
   notification_email {
       admin@bluemobi.cn
   }
   notification_email_from  lvs_admin@bluemobi.cn
   smtp_server 127.0.0.1
   smtp_connect_timeout 30
   router_id DR_BACKUP
}

vrrp_script check_nginx {
   script "/etc/keepalived/scripts/check_nginx.sh"
   interval 3
   weight -5
}

vrrp_instance http {
    state BACKUP
    interface eth0
    lvs sync daemon interface eth0
    dont_track_primary
    nopreempt

    track_interface {
    eth0
    eth1
    }

    mcast_src_ip 172.51.96.119
    garp_master_delay 6
    virtual_router_id 60
    priority 109
    advert_int 1

    authentication {
    auth_type PASS
    autp_pass 1234
    }

    virtual_ipaddress {
    172.51.96.175/32 brd 192.168.0.88 dev eth0 label eth0:1
    }

    virtual_routes {
    172.51.96.175/32 dev eth0
    }

    track_script {
    check_nginx
    }

    notify_master /etc/keepalived/scripts/state_master.sh
    notify_backup /etc/keepalived/scripts/state_backup.sh
    notify_fault  /etc/keepalived/scripts/state_fault.sh
}

3.分别在主调度server和备调度server编写以下脚本，如下：

i 当调度器为切换master server时，记录切换时间日志

vim /etc/keepalived/scripts/state_master.sh

代码如下:

#!/bin/bash
echo -e  >> $LOGFILE
host=CN-SH-DR01      #设置当前的主机名
LOGFILE="/var/log/keepalived-state.log"
echo "[Master]" >> $LOGFILE
date >> $LOGFILE
echo "The ${host}  Starting to become master server...." >> $LOGFILE 2>&1

echo "Please run the “ipvsadm -Ln”  check the keepalived state ..." >> $LOGFILE
echo ".........................................................................!">> $LOGFILE
echo >>$LOGFILE

ii 当调度器为切换backup server时，记录切换时间日志

vim /etc/keepalived/scripts/state_backup.sh

代码如下:

#!/bin/bash
echo -e >> $LOGFILE
host=CN-SH-DR01     #设置当前的主机名
LOGFILE="/var/log/keepalived-state.log"
echo "[Backup]" >> $LOGFILE
date >> $LOGFILE
echo "The ${host}  Starting to become Backup server...." >> $LOGFILE 2>&1

echo "Please run the “ipvsadm -Ln”  check the state ..." >> $LOGFILE
echo "........................................................................!">> $LOGFILE
echo  >> $LOGFILE

iii  当调度器出现错误时，记录错误时间日志

vim /etc/keepalived/scripts/state_fault.sh

代码如下:

#!/bin/bash
echo -e >> $LOGFILE
host=CN-SH-DR01      #设置当前的主机名
LOGFILE="/var/log/keepalived-state.log"
echo "[fault errot ]" >> $LOGFILE
date >> $LOGFILE
echo "The ${host}  is fault error...." >> $LOGFILE 2>&1
echo "Please check the server state ..." >> $LOGFILE
echo "........................................................................!">> $LOGFILE
echo  >> $LOGFILE

服务状态健康监测脚本

，比如nginx和keepalive在同一台server上，当nginx不可用时，

#!/bin/bash
#nginx="/usr/local/nginx/sbin/nginx"

PID=`ps -C nginx --no-heading|wc -l`
if [ "${PID}" = "0" ];
    then
         /etc/init.d/nginx start
    sleep 3
    LOCK=`ps -C nginx --no-heading|wc -l`
    if [ "${LOCK}" = "0" ];
    then
        /etc/init.d/keepalived restart
    fi
fi

1.3 重新启动keepalived服务

# service keepalived restart