HAproxy指南之haproxy双机热备 (案例篇)

3ewsd 发表于 2019-1-1 08:09:12

　　 haproxy的出现正是弥补nginx一些应用上不足，比如session会话保持，健康监控检测机制，负载算法等等，在很多应用环境中，nginx的代理性能会haproxy稍逊一些，不过在一些实际案例中，keepalive+nginx与keepalive+nginx往往会根据业务去选择，比如nginx有着haproxy没有的代理缓存的功能等等，如果需要用到缓存就可以使用nginx，总之:根据业务来选择这两者！
Keepalive+haproxy双机热备
如图所示为整体的拓扑图：
　　

http://s5.运维网.com/wyfs02/M02/7D/CD/wKiom1bwounzcfJGAADA6EKsNq8901.png
一.部署前说明：
(1)系统版本: centos 6.6（64位）
(2)角色及ip相关信息:
角色名称网络ip信息客户端(CIP)10.58.137.203/24Master_DReth0:172.51.96.105/24 &&eth1:192.168.0.105/24Backup_DReth0:172.51.96.119/24 &&eth1:192.168.0.119/24VIP172.51.96.175/24(3)相关中间件信息
keepalive版本信息: keepalived-1.2.15
nginx版本信息:haproxy-1.5.15（提供proxy代理）
　　二.部署操作：

haproxy部署
分别在Master_DR和backup_DR上安装haproxy，操作如下：
　　1.1 到haproxy官网下载haproxy源码包如下
cd ~
wget http://www.haproxy.org/download/1.5/src/haproxy-1.5.15.tar.gz
　　1.2 创建haproxy运行用户

　　
groupadd -r haproxy
useradd -r -g haproxy -M -s /sbin/nologin haproxy
　　1.3 编译安装haproxy：
　　
cd ~
tar zxvf haproxy-1.5.15.tar.gz -C /usr/local/src
cd /usr/local/src/haproxy-1.5.15
make TARGET=linux26 PREFIX=/usr/local/haproxy
make install PREFIX=/usr/local/haproxy
　　注意：TARGET=Linux26 是通过uname -a 来查看Linux内核版本的
　　1.4 创建haproxy主配置文件：
mkdir /etc/haproxy/
vim /etc/haproxyhaproxy.cfg配置代码内容如下：（注意主备server上haproxy.cfg配置要一致）

#---------------------------------------------------------------------
# Global settings
#---------------------------------------------------------------------
global
   log 127.0.0.1 local3
   maxconn 204800
   chroot /usr/local/haproxy
   userhaproxy
   group haproxy
   daemon
   nbproc 1
   pidfile /var/run/haproxy.pid
   stats socket /usr/local/haproxy/stats
   description haproxy server
#---------------------------------------------------------------------
# common defaults that all the 'listen' and 'backend' sections will
# use if not designated in their block
#---------------------------------------------------------------------
defaults
   log global
   mode http
   maxconn 10000
   optionhttplog
   optionhttpclose
   optiondontlognull
   optionforwardfor    except 127.0.0.0/8
   retries 3
   option redispatch
   option abortonclose
   balance roundrobin
   timeout http-request 10s
   timeout queue       1m
   timeout connect       10s
   timeout client       1m
   timeout server       1m
   timeout http-keep-alive 10s
   timeout check       10s
#---------------------------------------------------------------------
# use listen setting the haproxy status for site
#---------------------------------------------------------------------
listen admin_status
   bind *:3030
   mode http
   log 127.0.0.1 local3 err
   stats refresh 5s
   stats uri /status
   stats realm www.skeryp.com
   stats auth admin:admin
   stats hide-version
   stats admin if TRUE
#---------------------------------------------------------------------
# main listen which proxys to the backends
#---------------------------------------------------------------------
listenwww
   bind *:80
   maxconn 5000
   mode http
   log global
   option httplog
   option httpclose
   option forwardfor
   log       global
   default_backend default
#
backend default
   mode http
   optionhttpchk GET /index.html
   server default 127.0.0.1:81 cookie id1 check inter 2000 rise 2 fall 3 maxconn 50001.5 创建haproxy系统服务启动脚本：
关于haproxy服务脚本代码请访问:http://blief.blog.运维网.com/6170059/1750573
1.6 启动haproxy服务：

server haproxy restart

Keepalive部署
(1)分别在Master_DR和backup_DR上安装keepalive，操作如下：
1. 安装Keepalive所需要的相关依赖包:
yum install openssl-devel popt-devel libnl-devel kernel-devel-y
　　2. 编译安装keepalive

1.1 keepalived的源码获取
keepalived源码包我们可以到keepalived的官网:http://www.keepalived.org/去下载，相关说明文档亦可在其官网查看，比如keepalived的使用，相关配置说明，这里演示的版本为:1.2.15
cd ~
wget http://www.keepalived.org/software/keepalived-1.2.15.tar.gz
1.2 编译安装keepalived

ln -s /usr/src/kernels/2.6.32-573.18.1.el6.x86_64/ /usr/src/linux
tar zxvf keepalived-1.2.15.tar.gz -C /usr/local/src
cd /usr/local/src/keepalived-1.2.15/
./configure \
--prefix=/usr/local/keepalived \
--with-kernel-dir=/usr/src/kernels/2.6.32-573.18.1.el6.x86_64
#2.6.32-573.18.1.el6.x86_64 这个需要根据kernel来匹配，一般安装了kernel-devel即可查看
makemake install

cp /usr/local/keepalived/sbin/keepalived /usr/sbin/
cp /usr/local/keepalived/etc/sysconfig/keepalived /etc/sysconfig/

cp /usr/local/keepalived/etc/rc.d/init.d/keepalived /etc/init.d/
chkconfig --add keepalived
chkconfig --level 2345 keepalived on

mkdir -p /etc/keepalived
cp /usr/local/keepalived/etc/keepalived/keepalived.conf /etc/keepalived(3)分别配置Master_DR以及Backup_DR上的keepalive实例，如下所示:
1. master_dr配置代码示例（主调度器）

vim /etc/keepalived/keepalived.conf内容如下
! Configuration File for keepalived
global_defs {
notification_email {
   admin@bluemobi.cn
}
notification_email_fromlvs_admin@bluemobi.cn
smtp_server 127.0.0.1
smtp_connect_timeout 30
router_id DR_MASTER
}
vrrp_script check_haproxy {                            #表示创建一个脚本check_haproxy
script "/etc/keepalived/scripts/check_haproxy.sh" #引用的脚本路径
interval 3                                        #表示检测时间的间隔为3s
}
vrrp_instance http {
state BACKUP
interface eth0
dont_track_primary
nopreempt                      #不抢占master
track_interface {             #表示需要检测的网卡
eth0
eth1
}
mcast_src_ip 172.51.96.105    #表示设置组播的源地址
garp_master_delay 6
virtual_router_id 60
priority 110
advert_int 1
authentication {
auth_type PASS
autp_pass 1234
}
virtual_ipaddress {
172.51.96.175/24 brd172.51.96.255 dev eth0 label eth0:1
}
virtual_routes {
172.51.96.175/24 dev eth0
}
track_script {
check_haproxy
}
notify_master /etc/keepalived/scripts/state_master.sh
notify_backup /etc/keepalived/scripts/state_backup.sh
notify_fault/etc/keepalived/scripts/state_fault.sh
}
2. backup_dr配置示例（备调度器）

vim /etc/keepalived/keepalived.conf内容如下
! Configuration File for keepalived
global_defs {
notification_email {
   admin@bluemobi.cn
}
notification_email_fromadmin@bluemobi.cn
smtp_server 127.0.0.1
smtp_connect_timeout 30
router_id DR_BACKUP
}
vrrp_script check_haproxy {
script "/etc/keepalived/scripts/check_haproxy.sh"
interval 3
}
vrrp_instance http {
state BACKUP
interface eth0
dont_track_primary
track_interface {
eth0
eth1
}
mcast_src_ip 172.51.96.119
garp_master_delay 6
virtual_router_id 60
priority 105
advert_int 1
authentication {
auth_type PASS
autp_pass 1234
}
virtual_ipaddress {
172.51.96.175/24 brd172.51.96.255 dev eth0 label eth0:1
}
virtual_routes {
172.51.96.175/24 dev eth0
}
track_script {
check_haproxy
}
notify_master /etc/keepalived/scripts/state_master.sh
notify_backup /etc/keepalived/scripts/state_backup.sh
notify_fault/etc/keepalived/scripts/state_fault.sh
}3.分别在主调度server和备调度server编写以下脚本，如下：
　　i 当调度器为切换master server时，记录切换时间日志
　　vim /etc/keepalived/scripts/state_master.sh
代码如下:
#!/bin/bash
echo -e>> $LOGFILE
host=CN-SH-DR01    #设置当前的主机名
LOGFILE="/var/log/keepalived-state.log"
echo "" >> $LOGFILE
date >> $LOGFILE
echo "The ${host}Starting to become master server...." >> $LOGFILE 2>&1
echo "Please run the “ipvsadm -Ln”check the keepalived state ..." >> $LOGFILE
echo ".........................................................................!">> $LOGFILE
echo >>$LOGFILE
ii 当调度器为切换backup server时，记录切换时间日志
vim /etc/keepalived/scripts/state_backup.sh
代码如下:
#!/bin/bash
echo -e >> $LOGFILE
host=CN-SH-DR01 #设置当前的主机名
LOGFILE="/var/log/keepalived-state.log"
echo "" >> $LOGFILE
date >> $LOGFILE
echo "The ${host}Starting to become Backup server...." >> $LOGFILE 2>&1
echo "Please run the “ipvsadm -Ln”check the state ..." >> $LOGFILE
echo "........................................................................!">> $LOGFILE
echo>> $LOGFILEiii当调度器出现错误时，记录错误时间日志
vim /etc/keepalived/scripts/state_fault.sh
代码如下:
#!/bin/bash
echo -e >> $LOGFILE
host=CN-SH-DR01    #设置当前的主机名
LOGFILE="/var/log/keepalived-state.log"
echo "" >> $LOGFILE
date >> $LOGFILE
echo "The ${host}is fault error...." >> $LOGFILE 2>&1
echo "Please check the server state ..." >> $LOGFILE
echo "........................................................................!">> $LOGFILE
echo>> $LOGFILEiiii 服务状态健康监测脚本,比如当haproxy不可用时，及时切换到backup调度器上
vim /etc/keepalived/scripts/check_haproxy.sh
#!/bin/bash
#nginx="/usr/local/haproxy/sbin/haproxy"
PID=`ps -C haproxy --no-heading|wc -l`
if [ "${PID}" = "0" ];
then
      /etc/init.d/haproxy start
sleep 3
LOCK=`ps -C haproxy --no-heading|wc -l`
if [ "${LOCK}" = "0" ];
then
   /etc/init.d/keepalived stop
fi
fi(4)依次重启master_dr,backup-dr上keepalive服务

# service keepalived restart

三.测试验证：
在两台调度器安装web服务并创建相关测试页，如下：
master-dr主调度服务器的测试页面
# curl 172.51.96.105:81
this is master server
#backup-dr备调度服务器的测试页面
# curl 172.51.96.119:81
this is backup server
#通过messages查看vip抢夺情况，如下所示:
http://s3.运维网.com/wyfs02/M00/7D/36/wKiom1bia2qxNj9bAACBZNMPRdE520.png
　　

　　述上发现vip已经被master-dr抢夺，通过ifconfig看到master-dr已经存在vip地址，如下
http://s2.运维网.com/wyfs02/M02/7D/36/wKiom1bibCfgBJrvAABOykm2OCk871.png
　　

　　在客户端通过:http://vip 就可以访问到页面，此时访问的是master-dr的页面：
http://s2.运维网.com/wyfs02/M02/7D/35/wKioL1bib0bzefxiAACq4cBzqe4846.png
　　

　　我们在master-dr将keepalived服务停止掉，来模仿主调度器宕机情形：

# service keepalived stop
Stopping keepalived:
#
　　

　　分别查看master-dr和backup-dr的keepalive日志：
http://s5.运维网.com/wyfs02/M01/7D/37/wKiom1bicyjikM6DAABxiZz2Ep0073.png
http://s5.运维网.com/wyfs02/M00/7D/35/wKioL1bidCDRjtU2AACa0NJM3SM827.png
　　

　　此时vip已经由backup-dr接管了，因为master-dr上nginx服务异常，而keepalive会定时触发引用的检查脚本
“check_nginx”检查nginx状态，发现nginx可用就停止了keepalive服务，从而使vip顺利的飘逸到backup-dr上；
　　查看keepalive-state.log,可以看到master-dr和backup-dr会记录每个状态的的信息：同时日志记录脚本也会记录相关信息：
# tail -10/var/log/keepalived-state.log
The CN-SH-DR01Starting to become Backup server....
Please run the “ipvsadm -Ln”check the state ...
...............................................................................!

Fri Mar 11 15:28:48 CST 2016
The CN-SH-DR01is fault error....
Please check the server state ...
...............................................................................!# tail -10 /var/log/keepalived-state.log

Fri Mar 11 14:42:56 CST 2016
The CN-SH-DR02Starting to become Backup server....
Please run the “ipvsadm -Ln”check the state ...
...............................................................................!

Fri Mar 11 15:06:58 CST 2016
The CN-SH-DR02Starting to become master server....
Please run the “ipvsadm -Ln”check the state ...
...............................................................................!　　

　　再次访问http://vip 发现页面为backup-dr的页面：
http://s2.运维网.com/wyfs02/M01/7D/37/wKiom1bidCbwUT0vAADfkh2tNno096.png
　　

　　到这里，整个keepalive+haproxy双机热备就部署完成了
　　

　　总结:关于keepalive+haproxy双机热备适合很多web前端高可用集群应用，相对于keepalive+lvs案例应用，配置起来更方便，更简单，与它同样的经典应用案例：keepalive+nginx也是一个不错的双机热备方案，通常在需要高性能高可用反向代理场合，keepalive+haproxy是一个不错的选择方案。
　　

　　

页: [1]

运维网's Archiver

HAproxy指南之haproxy双机热备 (案例篇)