Haproxy+Keepalived+MySQL实现读均衡负载

sszxf

环境说明：
     本系统对数据安全性不高，TPS操作很低，主要是查询的环境。
本方案架构图如下：     应用服务器对只读的应用程序连接虚拟IP地址，连接到haproxy，然后通过haproxy将TCP协议转移到下面的3个数据库服务器中。Haproxy在此做4层的TCP交换服务。keepalived为了防止haproxy单点故障。      为什么不用LVS？    我们的环境服务器少，haproxy和lvs相比性能差不多，且haproxy自带有mysql check，不需要额外的写mysql侦测脚本。


1.haproxy安装

下载地址：http://www.haproxy.org/download/1.5/src/haproxy-1.5.5.tar.gz
依赖包安装：#yum install gcc gcc-c++ make zlib-devel bzip2-devel openssl-devel

安装pcre，自带的pcre-7.8-3.1.el6.x86_64貌似版本低了，导致无法安装proxy
#tar xzvf pcre-8.34.tar.gz
#cd pcre-8.34#./configure --prefix=/usr \--docdir=/usr/share/doc/pcre-8.34 \--enable-utf --enable-unicode-properties \--enable-pcregrep-libz --enable-pcregrep-libbz2#make#make check#make install
haproxy安装：
#tar -xzvf haproxy-1.5.5.tar.gz#cd haproxy-1.5.5# make TARGET=linux26 USE_STATIC_PCRE=1 \USE_REGPARM=1 USE_LINUX_TPROXY=1 USE_OPENSSL=1 USE_ZLIB=1 ARCH=x86_64# make install
--编辑启动文件# vi /etc/init.d/haproxy   --添加以下内容#!/bin/sh## custom haproxy init.d script, by Mattias Geniar## haproxy         starting and stopping the haproxy load balancer## chkconfig: 345 55 45# description: haproxy is a TCP loadbalancer# probe: true # Source function library.. /etc/rc.d/init.d/functions # Source networking configuration.. /etc/sysconfig/network # Check that networking is up.[ ${NETWORKING} = "no" ] && exit 0 [ -f /usr/local/sbin/haproxy ] || exit 0 [ -f /etc/haproxy/haproxy.conf ] || exit 0 # Define our actionscheckconfig() {        # Check the config file for errors        /usr/local/sbin/haproxy -c -q -f /etc/haproxy/haproxy.conf        if [ $? -ne 0 ]; then                 echo "Errors found in configuration file."                return 1        fi         # We're OK!        return 0} start() {        # Check config        /usr/local/sbin/haproxy -c -q -f /etc/haproxy/haproxy.conf        if [ $? -ne 0 ]; then                echo "Errors found in configuration file."                return 1        fi         echo -n "Starting HAProxy: "        daemon /usr/local/sbin/haproxy -D -f /etc/haproxy/haproxy.conf -p /var/run/haproxy.pid         RETVAL=$?        echo        [ $RETVAL -eq 0 ] && touch /var/lock/subsys/haproxy        return $RETVAL} stop() {        echo -n "Shutting down HAProxy: "        killproc haproxy -USR1         RETVAL=$?        echo        [ $RETVAL -eq 0 ] && rm -f /var/lock/subsys/haproxy        [ $RETVAL -eq 0 ] && rm -f /var/run/haproxy.pid        return $RETVAL} restart() {        /usr/local/sbin/haproxy -c -q -f /etc/haproxy/haproxy.conf        if [ $? -ne 0 ]; then                echo "Errors found in configuration file."                return 1        fi         stop        start} check() {        /usr/local/sbin/haproxy -c -q -V -f /etc/haproxy/haproxy.conf} rhstatus() {        status haproxy} reload() {        /usr/local/sbin/haproxy -c -q -f /etc/haproxy/haproxy.conf        if [ $? -ne 0 ]; then                echo "Errors found in configuration file."                return 1        fi         echo -n "Reloading HAProxy config: "        /usr/local/sbin/haproxy -f /etc/haproxy/haproxy.conf -p /var/run/haproxy.pid -sf $(cat /var/run/haproxy.pid)         success $"Reloading HAProxy config: "        echo} # Possible parameterscase "$1" in  start)        start        ;;  stop)        stop        ;;  status)        rhstatus        ;;  restart)        restart        ;;  reload)        reload        ;;  checkconfig)        check        ;;  *)        echo "Usage: haproxy {start|stop|status|restart|reload|checkconfig}"        exit 1esac exit 0
授权：#chmod +x /etc/init.d/haproxy


编辑haproxy配置文件：

可以参考说明：http://www.cnblogs.com/dkblog/archive/2012/03/13/2393321.html
#mkdir /etc/haproxy#vi /etc/haproxy/haproxy.confdefaults        log global        mode http        retries 3        option redispatch        maxconn 4096        timeout connect 50000        timeout client 50000        timeout server 50000 listen mysql_proxy 0.0.0.0:3307        mode tcp        balance roundrobin        option tcpka        option httpchk        option mysql-check user haproxy    #在mysql中创建无任何权限用户haproxy，且无密码        server mysqldb1 192.168.231.8:3306 weight 1        server mysqldb2 192.168.231.9:3306 weight 3        server mysqldb3 192.168.231.11:3306 weight 3listen stats *:8080       mode http       option httpclose       balance roundrobin       stats uri /       stats realm Haproxy\ Statistics       stats auth admin:admin

启动并检查：#service haproxy start# netstat -plantu | grep 3307tcp   0   0 0.0.0.0:3307    0.0.0.0:*   LISTEN  23626/haproxy


创建100次短连接测试：
其中db2和db3路由了43次，db1路由了14次

2.keepalived安装
下载地址：http://www.keepalived.org/software/
#tar zxvf keepalived-1.2.13.tar.gz#cd keepalived-1.2.13# ln -s /usr/src/kernels/2.6.32-220.el6.x86_64 /usr/src/linux#./configure --prefix=/ --mandir=/usr/local/share/man/ --with-kernel-dir=/usr/src/kernels/2.6.32-220.el6.x86_64#make#make install# cd /etc/keepalived/
配置参数文件：# mv keepalived.conf keepalived.conf.default#vi keepalived.conf! Configuration File for keepalived
vrrp_script chk_http_port {script "/etc/keepalived/check_haproxy.sh"interval 2weight 2

global_defs {router_id LVS_DEVEL}
vrrp_instance VI_1 {state MASTER interface bond0virtual_router_id 51priority 150 advert_int 1authentication {auth_type PASSauth_pass 1111}  track_script {chk_http_port}  virtual_ipaddress {192.168.231.18 }}}
配置侦测haproxy状态脚本：#vi /etc/keepalived/check_haproxy.sh#!/bin/bashA=`ps -C haproxy --no-header |wc -l`if [ $A -eq 0 ];then  /usr/local/sbin/haproxy -f /etc/haproxy/haproxy.confsleep 3
if [ `ps -C haproxy --no-header |wc -l` -eq 0 ];then  /etc/init.d/keepalived stopfifi
启动keepalived服务：# /etc/init.d/keepalived start

添加新的协议地址：#ip add[iyunv@mss-dn03 ~]# ip add1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN     link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00    inet 127.0.0.1/8 scope host lo    inet6 ::1/128 scope host        valid_lft forever preferred_lft forever2: eth0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq state DOWN qlen 1000    link/ether 90:e2:ba:23:87:04 brd ff:ff:ff:ff:ff:ff3: eth1: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq state DOWN qlen 1000    link/ether 90:e2:ba:23:87:05 brd ff:ff:ff:ff:ff:ff
9: bond0: <BROADCAST,MULTICAST,MASTER,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP     link/ether 6c:ae:8b:26:52:3b brd ff:ff:ff:ff:ff:ff    inet 192.168.231.7/24 brd 192.168.231.255 scope global bond0    inet6 fe80::6eae:8bff:fe26:523b/64 scope link tentative dadfailed        valid_lft forever preferred_lft forever
设置开启自启动：# chkconfig  --level 0123456 keepalived on# chkconfig --list keepalived

测试过程中发现，master宕机后，VIP需要几十秒才能ping通，切换很慢：arping -I bond0 -c 5 -s VIP GATEWAY# arping -I bond0 -c 5 -s 192.168.231.18 192.168.231.254


Getting started with HAProxy and MySQL replication（能实现读写路由）：
    http://blog.yannickjaquier.com/linux/getting-started-with-haproxy-and-mysql-replication.htmlHaproxy配置日志：    http://www.iyunv.com/Linux/2012-06/63585.htm