ubuntu+haproxy+keepalived主从负载
ubuntu+haproxy+keepalived主从负载 一.测试环境：系统：ubuntu server 12.04 haproxy版本：1.4.24keepalived版本：keepalived-1.2.7haporxy01：eth0：172.16.1.36eth1：192.168.100.36haporxy02：eth0：172.16.1.37eth1：192.168.100.37vip1：172.16.1.30vip2：172.16.1.31nignx1 website显示：nginx1nginx2 website显示：ningx2nginx3 website显示：nginx3nginx4 website显示：nginx4 二.网络结构 user | | (vip1) | (vip2) haproxy01-------keepalived-------haproxy02 / \ / \ / \ / \ / \ / \ / \ / \ -------------------------------------------------------- | nginx1 nginx2 nginx3 nginx4 | --------------------------------------------------------三.安装1.安装keepalvied主机haproxy01： wget http://www.keepalived.org/software/keepalived-1.2.7.tar.gztar xf keepalived-1.2.7.tar.gzcd keepalived-1.2.7./configure --prefix=/usr/local/ 提示：checking for openssl/ssl.h... noconfigure: error: !!! OpenSSL is not properly installed on your system. !!!!!! Can not include OpenSSL headers files. !!! 安装libssl.devapt-get install libssl.dev 继续：./configure --prefix=/usr/local/ 提示：checking for poptGetContext in -lpopt... noconfigure: error: Popt libraries is required 安装libpopt-devapt-get install libpopt-dev 继续：./configure --prefix=/usr/local/makemake install 2.编辑keepalived.confmkdir /etc/keepalived/vi /etc/keepalived/keepalived.conf ########keepalived.conf########global_defs { router_id LVS_DEVEL} vrrp_instance VI_1 { state MASTER# haproxy02:BACKUP interface eth0 virtual_router_id 51 priority 91 # 比haproxy02大 advert_int 1 authentication { auth_type PASS auth_pass 123456789 } virtual_ipaddress { 172.16.1.30 } vrrp_instance VI_2 { state BACKUP# haproxy02:MASTER interface eth0 virtual_router_id 52 priority 90 # 比haproxy02小 advert_int 1 authentication { auth_type PASS auth_pass 123456789 } virtual_ipaddress { 172.16.1.31 }########keepalived.conf######## 3.启动keepalived/usr/local/sbin/keepalvied -f /etc/keepalived/keepalived.conf4.查看vipip addr显示：eth0： inet 172.16.1.30/32 scope global eth0 inet 172.16.1.31/32 scope global eth0 #（haproxy02的keepalived未启动，vip2在haproxy01中，haproxy02中keepalived启动后，vip2自动跳转到haproxy02中eth0上） 主机haproxy02： keepalived安装同上，keepalived.conf如下： ########keepalived.conf########global_defs { router_id LVS_DEVEL} vrrp_instance VI_1 { state BACKUP# haproxy01:MASTER interface eth0 virtual_router_id 51 priority 90 # 比haproxy01小 advert_int 1 authentication { auth_type PASS auth_pass 123456789 } virtual_ipaddress { 172.16.1.30 } vrrp_instance VI_2 { state MASTER# haproxy01:BACKUP interface eth0 virtual_router_id 52 priority 91 # 比haproxy01大 advert_int 1 authentication { auth_type PASS auth_pass 123456789 } virtual_ipaddress { 172.16.1.31 }########keepalived.conf######## 启动keepalived/usr/local/sbin/keepalvied -f /etc/keepalived/keepalived.conf查看vip：ip addr显示：eth0： inet 172.16.1.31/32 scope global eth0 # (haproxy中eth0上vip2消失，只有vip1) 停止任意一台keepalived服务，另外一台均可自动生成vip1和vip2，确保2台haproxy高可用性 2.安装haproxywget http://haproxy.1wt.eu/download/1.4/src/haproxy-1.4.24.tar.gztar xf haproxy-1.4.24.tar.gz cd haproxy-1.4.24查看ubuntu版本信息：uname -a显示：Linux ubuntu37 3.2.0-51-generic #77-Ubuntu SMP Wed Jul 24 20:18:19 UTC 2013 x86_64 x86_64 x86_64 GNU/Linuxmake TARGET=37-ubuntumake PREFIX=/usr/local/haproxy install建立haproxy用户 useradd haproxy 2.编辑haproxy.cfgmkdir /etc/haproxyvi /etc/haproxy/haproxy.cfg ########haproxy.cfg########global log 127.0.0.1 local0 maxconn 51200 user haproxy group haproxy daemon defaults log127.0.0.1 local3 mode http option dontlognull balance roundrobin retries 2 option redispatch option abortonclose maxconn 51200 contimeout 5000 clitimeout 50000 srvtimeout 50000 listen haproxy01 bind 172.16.1.30:80 mode http optionhttpclose option forceclose optionforwardfor option originalto optionredispatch balance leastconn timeout check 5s stats uri /stats # haproxy负载监控页面 例：http://172.16.1.30/stats stats refresh 15s # 监控页面刷新时间 stats realm baison-test-Haproxy stats auth admin:admin # 监控页面账号密码 stats hide-version # 隐藏haproxy版本信息 cookieSESSION_COOKIEinsert indirect nocachemaxconn 40960 server nginx1 192.168.100.11:80 weight 1 cookie nginx1 check inter 2000 rise 2 fall 3 server nginx2 192.168.100.12:80 weight 1 cookie nginx2 check inter 2000 rise 2 fall 3 listen haproxy02 bind 172.16.1.31:80 mode http optionhttpclose option forceclose optionforwardfor option originalto optionredispatch balance leastconn cookieSESSION_COOKIEinsert indirect nocache maxconn 40960 server nginx3 192.168.100.13:80 weight 1 cookie nginx3 check inter 2000 rise 2 fall 3 server nginx4 192.168.100.14:80 weight 1 cookie nginx4 check inter 2000 rise 2 fall 3########haproxy.cfg########启动haproxy：/usr/local/haproxy/sbin/haproxy -f /etc/haproxy/haproxy.cfg 主机haproxy02：安装同上，haproxy.cfg同上启动haproxy：/usr/local/haproxy/sbin/haproxy -f /etc/haproxy/haproxy.cfg 四：测试通过浏览器访问：http://172.16.1.30页面显示：nginx1、nginx2轮询切换http://172.16.1.31页面显示：nginx3、nginx4轮询切换可通过监控页面查看负载情况：http://172.16.1.30/stats 停止任意一台haproxy服务，站点均不受影响 五：haproxy日志1.在/etc/rsyslog.conf中：添加haproxy日志路径:local0.* /var/log/haproxy.log local3.* /var/log/haproxy.log取消注释:#$ModLoad imudp==》$ModLoad imudp#$UDPServerRun 514 ==》$UDPServerRun 514重启rsyslog服务service rsyslog restart 2.核实services文件grep 514 /etc/services显示：syslog 514/udp 为ok!!!如不存在，手动添加：vi /etc/default/rsyslogRSYSLOGD_OPTIONS="-r -c 5"重启syslog服务，service rsyslog restart 六：优化haproxy服务器1.内核结果如下：sysctl -pnet.ipv4.ip_forward = 1net.ipv4.ip_nonlocal_bind = 1net.ipv4.tcp_syncookies = 1net.ipv4.tcp_tw_reuse = 1net.ipv4.ip_local_port_range = 1024 65535net.nf_conntrack_max = 1310720net.ipv4.tcp_tw_reuse = 1net.ipv4.tcp_fin_timeout = 15net.core.netdev_max_backlog = 4096net.core.rmem_max = 16777216net.core.somaxconn = 1310720net.core.wmem_max = 16777216net.ipv4.tcp_max_syn_backlog = 20480net.ipv4.tcp_max_tw_buckets = 400000net.ipv4.tcp_no_metrics_save = 1net.ipv4.tcp_rmem = 4096 87380 16777216net.ipv4.tcp_syn_retries = 2net.ipv4.tcp_synack_retries = 2net.ipv4.tcp_wmem = 4096 65536 16777216vm.min_free_kbytes = 65536net.ipv4.tcp_sack = 1net.ipv4.tcp_timestamps = 1net.ipv4.tcp_tw_recycle = 1拷贝至sysctl.conf中vi /etc/sysctl.conf 2.ulimit数值永久化执行：ulimit -SHn显示：1024永久化：a.在/etc/pam.d/login中，添加：pam_limits.so (有时候系统默认添加)查找文件位置： find / -name pam_limits.so显示：/lib/x86_64-linux-gnu/security/pam_limits.sob.在/etc/security/limits.conf中，添加：root soft nofile 10240 #实际值root hard nofile 10240 #实际值c.修改 /etc/rc.local 添加：echo 8061540 > /proc/sys/fs/file-max执行：echo 8061540 > /proc/sys/fs/file-max 3.nginx服务器内核参数优化sysctl -pnet.ipv4.tcp_fin_timeout = 30net.ipv4.tcp_keepalive_intvl = 2net.ipv4.tcp_keepalive_probes = 2net.ipv4.tcp_keepalive_time = 120net.ipv4.tcp_syn_retries = 10net.ipv4.tcp_sack = 1net.ipv4.tcp_timestamps = 1
页:
[1]