Lvs + Ngnix + Haproxy + Keepalived + Tomcat 实现三种HA软负载均衡和Tomcat Session共享
环境准备：一、11台测试机器
hostname:v1 ~ v10 (10台测试机)
ip:192.168.33.81(v1) ~ 192.168.33.90(v101)
由于在内网测试，需要搭建个内网yum源，方便安装软件。yum所在机器为192.168.33.101
二、待实现功能
下面分别使用haproxy/nginx/lvs实现HA + 负载均衡，软件环境如下：
v1:33.81 nginx
v2:33.82 nginx
v3:33.83 tomcat
v4:33.84 tomcat
v5:33.85 tomcat
v6:33.86 tomcat
v7:33.87 haproxy
v8:33.88 haproxy
v9:33.89 lvs
v10:33.90 lvs
1. 安装依赖包
yum -y install pcre-devel yum -y install openssl-devel yum -y install gcc yum -y install lrzsz yum -y install openssh-clients
2. 安装nginx
2.1. 上传、解压、重命名nginx
su - root
1、cd /usr/local
2、rz -y
3、tar -xzvf nginx-1.7.7.tar.gz
4、mv nginx-1.7.
1、进入解压后的目录，指定安装路径， 注：不指定prefix,则可执行文件默认放在/usr /local/bin,库文件默认放在/usr/local/lib,配置文件默认放在/usr/local/etc cd /usr/local/nginx ./configure --prefix=/usr/local/nginx --conf-path=/usr/local/nginx/nginx.conf 2、编译： make 3、安装： make install 4、启动 /usr/local/nginx/sbin/nginx 5、查看 http://192.168.xx.xxx 出现：welcome Nginx，就安装ok了。 6、停止 /usr/local/nginx/sbin/nginx –s stop
2.2. 安装nginx
2.3. 安装jdk
1、su - root 用户 2、进入usr目录 cd /usr 3、在usr目录下建立java安装目录 mkdir –m 755 java 4、将jdk-6u24-linux-i586.bin拷贝到java目录下 rz -y 4、安装jdk cd /usr/java chmod 755 jdk-6u24-linux-i586.bin ./jdk-6u24-linux-i586.bin‘ 注意：如果出现/lib/ld-linux.so.2: bad ELF interpreter:No such file or directory，安装下glic即可： yum install glibc.i686 5、安装完毕为他建立一个链接以节省目录长度 ln -s /usr/java/jdk1.6.0_24//usr/jdk 6、配置环境变量 vim /etc/profile 添加内容： vi /etc/profile export JAVA_HOME=/usr/jdk export PATH=$PATH:$JAVA_HOME/bin export CLASSPATH=.:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar export JAVA_HOME PATH CLASSPATH 7、执行下命令（source命令也称为“点命令”，也就是一个点符号（.）。source命令通常用于重新执行刚修改的初始化文件，使之立即生效，而不必注销并重新登录。） source /etc/profile
2.4. 安装tomcat
1、上传、解压： rz -y tar -zxvf apache-tomcat-6.0.37.tar.gz 2、重命名： mv apache-tomcat-6.0.37 tomcat
2.5. 重新配置nginx
1、cd /usr/local/nginx 2、vi /usr/local/nginx/nginx.conf
添加：
usernobody nobody;#定义Nginx运行的用户和用户组 worker_processes4;#nginx进程数，建议设置为等于CPU总核心数。 error_loglogs/error.loginfo;#全局错误日志定义类型，[ debug | info | notice | warn | error | crit ] worker_rlimit_nofile 1024;#一个nginx进程打开的最多文件描述符数目，所以建议与ulimit -n的值保持一致。 pidlogs/nginx.pid;#进程文件#工作模式及连接数上限 events { use epoll;#参考事件模型，use [ kqueue | rtsig | epoll |/dev/poll | select | poll ]; epoll模型是Linux2.6以上版本内核中的高性能网络I/O模型 worker_connections1024;#单个进程最大连接数（最大连接数=连接数*进程数） }#设定http服务器，利用它的反向代理功能提供负载均衡支持 http { include mime.types;#文件扩展名与文件类型映射表 default_typeapplication/octet-stream;#默认文件类型 #设定负载均衡的服务器列表 upstreamtomcatxxxcom{ server 192.168.56.200:8080; server 192.168.56.201:8080; } #设定日志格式 log_formatwww_xy_com'$remote_addr - $remote_user [$time_local] "$request" ' '$status $body_bytes_sent "$http_referer" ' '"$http_user_agent" "$http_x_forwarded_for"'; sendfile on;#开启高效文件传输模式，sendfile指令指定nginx是否调用sendfile函数来输出文件，对于普通应用设为 on，如果用来进行下载等应用磁盘IO重负载应用，可设置为off，以平衡磁盘与网络I/O处理速度，降低系统的负载。注意：如果图片显示不正常把这个改成off。 keepalive_timeout65;#长连接超时时间，单位是秒#gzipon; #设定虚拟主机，默认为监听80端口 server { listen 80; server_nametomcat.xxx.com;#域名可以有多个，用空格隔开#charset koi8-r; #设定本虚拟主机的访问日志 access_log/data/logs/access.logwww_xy_com; #对 "/" 启用反向代理 location /{ proxy_pass http://tomcatxxxcom; proxy_set_header Host $host; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-For$proxy_add_x_forwarded_for; }#error_page 500 502 503 504/50x.html; location =/50x.html { root html; } } }
3、创建logs所需要的文件夹/data /logs/
cd / mkdir –m 755 data cd data mkdir –m 755 logs
4、启动tomcat、nginx。
/usr/local/tomcat/bin/startup.sh /usr/local/nginx/sbin/nginx
5、修改hosts，加入
192.168.56.99 tomcat.xxx.com
6、访问http://redis.xxy.com
上面部分安装，我是用脚本装的。没按照上面来，使用上面配置时，启动nginx时，提示我" 7458#0: unknown directive "<feff>user" in /usr/local/nginx/nginx.conf:1",
于是，我就直接拿着nginx.conf.default修改了，改改也能用。下面是我修改后的配置(v1下：/usr/local/nginx/nginx.conf):
worker_processes1;error_loglogs/error.loginfo;pid logs/nginx.pid; events { use epoll; worker_connections1024;} http { include mime.types; default_typeapplication/octet-stream;# 配置需要代理的服务器列表 upstream servers { server v3:8080; server v4:8080;} log_formatmain'$remote_addr - $remote_user [$time_local] "$request" ''$status $body_bytes_sent "$http_referer" ''"$http_user_agent" "$http_x_forwarded_for"'; access_loglogs/access.logmain; sendfile on; keepalive_timeout65; server { listen 80; server_namelocalhost; access_loglogs/host.access.logmain; location /{#引用上面的配置 proxy_pass http://servers; proxy_set_header Host $host; proxy_set_header X-Real_IP $remote_addr; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; root html; indexindex.html index.htm;} error_page 500502503504/50x.html; location =/50x.html { root html;}} }
访问v1:80，使用rr策略自动轮巡v3,v4效果图：
http://blog.iyunv.com/u010811257/article/details/
http://blog.iyunv.com/u010811257/article/details/
对了，上面那句v3,v4的提示语，是我echo一句话把tomcat默认的ROOT/index.jsp 给覆盖了,用作测试
http://blog.iyunv.com/u010811257/article/details/
1.首先使用keepalived + nginx + tomcat 实现
测试过程中发现，我收到将nginx kill之后keepalived居然ip不飘，检查了配置也没发现问题。唯一能让keepavlied执行notify.sh脚本时候就是在启动kp时，
会检查nginx是否存活，不存活则启动。 现在已经实现了
于是又加了个脚本单独去监控kp ,脚本名称为：monitor.sh,代码在下边
v1的keepalived脚本:
!ConfigurationFilefor keepalived global_defs {} vrrp_script chk_nginx { script "killall -0 nginx" interval 1 weight -2} vrrp_instance VI_1 { state MASTER interface eth0 virtual_router_id 1 priority 100 advert_int 1 authentication { auth_type PASS auth_pass 1111} track_script { chk_nginx} virtual_ipaddress {192.168.33.181/24} notify_master "/etc/keepalived/notify.sh master" notify_backup "/etc/keepalived/notify.sh backup" notify_fault "/etc/keepalived/notify.sh fault"} vrrp_instance VI_2 { state BACKUP interface eth0 virtual_router_id 2 priority 98 advert_int 1 authentication { auth_type PASS auth_pass 1111} track_script { chk_nginx} virtual_ipaddress {192.168.33.182/24} notify_master "/etc/keepalived/notify.sh master" notify_backup "/etc/keepalived/notify.sh backup" notify_fault "/etc/keepalived/notify.sh fault"}
v2的keepalived脚本：
!ConfigurationFilefor keepalived global_defs {} vrrp_script chk_nginx { script "killall -0 nginx" interval 1 weight -2}vrrp_instance VI_1 { state BACKUP interface eth0 virtual_router_id 1 priority 98 advert_int 1 authentication { auth_type PASS auth_pass 1111} track_script { chk_nginx} virtual_ipaddress {192.168.33.181/24} notify_master "/etc/keepalived/notify.sh master" notify_backup "/etc/keepalived/notify.sh backup" notify_fault "/etc/keepalived/notify.sh fault"}vrrp_instance VI_2 { state MASTER interface eth0 virtual_router_id 2 priority 100 advert_int 1 authentication { auth_type PASS auth_pass 1111} track_script { chk_nginx} virtual_ipaddress {192.168.33.182/24} notify_master "/etc/keepalived/notify.sh master" notify_backup "/etc/keepalived/notify.sh backup" notify_fault "/etc/keepalived/notify.sh fault"}
v1,v2共同脚本：
monitor.sh:
#!/bin/bashwhile true;do A=`ps -ef|grep nginx |wc -l` B=`ps -ef|grep keepalived |wc -l`if[ $A -eq 1];then echo 'restart nginx!!!!'/usr/local/nginx/sbin/nginxif[ $A -eq 1];thenif[ $B -gt 1];then killall keepalived service keepalived start &fififi if[ $B -eq 1];then service keepalived start &fi sleep 5done
notify.sh
#!/bin/bash# keepalived notify scriptcontact='root@localhost'notify(){ mailsubject="`hostname` to be $1: vip floating" mailbody="`date '+%F %H:%M:%S'`: vrrp transition, `hostname` changed to be $1" echo $mailbody | mail -s "$mailsubject" $contact} case"$1"in master) notify master echo "变成主了，触发master！">>/root/k_tips.log/usr/local/nginx/sbin/nginx exit 0;; backup) notify backup echo "变成从了，触发backup事件！">>/root/k_tips.log/usr/local/nginx/sbin/nginx exit 0;; fault) notify fault echo "脑列了，触发fault事件！">>/root/k_tips.log/usr/local/nginx/sbin/nginx -s stop exit 0;;*) echo 'Usage: notify.sh {master|backup|fault}' exit 1;;esac
2.使用keepalived + haproxy + tomcat 实现
2.1 安装haproxy
yum -y install haproxy
2.2 修改配置
# to have these messages end up in /var/log/haproxy.log you will# need to:# # 1) configure syslog to accept network log events.This is done# by adding the '-r' option to the SYSLOGD_OPTIONS in# /etc/sysconfig/syslog# # 2) configure local2 events to go to the /var/log/haproxy.log# file. A line like the following can be added to# /etc/sysconfig/syslog# # local2.* /var/log/haproxy.log# # 上面说我们要把日志写到/var/log/haproxy.log需要修改以下两个地方：1./etc/sysconfig/syslog , 这个我现在用的centos6.6已经没有了，取代它的是同目录下的rsyslogvi /etc/sysconfig/rsyslog这里对它做了以下更改：修改前：SYSLOGD_OPTIONS="-c 5"修改后：SYSLOGD_OPTIONS="-c 2 -r"
2. /etc/rsyslog.conf 修改日志存放位置vi /etc/rsyslog.conf
我在local7.*下面加了一行，修改后是这样的：# Save boot messages also to boot.loglocal7.* /var/log/boot.loglocal2.* /var/log/haproxy.log
好了，基本的配置可以了。
下面开始修改/etc/haproxy/haproxy.cfg
3.使用keepalived + lvs + tomcat 实现
未完待续。。。
页:
[1]