Tomcat 负载均衡及cluster session
一、nginx负载均衡tomcat二、apache基于http负载均衡tomcat
三、apache基于ajp负载均衡tomcat
四、haproxy负载均衡tomcat
五、tomcat cluster session
一、nginx负载均衡tomcat
环境：
IP:172.16.110.47 OS:CentOS7.3 Nginx:1.10.2
IP:172.16.110.25 OS:CentOS7.3 Tomcat:8.0.39 jdk:java-1.8.0-openjdk
IP:172.16.110.49 OS:CentOS7.3 Tomcat:8.0.39 jdk:java-1.8.0-openjdk
架构图：
http://s3.运维网.com/wyfs02/M00/8D/15/wKiom1iFimTD1TK8AABbAU1ErS4135.png-wh_500x0-wm_3-wmp_4-s_114598464.png
1.安装tomcat
在172.16.110.49上部署
1.1.安装部署tomcat
1.1.1.部署jdk
yum install java 1.1.2.查看jdk及java版本
# rpm -qa | grep jdk
java-1.8.0-openjdk-1.8.0.121-0.b13.el7_3.x86_64
java-1.8.0-openjdk-headless-1.8.0.121-0.b13.el7_3.x86_64
# java -version
openjdk version "1.8.0_121"
OpenJDK Runtime Environment (build 1.8.0_121-b13)
OpenJDK 64-Bit Server VM (build 25.121-b13, mixed mode) 1.2.安装tomcat
tar -xf apache-tomcat-8.0.39.tar.gz -C /usr/local/
cd /usr/local/
ln -s apache-tomcat-8.0.39/ tomcat 1.3.配置环境变量
vim /etc/profile
export PATH=$PATH:/usr/local/tomcat/bin
source /etc/profile 1.4.配置虚拟主机
1.4.1.部署目录
# mkdir -pv /data/kaka/{lib,classes,WEB-INF,META-INF,servlets} 1.4.2.配置访问页面
# vim index.jsp
ckl JSP~
1.4.3.配置tomcat虚拟主机为默认页面
# cd /usr/local/tomcat/conf/
vim server.xml
...
...
... 1.4.4.启动进程
# catalina.sh start
Using CATALINA_BASE: /usr/local/tomcat
Using CATALINA_HOME: /usr/local/tomcat
Using CATALINA_TMPDIR: /usr/local/tomcat/temp
Using JRE_HOME: /usr
Using CLASSPATH: /usr/local/tomcat/bin/bootstrap.jar:/usr/local/tomcat/bin/tomcat-juli.jar
Tomcat started. 1.5.在172.16.110.47上测试访问
# curl http://172.16.110.49:8080
ckl JSP~
this page on 49 !
2.在172.16.110.25上部署tomcat
2.1.1.部署jdk
yum install java 2.1.2.查看jdk及java版本
# rpm -qa | grep jdk
java-1.8.0-openjdk-1.8.0.121-0.b13.el7_3.x86_64
java-1.8.0-openjdk-headless-1.8.0.121-0.b13.el7_3.x86_64
# java -version
openjdk version "1.8.0_121"
OpenJDK Runtime Environment (build 1.8.0_121-b13)
OpenJDK 64-Bit Server VM (build 25.121-b13, mixed mode) 2.2.安装tomcat
tar -xf apache-tomcat-8.0.39.tar.gz -C /usr/local/
cd /usr/local/
ln -s apache-tomcat-8.0.39/ tomcat 2.3.配置环境变量
vim /etc/profile
export PATH=$PATH:/usr/local/tomcat/bin
source /etc/profile 2.4.配置虚拟主机
2.4.1.部署目录
# mkdir -pv /data/kaka/{lib,classes,WEB-INF,META-INF,servlets} 2.4.2.配置访问页面
# vim index.jsp
ckl JSP~
2.4.3.配置tomcat虚拟主机为默认页面
# cd /usr/local/tomcat/conf/
vim server.xml
...
...
... 2.5.在172.16.110.47上测试访问
# curl http://172.16.110.25:8080
ckl JSP~
this page on 25 !
3.安装配置nginx(172.16.110.47)
3.1.安装nginx
编译安装参考其他文档，这里使用yum安装
yum install nginx 3.2.配置nginx做负载均衡
3.2.1.配置主文件
# vim nginx.conf
usernginx;
worker_processes1;
error_log/var/log/nginx/error.log warn;
pid /var/run/nginx.pid;
events {
worker_connections1024;
}
http {
include /etc/nginx/mime.types;
default_typeapplication/octet-stream;
log_formatmain'$remote_addr - $remote_user [$time_local] "$request" '
'$status $body_bytes_sent "$http_referer" '
'"$http_user_agent" "$http_x_forwarded_for"';
access_log/var/log/nginx/access.logmain;
sendfile on;
#tcp_nopush on;
keepalive_timeout65;
#gzipon;
upstream tomcat_serv {
server 172.16.110.25:8080;
server 172.16.110.49:8080;
}
include /etc/nginx/conf.d/*.conf;
} 3.2.2.配置子文件
vim conf.d/default.conf
server {
listen 80;
server_namelocalhost;
location / {
root /usr/share/nginx/html;
indexindex.html index.htm index.jsp;
}
location ~* \.(jpg|png|bmp|gif)$ {
root/usr/share/nginx/html;
expires 30d;
}
location ~* \.(jsp|do)$ {
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_buffering off;
proxy_pass http://tomcat_serv;
}
error_page 500 502 503 504/50x.html;
location = /50x.html {
root /usr/share/nginx/html;
}
} 3.3.重启服务
# systemctl restart nginx.service 3.4.测试访问
http://172.16.110.47/index.jsp
http://s1.运维网.com/wyfs02/M00/8D/12/wKioL1iFhtaDSGPlAAAQCRgullA572.png-wh_500x0-wm_3-wmp_4-s_1690178307.png
http://s1.运维网.com/wyfs02/M01/8D/15/wKiom1iFhtax-74nAAAQO5vNEFE652.png-wh_500x0-wm_3-wmp_4-s_1444733132.png
4.关于nginx session绑定：
4.1.修改配置文件
vim nginx.conf
...
upstream tomcat_serv {
ip_hash;
server 172.16.110.25:8080;
server 172.16.110.49:8080;
}
... 4.2.测试访问
http://172.16.110.47/index.jsphttp://s5.运维网.com/wyfs02/M01/8D/12/wKioL1iFhujhb5eAAAD3WM9t2b4225.png-wh_500x0-wm_3-wmp_4-s_578883926.png
http://s4.运维网.com/wyfs02/M02/8D/12/wKioL1iFhviRRtmLAAD41HFTukY112.png-wh_500x0-wm_3-wmp_4-s_2094814669.png
二、apache基于http负载均衡tomcat
环境：
IP:172.16.110.48 OS:CentOS7.3 apache:2.4.6
IP:172.16.110.25 OS:CentOS7.3 Tomcat:8.0.39 jdk:java-1.8.0-openjdk
IP:172.16.110.49 OS:CentOS7.3 Tomcat:8.0.39 jdk:java-1.8.0-openjdk
架构图：
http://s3.运维网.com/wyfs02/M00/8D/13/wKioL1iFiouDu43dAABardMCQv8295.png-wh_500x0-wm_3-wmp_4-s_1583784816.png
1.tomcat在25和49配置
配置不变，使用原有的配置
2.配置apache
2.1.安装apache
yum install httpd 查看版本：
# rpm -qa | grep httpd
httpd-tools-2.4.6-45.el7.centos.x86_64
httpd-2.4.6-45.el7.centos.x86_64 查看apache支持的模块：
# httpd -M | grep proxy
AH00558: httpd: Could not reliably determine the server's fully qualified domain name, using localhost.localdomain. Set the 'ServerName' directive globally to suppress this message
proxy_module (shared)
proxy_ajp_module (shared)
proxy_balancer_module (shared)
proxy_connect_module (shared)
proxy_express_module (shared)
proxy_fcgi_module (shared)
proxy_fdpass_module (shared)
proxy_ftp_module (shared)
proxy_http_module (shared)
proxy_scgi_module (shared)
proxy_wstunnel_module (shared) 2.2.配置文件修改
2.2.1.官网文档地址http://apache.chinahtml.com/mod/mod_proxy.html
官网示例：
http://s1.运维网.com/wyfs02/M02/8D/15/wKiom1iFhxjyNVTAAAAtdQGrxrM610.png
2.2.增加配置
cd /etc/httpd/conf.d/
vim http_tomcat.conf
ProxyRequests Off
BalancerMember http://172.16.110.25:8080 loadfactor=10
BalancerMember http://172.16.110.49:8080 loadfactor=10 route=ckltom2
ServerAdmin admin@ckl.com
ServerName www.ckl.com
ProxyPass / balancer://tom_cluster1
ProxyPa***everse / balancer://tom_cluster1
2.3.启动进程
systemctl start httpd.service 2.4.测试访问
http://172.16.110.48/
http://s4.运维网.com/wyfs02/M01/8D/12/wKioL1iFhzTgDC7RAAAMTaLorKg935.png-wh_500x0-wm_3-wmp_4-s_1925326743.png
http://s4.运维网.com/wyfs02/M02/8D/15/wKiom1iFhzWxjEUmAAAL9TN0Dy0459.png-wh_500x0-wm_3-wmp_4-s_1822265805.png
3.apache基于http代理的session绑定
参考：http://serverfault.com/questions/303675/apache-load-balancing-doesnt-set-cookie-with-route-information
3.1.修改apache配置文件
cd /etc/httpd/conf.d/
vim http_tomcat.conf
ProxyRequests Off
Header add Set-Cookie "ROUTEID=.%{BALANCER_WORKER_ROUTE}e; path=/" env=BALANCER_ROUTE_CHANGED
BalancerMember http://172.16.110.25:8080 loadfactor=10 route=ckltom1
BalancerMember http://172.16.110.49:8080 loadfactor=10 route=ckltom2
ProxySet stickysession=ROUTEID
ServerAdmin admin@ckl.com
ServerName www.ckl.com
ProxyPass / balancer://tom_cluster1
ProxyPa***everse / balancer://tom_cluster1
3.2.重启进程
systemctl restart httpd.service 3.3.测试访问
http://172.16.110.48/
http://s4.运维网.com/wyfs02/M00/8D/15/wKiom1iFh2uDWqc3AAD7cftxunQ608.png-wh_500x0-wm_3-wmp_4-s_1068024521.png
http://s2.运维网.com/wyfs02/M00/8D/12/wKioL1iFh2_g61IxAAECRX3gHF4141.png-wh_500x0-wm_3-wmp_4-s_2368589935.png
三、apache基于ajp负载均衡tomcat
IP:172.16.110.48 OS:CentOS7.3 apache:2.4.6
IP:172.16.110.25 OS:CentOS7.3 Tomcat:8.0.39 jdk:java-1.8.0-openjdk
IP:172.16.110.49 OS:CentOS7.3 Tomcat:8.0.39 jdk:java-1.8.0-openjdk
架构图：
http://s1.运维网.com/wyfs02/M02/8D/15/wKiom1iFip6Q-TphAABardMCQv8071.png-wh_500x0-wm_3-wmp_4-s_3231760247.png
1.tomcat在25和49配置
配置不变，使用原有的配置
2.配置apache基于ajp的方式
2.1.配置文件修改
2.2.1.官网文档地址
http://apache.chinahtml.com/mod/mod_proxy.html
cd /etc/httpd/conf.d/
vim ajp_tomcat.conf
ProxyRequests Off
BalancerMember ajp://172.16.110.25:8009 loadfactor=10
BalancerMember ajp://172.16.110.49:8009 loadfactor=10
ServerAdmin admin@ckl.com
ServerName www.ckl.com
ProxyPass / balancer://tom_cluster1
ProxyPa***everse / balancer://tom_cluster1
2.3.启动进程
systemctl start httpd.service 2.4.测试访问
http://172.16.110.48/
http://s2.运维网.com/wyfs02/M01/8D/15/wKiom1iFh6DwZ6nxAAAMTaLorKg377.png-wh_500x0-wm_3-wmp_4-s_1306490787.png
http://s2.运维网.com/wyfs02/M00/8D/12/wKioL1iFh6CA66gmAAAL9TN0Dy0230.png-wh_500x0-wm_3-wmp_4-s_89088503.png
3.apache基于ajp代理的session绑定
参考：http://serverfault.com/questions/303675/apache-load-balancing-doesnt-set-cookie-with-route-information
3.1.修改apache配置文件
cd /etc/httpd/conf.d/
vim http_tomcat.conf
ProxyRequests Off
Header add Set-Cookie "ROUTEID=.%{BALANCER_WORKER_ROUTE}e; path=/" env=BALANCER_ROUTE_CHANGED
BalancerMember http://172.16.110.25:8080 loadfactor=10 route=kaka1
BalancerMember http://172.16.110.49:8080 loadfactor=10 route=kaka2
ProxySet stickysession=ROUTEID
ServerAdmin admin@ckl.com
ServerName www.ckl.com
ProxyPass / balancer://tom_cluster1
ProxyPa***everse / balancer://tom_cluster1
3.2.重启进程
systemctl restart httpd.service 3.3.测试访问
http://172.16.110.48/
暂时未成功
四、haproxy负载均衡tomcat
环境：
IP:172.16.110.50 OS:CentOS7.3 haproxy:1.5.18
IP:172.16.110.25 OS:CentOS7.3 Tomcat:8.0.39 jdk:java-1.8.0-openjdk
IP:172.16.110.49 OS:CentOS7.3 Tomcat:8.0.39 jdk:java-1.8.0-openjdk
架构图：
http://s1.运维网.com/wyfs02/M02/8D/13/wKioL1iFiq2TgQKeAABaR3I735c703.png-wh_500x0-wm_3-wmp_4-s_3494483026.png
1.添加用户
# groupadd -g 200 haproxy
# adduser -u 200 -g 200 -s /sbin/nologin haproxy 2.安装haproxy
yum install haproxy 3.配置haproxy负责均衡
cd /etc/haproxy
vim haproxy.cfg
global
log 127.0.0.1 local2
chroot /var/lib/haproxy
pidfile /var/run/haproxy.pid
maxconn 4000
user haproxy
group haproxy
daemon
stats socket /var/lib/haproxy/stats
defaults
mode http
log global
option httplog
option dontlognull
option http-server-close
option forwardfor except 127.0.0.0/8
option redispatch
retries 3
timeout http-request 10s
timeout queue 1m
timeout connect 10s
timeout client 1m
timeout server 1m
timeout http-keep-alive 10s
timeout check 10s
maxconn 3000
frontendmain *:80
use_backend webserver
frontendstats_ser
bind *:8800
use_backend web_stats
backend webserver
balance roundrobin
serverweb1 172.16.110.25:8080 weight 1 maxconn 500 maxqueue 300 cookie cklser1 check
serverweb2 172.16.110.49:8080 weight 1 maxconn 500 maxqueue 300 cookie cklser2 check
backend web_stats
server web1 172.16.110.25:8080 check
server web2 172.16.110.49:8080 check
stats enable
stats hide-version
stats uri /ckl?stats
stats realm Haproxy\ Statistics
stats auth ckl:1QAZnji9
stats auth zld:4rfvBHU8
stats auth admin:7uj4rf99
stats admin if TRUE
stats refresh 5s 4.启动进程
systemctl start haproxy.service 5.测试访问
http://172.16.110.50/
http://s3.运维网.com/wyfs02/M02/8D/15/wKiom1iFh_ORR1mgAAAMFs5a_rI108.png-wh_500x0-wm_3-wmp_4-s_3881587914.png
http://s3.运维网.com/wyfs02/M02/8D/12/wKioL1iFh_PBHVYmAAAMVLxK2Gs957.png-wh_500x0-wm_3-wmp_4-s_207310341.png
session绑定简单略过
五、tomcat cluster session
文档参考地址：http://tomcat.apache.org/tomcat-8.0-doc/cluster-howto.html
IP:172.16.110.48 OS:CentOS7.3 apache:2.4.6
IP:172.16.110.25 OS:CentOS7.3 Tomcat:8.0.39 jdk:java-1.8.0-openjdk
IP:172.16.110.49 OS:CentOS7.3 Tomcat:8.0.39 jdk:java-1.8.0-openjdk
架构图：
http://s5.运维网.com/wyfs02/M00/8D/4D/wKioL1iXBmyzyXaHAADjGAU6rFg735.png
1.nginx去掉session绑定
vim nginx.conf
....
upstream tomcat_serv {
#ip_hash;
server 172.16.110.25:8080;
server 172.16.110.49:8080;
}
.... 2.重启nginx
systemctl restart nginx.service 3.增加新的页面测试跳转访问
3.1.在172.16.110.25上添加：
# cd /data/kaka/
# cat index.jsp
ckl JSP~
3.2.在172.16.110.49上添加
# cd /data/kaka/
# cat index.jsp
ckl JSP~
3.3.测试访问：
http://s4.运维网.com/wyfs02/M00/8D/4C/wKioL1iW-d7BHSCAAABD5sjrwMY468.png-wh_500x0-wm_3-wmp_4-s_514569414.png
http://s3.运维网.com/wyfs02/M00/8D/4C/wKioL1iW-d-CLN8zAABDv3KEcHk069.png-wh_500x0-wm_3-wmp_4-s_2263520886.png
4..配置cluster session
4.1.172.16.110.25上配置
cd /usr/local/tomcat/conf 4.1.1.在server.conf host中增加如下：
配置文件说明：
4.1.2.添加web.xml并修改
# cp /usr/local/tomcat/conf/web.xml /data/kaka/WEB-INF/
# vim web.xml
....
4.2.172.16.110.49上配置
cd /usr/local/tomcat/conf 4.2.1.在server.conf host中增加如下：
4.2.2.添加web.xml并修改
# cp /usr/local/tomcat/conf/web.xml /data/kaka/WEB-INF/
# vim web.xml
....
4.3.重启进程
4.3.1.重启172.16.110.25
# ./shutdown.sh
Using CATALINA_BASE: /usr/local/tomcat
Using CATALINA_HOME: /usr/local/tomcat
Using CATALINA_TMPDIR: /usr/local/tomcat/temp
Using JRE_HOME: /usr
Using CLASSPATH: /usr/local/tomcat/bin/bootstrap.jar:/usr/local/tomcat/bin/tomcat-juli.jar
# ./startup.sh
Using CATALINA_BASE: /usr/local/tomcat
Using CATALINA_HOME: /usr/local/tomcat
Using CATALINA_TMPDIR: /usr/local/tomcat/temp
Using JRE_HOME: /usr
Using CLASSPATH: /usr/local/tomcat/bin/bootstrap.jar:/usr/local/tomcat/bin/tomcat-juli.jar
Tomcat started. 查看日志：
tail -100 catalina.out
....
05-Feb-2017 18:55:59.680 INFO org.apache.catalina.tribes.membership.McastServiceImpl.waitForMembers Sleeping for 1000 milliseconds to establish cluster membership, start level:4
05-Feb-2017 18:55:59.732 INFO org.apache.catalina.ha.tcp.SimpleTcpCluster.memberAdded Replication member added:org.apache.catalina.tribes.membership.MemberImpl
05-Feb-2017 18:56:00.681 INFO org.apache.catalina.tribes.membership.McastServiceImpl.waitForMembers Done sleeping, membership established, start level:4
05-Feb-2017 18:56:00.711 INFO org.apache.catalina.tribes.membership.McastServiceImpl.waitForMembers Sleeping for 1000 milliseconds to establish cluster membership, start level:8
05-Feb-2017 18:56:00.785 INFO -1] org.apache.catalina.tribes.io.BufferPool.getBufferPool Created a buffer pool with max size:104857600 bytes of type: org.apache.catalina.tribes.io.BufferPool15Impl
05-Feb-2017 18:56:01.712 INFO org.apache.catalina.tribes.membership.McastServiceImpl.waitForMembers Done sleeping, membership established, start level:8
05-Feb-2017 18:56:01.716 INFO org.apache.catalina.ha.deploy.FarmWarDeployer.start Cluster FarmWarDeployer started.
05-Feb-2017 18:56:01.793 INFO org.apache.catalina.ha.session.DeltaManager.startInternal Register managerto cluster element Host with name www1.ckl.com
05-Feb-2017 18:56:01.793 INFO org.apache.catalina.ha.session.DeltaManager.startInternal Starting clustering manager at 4.3.2.重启172.16.110.49
# ./shutdown.sh
Using CATALINA_BASE: /usr/local/tomcat
Using CATALINA_HOME: /usr/local/tomcat
Using CATALINA_TMPDIR: /usr/local/tomcat/temp
Using JRE_HOME: /usr
Using CLASSPATH: /usr/local/tomcat/bin/bootstrap.jar:/usr/local/tomcat/bin/tomcat-juli.jar
# ./startup.sh
Using CATALINA_BASE: /usr/local/tomcat
Using CATALINA_HOME: /usr/local/tomcat
Using CATALINA_TMPDIR: /usr/local/tomcat/temp
Using JRE_HOME: /usr
Using CLASSPATH: /usr/local/tomcat/bin/bootstrap.jar:/usr/local/tomcat/bin/tomcat-juli.jar
Tomcat started. 查看日志：
tail -100 catalina.out
.....
05-Feb-2017 18:46:06.336 INFO org.apache.catalina.ha.tcp.SimpleTcpCluster.memberAdded Replication member added:org.apache.catalina.tribes.membership.MemberImpl
05-Feb-2017 18:46:06.915 INFO org.apache.catalina.tribes.membership.McastServiceImpl.waitForMembers Done sleeping, membership established, start level:4
05-Feb-2017 18:46:06.917 INFO org.apache.catalina.tribes.membership.McastServiceImpl.waitForMembers Sleeping for 1000 milliseconds to establish cluster membership, start level:8
05-Feb-2017 18:46:06.994 INFO -1] org.apache.catalina.tribes.io.BufferPool.getBufferPool Created a buffer pool with max size:104857600 bytes of type: org.apache.catalina.tribes.io.BufferPool15Impl
05-Feb-2017 18:46:07.918 INFO org.apache.catalina.tribes.membership.McastServiceImpl.waitForMembers Done sleeping, membership established, start level:8
05-Feb-2017 18:46:07.920 INFO org.apache.catalina.ha.deploy.FarmWarDeployer.start Cluster FarmWarDeployer started.
05-Feb-2017 18:46:07.999 INFO org.apache.catalina.ha.session.DeltaManager.startInternal Register managerto cluster element Host with name www1.ckl.com
05-Feb-2017 18:46:07.999 INFO org.apache.catalina.ha.session.DeltaManager.startInternal Starting clustering manager at 5.测试访问
http://s3.运维网.com/wyfs02/M02/8D/4D/wKioL1iXBcmyBkP9AABDtRXI4MQ302.png
http://s3.运维网.com/wyfs02/M00/8D/50/wKiom1iXBcmgAplBAABB78kkCmM463.png
访问的服务器变化，但SESSION未变化，说明已经共享了
页:
[1]