设为首页 收藏本站
查看: 890|回复: 0

[经验分享] Tomcat 负载均衡及cluster session

[复制链接]

尚未签到

发表于 2018-12-1 06:28:21 | 显示全部楼层 |阅读模式
  一、nginx负载均衡tomcat
  二、apache基于http负载均衡tomcat
  三、apache基于ajp负载均衡tomcat
  四、haproxy负载均衡tomcat
  五、tomcat cluster session
  

  一、nginx负载均衡tomcat
  环境:
  IP:172.16.110.47 OS:CentOS7.3 Nginx:1.10.2
  IP:172.16.110.25 OS:CentOS7.3 Tomcat:8.0.39 jdk:java-1.8.0-openjdk
  IP:172.16.110.49 OS:CentOS7.3 Tomcat:8.0.39 jdk:java-1.8.0-openjdk
  架构图:

  1.安装tomcat
  在172.16.110.49上部署
  1.1.安装部署tomcat
  1.1.1.部署jdk
yum install java  1.1.2.查看jdk及java版本
# rpm -qa | grep jdk
java-1.8.0-openjdk-1.8.0.121-0.b13.el7_3.x86_64
java-1.8.0-openjdk-headless-1.8.0.121-0.b13.el7_3.x86_64
# java -version
openjdk version "1.8.0_121"
OpenJDK Runtime Environment (build 1.8.0_121-b13)
OpenJDK 64-Bit Server VM (build 25.121-b13, mixed mode)  1.2.安装tomcat
tar -xf apache-tomcat-8.0.39.tar.gz -C /usr/local/
cd /usr/local/
ln -s apache-tomcat-8.0.39/ tomcat  1.3.配置环境变量
vim /etc/profile
export PATH=$PATH:/usr/local/tomcat/bin
source /etc/profile  1.4.配置虚拟主机
  1.4.1.部署目录
# mkdir -pv /data/kaka/{lib,classes,WEB-INF,META-INF,servlets}  1.4.2.配置访问页面
# vim index.jsp

   
           ckl JSP~
   
   
           
   
  1.4.3.配置tomcat虚拟主机为默认页面
# cd /usr/local/tomcat/conf/
vim server.xml
...

...

        
        
      
...  1.4.4.启动进程
# catalina.sh start
Using CATALINA_BASE:   /usr/local/tomcat
Using CATALINA_HOME:   /usr/local/tomcat
Using CATALINA_TMPDIR: /usr/local/tomcat/temp
Using JRE_HOME:        /usr
Using CLASSPATH:       /usr/local/tomcat/bin/bootstrap.jar:/usr/local/tomcat/bin/tomcat-juli.jar
Tomcat started.  1.5.在172.16.110.47上测试访问
# curl http://172.16.110.49:8080

   
           ckl JSP~
   
   
           this page on 49 !
   
  2.在172.16.110.25上部署tomcat
  2.1.1.部署jdk
yum install java  2.1.2.查看jdk及java版本
# rpm -qa | grep jdk
java-1.8.0-openjdk-1.8.0.121-0.b13.el7_3.x86_64
java-1.8.0-openjdk-headless-1.8.0.121-0.b13.el7_3.x86_64
# java -version
openjdk version "1.8.0_121"
OpenJDK Runtime Environment (build 1.8.0_121-b13)
OpenJDK 64-Bit Server VM (build 25.121-b13, mixed mode)  2.2.安装tomcat
tar -xf apache-tomcat-8.0.39.tar.gz -C /usr/local/
cd /usr/local/
ln -s apache-tomcat-8.0.39/ tomcat  2.3.配置环境变量
vim /etc/profile
export PATH=$PATH:/usr/local/tomcat/bin
source /etc/profile  2.4.配置虚拟主机
  2.4.1.部署目录
# mkdir -pv /data/kaka/{lib,classes,WEB-INF,META-INF,servlets}  2.4.2.配置访问页面
# vim index.jsp

   
           ckl JSP~
   
   
           
   
  2.4.3.配置tomcat虚拟主机为默认页面
# cd /usr/local/tomcat/conf/
vim server.xml
...

...

        
        
      
...  2.5.在172.16.110.47上测试访问
# curl http://172.16.110.25:8080

   
           ckl JSP~
   
   
           this page on 25 !
   
  3.安装配置nginx(172.16.110.47)
  3.1.安装nginx
  编译安装参考其他文档,这里使用yum安装
yum install nginx  3.2.配置nginx做负载均衡
  3.2.1.配置主文件
# vim nginx.conf                 
user  nginx;
worker_processes  1;
error_log  /var/log/nginx/error.log warn;
pid        /var/run/nginx.pid;
events {
    worker_connections  1024;
}
http {
    include       /etc/nginx/mime.types;
    default_type  application/octet-stream;
    log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
                      '$status $body_bytes_sent "$http_referer" '
                      '"$http_user_agent" "$http_x_forwarded_for"';
    access_log  /var/log/nginx/access.log  main;
    sendfile        on;
    #tcp_nopush     on;
    keepalive_timeout  65;
    #gzip  on;
    upstream tomcat_serv {
        server 172.16.110.25:8080;
        server 172.16.110.49:8080;
    }
    include /etc/nginx/conf.d/*.conf;
}  3.2.2.配置子文件
vim conf.d/default.conf
server {
    listen       80;
    server_name  localhost;

    location / {
        root   /usr/share/nginx/html;
        index  index.html index.htm index.jsp;
    }
    location ~* \.(jpg|png|bmp|gif)$ {
        root  /usr/share/nginx/html;
        expires 30d;
    }
    location ~* \.(jsp|do)$ {
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_buffering off;
        proxy_pass http://tomcat_serv;
    }
    error_page   500 502 503 504  /50x.html;
    location = /50x.html {
        root   /usr/share/nginx/html;
    }
}  3.3.重启服务
# systemctl restart nginx.service  3.4.测试访问
  http://172.16.110.47/index.jsp


  

  4.关于nginx session绑定:
  4.1.修改配置文件
vim nginx.conf
...
  upstream tomcat_serv {
        ip_hash;
        server 172.16.110.25:8080;
        server 172.16.110.49:8080;
    }
...  4.2.测试访问
http://172.16.110.47/index.jsp

  

  二、apache基于http负载均衡tomcat
  环境:
  IP:172.16.110.48 OS:CentOS7.3 apache:2.4.6
  IP:172.16.110.25 OS:CentOS7.3 Tomcat:8.0.39 jdk:java-1.8.0-openjdk
  IP:172.16.110.49 OS:CentOS7.3 Tomcat:8.0.39 jdk:java-1.8.0-openjdk
  架构图:

  1.tomcat在25和49配置
  配置不变,使用原有的配置
  2.配置apache
  2.1.安装apache
yum install httpd  查看版本:
# rpm -qa | grep httpd
httpd-tools-2.4.6-45.el7.centos.x86_64
httpd-2.4.6-45.el7.centos.x86_64  查看apache支持的模块:
# httpd -M | grep proxy
AH00558: httpd: Could not reliably determine the server's fully qualified domain name, using localhost.localdomain. Set the 'ServerName' directive globally to suppress this message
proxy_module (shared)
proxy_ajp_module (shared)
proxy_balancer_module (shared)
proxy_connect_module (shared)
proxy_express_module (shared)
proxy_fcgi_module (shared)
proxy_fdpass_module (shared)
proxy_ftp_module (shared)
proxy_http_module (shared)
proxy_scgi_module (shared)
proxy_wstunnel_module (shared)  2.2.配置文件修改
  2.2.1.官网文档地址http://apache.chinahtml.com/mod/mod_proxy.html
  官网示例:

  2.2.增加配置
cd /etc/httpd/conf.d/
vim http_tomcat.conf
ProxyRequests Off

  BalancerMember http://172.16.110.25:8080 loadfactor=10
  BalancerMember http://172.16.110.49:8080 loadfactor=10 route=ckltom2


  ServerAdmin admin@ckl.com
  ServerName www.ckl.com
  ProxyPass / balancer://tom_cluster1
  ProxyPa***everse / balancer://tom_cluster1
  2.3.启动进程
systemctl start httpd.service  2.4.测试访问
  http://172.16.110.48/


  3.apache基于http代理的session绑定
  参考:http://serverfault.com/questions/303675/apache-load-balancing-doesnt-set-cookie-with-route-information
  3.1.修改apache配置文件
cd /etc/httpd/conf.d/
vim http_tomcat.conf
ProxyRequests Off
Header add Set-Cookie "ROUTEID=.%{BALANCER_WORKER_ROUTE}e; path=/" env=BALANCER_ROUTE_CHANGED

  BalancerMember http://172.16.110.25:8080 loadfactor=10 route=ckltom1
  BalancerMember http://172.16.110.49:8080 loadfactor=10 route=ckltom2
  ProxySet stickysession=ROUTEID


  ServerAdmin admin@ckl.com
  ServerName www.ckl.com
  ProxyPass / balancer://tom_cluster1
  ProxyPa***everse / balancer://tom_cluster1
  3.2.重启进程
systemctl restart httpd.service  3.3.测试访问
  http://172.16.110.48/


  

  三、apache基于ajp负载均衡tomcat
  IP:172.16.110.48 OS:CentOS7.3 apache:2.4.6
  IP:172.16.110.25 OS:CentOS7.3 Tomcat:8.0.39 jdk:java-1.8.0-openjdk
  IP:172.16.110.49 OS:CentOS7.3 Tomcat:8.0.39 jdk:java-1.8.0-openjdk
  

  架构图:

  1.tomcat在25和49配置
  配置不变,使用原有的配置
  2.配置apache基于ajp的方式
  2.1.配置文件修改
  2.2.1.官网文档地址
  http://apache.chinahtml.com/mod/mod_proxy.html
cd /etc/httpd/conf.d/
vim ajp_tomcat.conf
ProxyRequests Off

  BalancerMember ajp://172.16.110.25:8009 loadfactor=10
  BalancerMember ajp://172.16.110.49:8009 loadfactor=10


  ServerAdmin admin@ckl.com
  ServerName www.ckl.com
  ProxyPass / balancer://tom_cluster1
  ProxyPa***everse / balancer://tom_cluster1
  2.3.启动进程
systemctl start httpd.service  2.4.测试访问
  http://172.16.110.48/


  3.apache基于ajp代理的session绑定
  参考:http://serverfault.com/questions/303675/apache-load-balancing-doesnt-set-cookie-with-route-information
  3.1.修改apache配置文件
cd /etc/httpd/conf.d/
vim http_tomcat.conf
ProxyRequests Off
Header add Set-Cookie "ROUTEID=.%{BALANCER_WORKER_ROUTE}e; path=/" env=BALANCER_ROUTE_CHANGED

  BalancerMember http://172.16.110.25:8080 loadfactor=10 route=kaka1
  BalancerMember http://172.16.110.49:8080 loadfactor=10 route=kaka2
  ProxySet stickysession=ROUTEID


  ServerAdmin admin@ckl.com
  ServerName www.ckl.com
  ProxyPass / balancer://tom_cluster1
  ProxyPa***everse / balancer://tom_cluster1
  3.2.重启进程
systemctl restart httpd.service  3.3.测试访问
  http://172.16.110.48/
  暂时未成功
  

  四、haproxy负载均衡tomcat
  环境:
  IP:172.16.110.50 OS:CentOS7.3 haproxy:1.5.18
  IP:172.16.110.25 OS:CentOS7.3 Tomcat:8.0.39 jdk:java-1.8.0-openjdk
  IP:172.16.110.49 OS:CentOS7.3 Tomcat:8.0.39 jdk:java-1.8.0-openjdk
  架构图:

  1.添加用户
# groupadd -g 200 haproxy
# adduser -u 200 -g 200 -s /sbin/nologin haproxy  2.安装haproxy
yum install haproxy  3.配置haproxy负责均衡
cd /etc/haproxy
vim haproxy.cfg
global
    log         127.0.0.1 local2
    chroot      /var/lib/haproxy
    pidfile     /var/run/haproxy.pid
    maxconn     4000
    user        haproxy
    group       haproxy
    daemon
    stats socket /var/lib/haproxy/stats
defaults
    mode                    http
    log                     global
    option                  httplog
    option                  dontlognull
    option http-server-close
    option forwardfor       except 127.0.0.0/8
    option                  redispatch
    retries                 3
    timeout http-request    10s
    timeout queue           1m
    timeout connect         10s
    timeout client          1m
    timeout server          1m
    timeout http-keep-alive 10s
    timeout check           10s
    maxconn                 3000
frontend  main *:80
    use_backend webserver
frontend  stats_ser
    bind *:8800
    use_backend web_stats
backend webserver
    balance    roundrobin
    server  web1 172.16.110.25:8080 weight 1 maxconn 500 maxqueue 300 cookie cklser1 check
    server  web2 172.16.110.49:8080 weight 1 maxconn 500 maxqueue 300 cookie cklser2 check
backend web_stats
    server web1 172.16.110.25:8080 check
    server web2 172.16.110.49:8080 check
    stats enable
    stats hide-version
    stats uri     /ckl?stats
    stats realm   Haproxy\ Statistics
    stats auth    ckl:1QAZnji9
    stats auth    zld:4rfvBHU8
    stats auth    admin:7uj4rf99
    stats admin if TRUE
    stats refresh 5s  4.启动进程
systemctl start haproxy.service  5.测试访问
  http://172.16.110.50/


  session绑定简单略过
  

  五、tomcat cluster session
  文档参考地址:http://tomcat.apache.org/tomcat-8.0-doc/cluster-howto.html
IP:172.16.110.48 OS:CentOS7.3 apache:2.4.6
IP:172.16.110.25 OS:CentOS7.3 Tomcat:8.0.39 jdk:java-1.8.0-openjdk
IP:172.16.110.49 OS:CentOS7.3 Tomcat:8.0.39 jdk:java-1.8.0-openjdk


架构图:
http://s5.运维网.com/wyfs02/M00/8D/4D/wKioL1iXBmyzyXaHAADjGAU6rFg735.png
  1.nginx去掉session绑定
vim nginx.conf
....
  upstream tomcat_serv {
        #ip_hash;
        server 172.16.110.25:8080;
        server 172.16.110.49:8080;
    }
....  2.重启nginx
systemctl restart nginx.service  3.增加新的页面测试跳转访问
  3.1.在172.16.110.25上添加:
# cd /data/kaka/
# cat index.jsp

   
           ckl JSP~
   
   
           
         
         
         
         
         
         
   
  3.2.在172.16.110.49上添加

# cd /data/kaka/
# cat index.jsp

   
           ckl JSP~
   
   
           
           
           
           
           
           
           
   
  3.3.测试访问:


  4..配置cluster session
  4.1.172.16.110.25上配置
cd /usr/local/tomcat/conf  4.1.1.在server.conf host中增加如下:

        
        
        
        
      
      
         
         
         
            
            
            
              
            
            
            
         
         
         
         
         
        
        
        
        配置文件说明:

  4.1.2.添加web.xml并修改
# cp /usr/local/tomcat/conf/web.xml /data/kaka/WEB-INF/
# vim web.xml

....

  4.2.172.16.110.49上配置
cd /usr/local/tomcat/conf  4.2.1.在server.conf host中增加如下:

         
         
         
            
            
            
              
            
            
            
         
         
         
         
         
        
        
        
      4.2.2.添加web.xml并修改
# cp /usr/local/tomcat/conf/web.xml /data/kaka/WEB-INF/
# vim web.xml

....

  4.3.重启进程
  4.3.1.重启172.16.110.25
[root@localhost bin]# ./shutdown.sh
Using CATALINA_BASE:   /usr/local/tomcat
Using CATALINA_HOME:   /usr/local/tomcat
Using CATALINA_TMPDIR: /usr/local/tomcat/temp
Using JRE_HOME:        /usr
Using CLASSPATH:       /usr/local/tomcat/bin/bootstrap.jar:/usr/local/tomcat/bin/tomcat-juli.jar
[root@localhost bin]# ./startup.sh
Using CATALINA_BASE:   /usr/local/tomcat
Using CATALINA_HOME:   /usr/local/tomcat
Using CATALINA_TMPDIR: /usr/local/tomcat/temp
Using JRE_HOME:        /usr
Using CLASSPATH:       /usr/local/tomcat/bin/bootstrap.jar:/usr/local/tomcat/bin/tomcat-juli.jar
Tomcat started.  查看日志:
tail -100 catalina.out
....
05-Feb-2017 18:55:59.680 INFO [Catalina-startStop-1] org.apache.catalina.tribes.membership.McastServiceImpl.waitForMembers Sleeping for 1000 milliseconds to establish cluster membership, start level:4
05-Feb-2017 18:55:59.732 INFO [Membership-MemberAdded.] org.apache.catalina.ha.tcp.SimpleTcpCluster.memberAdded Replication member added:org.apache.catalina.tribes.membership.MemberImpl[tcp://{172, 16, 110, 49}:4000,{172, 16, 110, 49},4000, alive=593839, securePort=-1, UDP Port=-1, id={-10 -69 -6 -94 -11 38 78 -2 -89 6 86 12 7 33 1 58 }, payload={}, command={}, domain={}, ]
05-Feb-2017 18:56:00.681 INFO [Catalina-startStop-1] org.apache.catalina.tribes.membership.McastServiceImpl.waitForMembers Done sleeping, membership established, start level:4
05-Feb-2017 18:56:00.711 INFO [Catalina-startStop-1] org.apache.catalina.tribes.membership.McastServiceImpl.waitForMembers Sleeping for 1000 milliseconds to establish cluster membership, start level:8
05-Feb-2017 18:56:00.785 INFO [Tribes-Task-Receiver[www1.ckl.com-Channel]-1] org.apache.catalina.tribes.io.BufferPool.getBufferPool Created a buffer pool with max size:104857600 bytes of type: org.apache.catalina.tribes.io.BufferPool15Impl
05-Feb-2017 18:56:01.712 INFO [Catalina-startStop-1] org.apache.catalina.tribes.membership.McastServiceImpl.waitForMembers Done sleeping, membership established, start level:8
05-Feb-2017 18:56:01.716 INFO [Catalina-startStop-1] org.apache.catalina.ha.deploy.FarmWarDeployer.start Cluster FarmWarDeployer started.
05-Feb-2017 18:56:01.793 INFO [www1.ckl.com-startStop-1] org.apache.catalina.ha.session.DeltaManager.startInternal Register manager  to cluster element Host with name www1.ckl.com
05-Feb-2017 18:56:01.793 INFO [www1.ckl.com-startStop-1] org.apache.catalina.ha.session.DeltaManager.startInternal Starting clustering manager at  4.3.2.重启172.16.110.49
[root@localhost bin]# ./shutdown.sh
Using CATALINA_BASE:   /usr/local/tomcat
Using CATALINA_HOME:   /usr/local/tomcat
Using CATALINA_TMPDIR: /usr/local/tomcat/temp
Using JRE_HOME:        /usr
Using CLASSPATH:       /usr/local/tomcat/bin/bootstrap.jar:/usr/local/tomcat/bin/tomcat-juli.jar
[root@localhost bin]# ./startup.sh
Using CATALINA_BASE:   /usr/local/tomcat
Using CATALINA_HOME:   /usr/local/tomcat
Using CATALINA_TMPDIR: /usr/local/tomcat/temp
Using JRE_HOME:        /usr
Using CLASSPATH:       /usr/local/tomcat/bin/bootstrap.jar:/usr/local/tomcat/bin/tomcat-juli.jar
Tomcat started.  查看日志:
tail -100 catalina.out
.....
05-Feb-2017 18:46:06.336 INFO [Membership-MemberAdded.] org.apache.catalina.ha.tcp.SimpleTcpCluster.memberAdded Replication member added:org.apache.catalina.tribes.membership.MemberImpl[tcp://{172, 16, 110, 25}:4000,{172, 16, 110, 25},4000, alive=9045, securePort=-1, UDP Port=-1, id={39 -24 -21 62 33 -5 77 111 -85 -15 9 -122 95 37 -8 -74 }, payload={}, command={}, domain={}, ]
05-Feb-2017 18:46:06.915 INFO [Catalina-startStop-1] org.apache.catalina.tribes.membership.McastServiceImpl.waitForMembers Done sleeping, membership established, start level:4
05-Feb-2017 18:46:06.917 INFO [Catalina-startStop-1] org.apache.catalina.tribes.membership.McastServiceImpl.waitForMembers Sleeping for 1000 milliseconds to establish cluster membership, start level:8
05-Feb-2017 18:46:06.994 INFO [Tribes-Task-Receiver[www1.ckl.com-Channel]-1] org.apache.catalina.tribes.io.BufferPool.getBufferPool Created a buffer pool with max size:104857600 bytes of type: org.apache.catalina.tribes.io.BufferPool15Impl
05-Feb-2017 18:46:07.918 INFO [Catalina-startStop-1] org.apache.catalina.tribes.membership.McastServiceImpl.waitForMembers Done sleeping, membership established, start level:8
05-Feb-2017 18:46:07.920 INFO [Catalina-startStop-1] org.apache.catalina.ha.deploy.FarmWarDeployer.start Cluster FarmWarDeployer started.
05-Feb-2017 18:46:07.999 INFO [www1.ckl.com-startStop-1] org.apache.catalina.ha.session.DeltaManager.startInternal Register manager  to cluster element Host with name www1.ckl.com
05-Feb-2017 18:46:07.999 INFO [www1.ckl.com-startStop-1] org.apache.catalina.ha.session.DeltaManager.startInternal Starting clustering manager at  5.测试访问


  访问的服务器变化,但SESSION未变化,说明已经共享了
  





运维网声明 1、欢迎大家加入本站运维交流群:群②:261659950 群⑤:202807635 群⑦870801961 群⑧679858003
2、本站所有主题由该帖子作者发表,该帖子作者与运维网享有帖子相关版权
3、所有作品的著作权均归原作者享有,请您和我们一样尊重他人的著作权等合法权益。如果您对作品感到满意,请购买正版
4、禁止制作、复制、发布和传播具有反动、淫秽、色情、暴力、凶杀等内容的信息,一经发现立即删除。若您因此触犯法律,一切后果自负,我们对此不承担任何责任
5、所有资源均系网友上传或者通过网络收集,我们仅提供一个展示、介绍、观摩学习的平台,我们不对其内容的准确性、可靠性、正当性、安全性、合法性等负责,亦不承担任何法律责任
6、所有作品仅供您个人学习、研究或欣赏,不得用于商业或者其他用途,否则,一切后果均由您自己承担,我们对此不承担任何法律责任
7、如涉及侵犯版权等问题,请您及时通知我们,我们将立即采取措施予以解决
8、联系人Email:admin@iyunv.com 网址:www.yunweiku.com

所有资源均系网友上传或者通过网络收集,我们仅提供一个展示、介绍、观摩学习的平台,我们不对其承担任何法律责任,如涉及侵犯版权等问题,请您及时通知我们,我们将立即处理,联系人Email:kefu@iyunv.com,QQ:1061981298 本贴地址:https://www.yunweiku.com/thread-641698-1-1.html 上篇帖子: Tomcat调优指南 下篇帖子: Tomcat安全加固配置手册
您需要登录后才可以回帖 登录 | 立即注册

本版积分规则

扫码加入运维网微信交流群X

扫码加入运维网微信交流群

扫描二维码加入运维网微信交流群,最新一手资源尽在官方微信交流群!快快加入我们吧...

扫描微信二维码查看详情

客服E-mail:kefu@iyunv.com 客服QQ:1061981298


QQ群⑦:运维网交流群⑦ QQ群⑧:运维网交流群⑧ k8s群:运维网kubernetes交流群


提醒:禁止发布任何违反国家法律、法规的言论与图片等内容;本站内容均来自个人观点与网络等信息,非本站认同之观点.


本站大部分资源是网友从网上搜集分享而来,其版权均归原作者及其网站所有,我们尊重他人的合法权益,如有内容侵犯您的合法权益,请及时与我们联系进行核实删除!



合作伙伴: 青云cloud

快速回复 返回顶部 返回列表