ahua671 发表于 2015-8-6 12:39:19

Nginx+Memcached+Tomcat集群配置实践(Sticky Session)

准备工作


创建一个简单的web应用,名为session。其中有两个页面,分别如下所示:

[*]页面login.jsp






view plaincopy

[*]
[*]
[*]
[*]
[*]登录页面
[*]
[*]
[*]
[*]
[*]
[*]
[*]
[*]      
[*]            
[*]               用户登录
[*]            
[*]            
[*]               
[*]                  用户名
[*]                     
[*]               
[*]               
[*]                  密 码
[*]                     
[*]               
[*]               
[*]                     
[*]                           
[*]                     
[*]               
[*]            
[*]      
[*]
[*]




[*]登录成功页面success.jsp






view plaincopy

[*]
[*]
[*]
[*]
[*]登录成功页面
[*]
[*]
[*]
[*]
[*]
[*]
[*]
[*]
[*]      
[*]            
[*]               
[*]                  当前登录用户:
[*]               
[*]            
[*]      
[*]
[*]


还有一个Servlet,负责请求的转发,并设置session数据,如下所示:





view plaincopy

[*]package org.shirdrn.cluster.tomcat.servlet;
[*]
[*]import java.io.IOException;
[*]import java.util.HashMap;
[*]import java.util.Map;
[*]
[*]import javax.servlet.ServletException;
[*]import javax.servlet.http.HttpServlet;
[*]import javax.servlet.http.HttpServletRequest;
[*]import javax.servlet.http.HttpServletResponse;
[*]
[*]import org.apache.commons.logging.Log;
[*]import org.apache.commons.logging.LogFactory;
[*]
[*]public class LoginSevlet extends HttpServlet {
[*]   private static final long serialVersionUID = 1L;
[*]   private static final Log LOG = LogFactory.getLog(LoginSevlet.class);
[*]    private static Map accountDB = new HashMap();
[*]    static {
[*]         accountDB.put("shirdrn", "123456");
[*]         accountDB.put("admin", "999999");
[*]         accountDB.put("dev", "000888");
[*]         accountDB.put("007", "007007");
[*]    }
[*]   
[*]    public LoginSevlet() {
[*]      super();
[*]    }
[*]
[*]   protected void doGet(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException {
[*]          this.doPost(request, response);
[*]   }
[*]
[*]   protected void doPost(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException {
[*]          String userName = request.getParameter("userName");
[*]          String password = request.getParameter("password");
[*]          LOG.info("Raw input:userName=" + userName + ",password=" + password);
[*]         
[*]          if(userName!=null && password!=null
[*]                  && accountDB.containsKey(userName)
[*]                  && accountDB.get(userName).equals(password)) {
[*]               LOG.info("Login;status=SUCCESS");
[*]               request.getSession().setAttribute("userName", userName);
[*]               request.getRequestDispatcher("success.jsp").forward(request, response);               
[*]          } else {
[*]               LOG.info("Login;status=FAIL");
[*]               request.getRequestDispatcher("login.jsp").forward(request, response);
[*]          }
[*]   }
[*]
[*]}




这里模拟了一个账号数据库,对用户的登录请求进行处理。

最后打成WAR包,部署到Tomcat集群中的每个结点上。



Memcached安装、配置、启动



在Ubuntu系统下,如果没有安装Memcached,可以执行如下命令进行安装:





view plaincopy

[*]sudo apt-get install memcached


然后,启动Memcached服务进程,这里,我启动了两个服务进程,分别监听端口11211、11311,启动命令如下所示:





view plaincopy

[*]sudo /usr/bin/memcached -m 64 -d -u memcache -p 11211 -l 192.168.1.104 -c 5000 -P /tmp/n1-memcached.pid
[*]sudo /usr/bin/memcached -m 64 -d -u memcache -p 11311 -l 192.168.1.104 -c 5000 -P /tmp/n2-memcached.pid


查询Memcached服务状态:





view plaincopy

[*]shirdrn@dev:~/servers/cluster/nginx_tomcat_memcached$ ps -ef | grep memcached
[*]memcache1578   10 Jan25 ?      00:00:00 /usr/bin/memcached -m 64 -p 11211 -u memcache -l 127.0.0.1
[*]memcache3300   10 Jan25 ?      00:00:00 /usr/bin/memcached -m 64 -d -u memcache -p 11211 -l 192.168.1.104 -c 5000 -P /tmp/n1-memcached.pid
[*]memcache3308   10 00:00 ?      00:00:00 /usr/bin/memcached -m 64 -d -u memcache -p 11311 -l 192.168.1.104 -c 5000 -P /tmp/n2-memcached.pid
[*]shirdrn   331830950 00:00 pts/5    00:00:00 grep --color=auto memcached





Tomcat集群配置



Tomcat
集群配置,集群中各个结点通过共享存储在缓存Memcached中session来实现session的共享:如果有一台机器上的Tomcat服务停掉
了,对于其他对等服务器上的session数据仍然可以从Memcached缓存中读取,从而不会发生session丢失的问题。

对于满足这种配置的实现方案,可以在https://code.google.com/p/memcached-session-manager和https://spymemcached.googlecode.com上下载相关的jar文件。因为基于不同的序列化方案,可以有多种配置方法,下面是选择Javolution序列化框架,需要提供如下库文件:



[*]https://memcached-session-manager.googlecode.com/files/memcached-session-manager-1.6.3.jar
[*]https://memcached-session-manager.googlecode.com/files/memcached-session-manager-tc7-1.6.3.jar
[*]https://memcached-session-manager.googlecode.com/files/msm-javolution-serializer-1.6.2.jar
[*]http://memcached-session-manager.googlecode.com/svn/maven/javolution/javolution/5.4.3.1/javolution-5.4.3.1.jar
[*]https://spymemcached.googlecode.com/files/spymemcached-2.8.4.jar


下载上述jar文件,拷贝到$CATALINA_HOME/lib目录下,然后需要配置$CATALINA_HOME/conf/server.xml文件。

Tomcat集群中,每个结点的$CATALINA_HOME/conf/server.xml基本配置都是相同的,不同配置内容分别如下:



[*]如果在同一台机器上,要保证各个服务端口不相冲突
[*]元素中failoverNodes属性值不同


例如,我们在同一台机器上配置了两个Tomcat服务器实例,所在目录分别为tomcat-1和tomcat-2,对应的conf/server.xml配置内容如下:



[*]tomcat-1/conf/server.xml






view plaincopy

[*]
[*]
[*]
[*]      
[*]      
[*]      
[*]      
[*]      
[*]
[*]      
[*]            
[*]      
[*]
[*]      
[*]            
[*]            
[*]            
[*]
[*]               
[*]                     
[*]               
[*]
[*]               
[*]                     
[*]                           
[*]                     
[*]                     
[*]               
[*]
[*]            
[*]      
[*]


这里面,failover的Memcached结点是n1,也就是说,如果tomcat-1和n1在同一台机器上的话,session数据会优先复制到存储到Memcached结点n2,这样即使n1所在的结点宕机了,n2所在结点仍然存储了之前的session数据。



[*]tomcat-2/conf/server.xml






view plaincopy

[*]
[*]
[*]
[*]      
[*]      
[*]      
[*]      
[*]      
[*]
[*]      
[*]            
[*]      
[*]
[*]      
[*]            
[*]            
[*]            
[*]
[*]               
[*]                     
[*]               
[*]
[*]               
[*]                     
[*]                           
[*]                     
[*]                     
[*]               
[*]
[*]            
[*]      
[*]




和上面
类似,failover的Memcached结点是n2,也就是说,如果tomcat-2和n2在同一台机器上的话,Session数据会优先复制到存储
到Memcached结点n1,这样即使n2所在的结点宕机了,n1所在结点仍然存储了之前的Session数据。



上面的
配置实现了Tomcat集群中,通过Memcached实现了Sticky
Session(粘性Session),主要是通过配置failover结点来达到目的的。如果采用Non-sticky
Session方式,就不需要考虑failover的问题,详细配置扩参考文档(https://code.google.com/p/memcached-session-manager/wiki/SetupAndConfiguration),下面截取一个片段:





  
The following example shows a configuration for non-sticky sessions. In this case there's no need for failoverNodes,
as sessions are served by all tomcats round-robin and they're not bound
to a single tomcat. For non-sticky sessions the configuration (for
both/all tomcats) would look like this:


...







分别启动tomcat-1和tomcat-2,如下所示:





view plaincopy

[*]shirdrn@dev:~/servers/cluster/nginx_tomcat_memcached$ tomcat-1/bin/catalina.sh start
[*]Using CATALINA_BASE:   /home/shirdrn/servers/cluster/nginx_tomcat_memcached/tomcat-1
[*]Using CATALINA_HOME:   /home/shirdrn/servers/cluster/nginx_tomcat_memcached/tomcat-1
[*]Using CATALINA_TMPDIR: /home/shirdrn/servers/cluster/nginx_tomcat_memcached/tomcat-1/temp
[*]Using JRE_HOME:      /home/hadoop/installation/jdk1.6.0_30
[*]Using CLASSPATH:       /home/shirdrn/servers/cluster/nginx_tomcat_memcached/tomcat-1/bin/bootstrap.jar:/home/shirdrn/servers/cluster/nginx_tomcat_memcached/tomcat-1/bin/tomcat-juli.jar
[*]
[*]shirdrn@dev:~/servers/cluster/nginx_tomcat_memcached$ tomcat-2/bin/catalina.sh start
[*]Using CATALINA_BASE:   /home/shirdrn/servers/cluster/nginx_tomcat_memcached/tomcat-2
[*]Using CATALINA_HOME:   /home/shirdrn/servers/cluster/nginx_tomcat_memcached/tomcat-2
[*]Using CATALINA_TMPDIR: /home/shirdrn/servers/cluster/nginx_tomcat_memcached/tomcat-2/temp
[*]Using JRE_HOME:      /home/hadoop/installation/jdk1.6.0_30
[*]Using CLASSPATH:       /home/shirdrn/servers/cluster/nginx_tomcat_memcached/tomcat-2/bin/bootstrap.jar:/home/shirdrn/servers/cluster/nginx_tomcat_memcached/tomcat-2/bin/tomcat-juli.jar


可以查看两个tomcat的启动日志,对比一下:

[*]tomcat-1的日志






view plaincopy

[*]shirdrn@dev:~/servers/cluster/nginx_tomcat_memcached$ tail -100f tomcat-1/logs/catalina.out
[*]Jan 26, 2013 12:03:25 AM org.apache.catalina.core.AprLifecycleListener init
[*]INFO: The APR based Apache Tomcat Native library which allows optimal performance in production environments was not found on the java.library.path: /home/hadoop/installation/jdk1.6.0_30/jre/lib/i386/client:/home/hadoop/installation/jdk1.6.0_30/jre/lib/i386:/home/hadoop/installation/jdk1.6.0_30/jre/../lib/i386:/usr/java/packages/lib/i386:/lib:/usr/lib
[*]Jan 26, 2013 12:03:26 AM org.apache.coyote.AbstractProtocol init
[*]INFO: Initializing ProtocolHandler ["http-bio-8088"]
[*]Jan 26, 2013 12:03:26 AM org.apache.coyote.AbstractProtocol init
[*]INFO: Initializing ProtocolHandler ["ajp-bio-8089"]
[*]Jan 26, 2013 12:03:26 AM org.apache.catalina.startup.Catalina load
[*]INFO: Initialization processed in 1743 ms
[*]Jan 26, 2013 12:03:26 AM org.apache.catalina.core.StandardService startInternal
[*]INFO: Starting service Catalina
[*]Jan 26, 2013 12:03:26 AM org.apache.catalina.core.StandardEngine startInternal
[*]INFO: Starting Servlet Engine: Apache Tomcat/7.0.26
[*]Jan 26, 2013 12:03:27 AM de.javakaffee.web.msm.MemcachedSessionService startInternal
[*]INFO: MemcachedSessionService starts initialization... (configured nodes definition n1:192.168.1.104:11211,n2:192.168.1.104:11311, failover nodes n1)
[*]2013-01-26 00:03:27.441 INFO net.spy.memcached.MemcachedConnection:Added {QA sa=/192.168.1.104:11211, #Rops=0, #Wops=0, #iq=0, topRop=null, topWop=null, toWrite=0, interested=0} to connect queue
[*]2013-01-26 00:03:27.442 INFO net.spy.memcached.MemcachedConnection:Added {QA sa=/192.168.1.104:11311, #Rops=0, #Wops=0, #iq=0, topRop=null, topWop=null, toWrite=0, interested=0} to connect queue
[*]Jan 26, 2013 12:03:27 AM de.javakaffee.web.msm.RequestTrackingHostValve   
[*]INFO: Setting ignorePattern to .*\.(png|gif|jpg|css|js)$
[*]2013-01-26 00:03:27.459 INFO net.spy.memcached.MemcachedConnection:Connection state changed for sun.nio.ch.SelectionKeyImpl@5b0668
[*]2013-01-26 00:03:27.460 INFO net.spy.memcached.MemcachedConnection:Connection state changed for sun.nio.ch.SelectionKeyImpl@1d5a0
[*]Jan 26, 2013 12:03:27 AM de.javakaffee.web.msm.MemcachedSessionService setLockingMode
[*]INFO: Setting lockingMode to null
[*]Jan 26, 2013 12:03:27 AM de.javakaffee.web.msm.MemcachedSessionService createTranscoderFactory
[*]INFO: Creating transcoder factory de.javakaffee.web.msm.serializer.javolution.JavolutionTranscoderFactory
[*]Jan 26, 2013 12:03:27 AM de.javakaffee.web.msm.MemcachedSessionService startInternal
[*]INFO: MemcachedSessionService finished initialization, sticky true, operation timeout 1000, with node ids and failover node ids
[*]Jan 26, 2013 12:03:27 AM org.apache.catalina.startup.HostConfig deployDirectory
[*]INFO: Deploying web application directory /home/shirdrn/servers/cluster/nginx_tomcat_memcached/tomcat-1/webapps/host-manager
[*]Jan 26, 2013 12:03:27 AM org.apache.catalina.startup.HostConfig deployDirectory
[*]INFO: Deploying web application directory /home/shirdrn/servers/cluster/nginx_tomcat_memcached/tomcat-1/webapps/manager
[*]Jan 26, 2013 12:03:27 AM org.apache.catalina.startup.HostConfig deployDirectory
[*]INFO: Deploying web application directory /home/shirdrn/servers/cluster/nginx_tomcat_memcached/tomcat-1/webapps/docs
[*]Jan 26, 2013 12:03:27 AM org.apache.catalina.startup.HostConfig deployDirectory
[*]INFO: Deploying web application directory /home/shirdrn/servers/cluster/nginx_tomcat_memcached/tomcat-1/webapps/examples
[*]Jan 26, 2013 12:03:28 AM org.apache.catalina.startup.HostConfig deployDirectory
[*]INFO: Deploying web application directory /home/shirdrn/servers/cluster/nginx_tomcat_memcached/tomcat-1/webapps/ROOT
[*]Jan 26, 2013 12:03:28 AM org.apache.coyote.AbstractProtocol start
[*]INFO: Starting ProtocolHandler ["http-bio-8088"]
[*]Jan 26, 2013 12:03:28 AM org.apache.coyote.AbstractProtocol start
[*]INFO: Starting ProtocolHandler ["ajp-bio-8089"]
[*]Jan 26, 2013 12:03:28 AM org.apache.catalina.startup.Catalina start
[*]INFO: Server startup in 1487 ms






[*]tomcat-2的日志






view plaincopy

[*]shirdrn@dev:~/servers/cluster/nginx_tomcat_memcached$ tail -100f tomcat-2/logs/catalina.out
[*]Jan 26, 2013 12:04:34 AM org.apache.catalina.core.AprLifecycleListener init
[*]INFO: The APR based Apache Tomcat Native library which allows optimal performance in production environments was not found on the java.library.path: /home/hadoop/installation/jdk1.6.0_30/jre/lib/i386/client:/home/hadoop/installation/jdk1.6.0_30/jre/lib/i386:/home/hadoop/installation/jdk1.6.0_30/jre/../lib/i386:/usr/java/packages/lib/i386:/lib:/usr/lib
[*]Jan 26, 2013 12:04:34 AM org.apache.coyote.AbstractProtocol init
[*]INFO: Initializing ProtocolHandler ["http-bio-9088"]
[*]Jan 26, 2013 12:04:34 AM org.apache.coyote.AbstractProtocol init
[*]INFO: Initializing ProtocolHandler ["ajp-bio-9089"]
[*]Jan 26, 2013 12:04:34 AM org.apache.catalina.startup.Catalina load
[*]INFO: Initialization processed in 742 ms
[*]Jan 26, 2013 12:04:34 AM org.apache.catalina.core.StandardService startInternal
[*]INFO: Starting service Catalina
[*]Jan 26, 2013 12:04:34 AM org.apache.catalina.core.StandardEngine startInternal
[*]INFO: Starting Servlet Engine: Apache Tomcat/7.0.26
[*]Jan 26, 2013 12:05:07 AM org.apache.catalina.util.SessionIdGenerator createSecureRandom
[*]INFO: Creation of SecureRandom instance for session ID generation using took milliseconds.
[*]Jan 26, 2013 12:05:07 AM de.javakaffee.web.msm.MemcachedSessionService startInternal
[*]INFO: MemcachedSessionService starts initialization... (configured nodes definition n1:192.168.1.104:11211,n2:192.168.1.104:11311, failover nodes n2)
[*]2013-01-26 00:05:07.873 INFO net.spy.memcached.MemcachedConnection:Added {QA sa=/192.168.1.104:11211, #Rops=0, #Wops=0, #iq=0, topRop=null, topWop=null, toWrite=0, interested=0} to connect queue
[*]2013-01-26 00:05:07.876 INFO net.spy.memcached.MemcachedConnection:Added {QA sa=/192.168.1.104:11311, #Rops=0, #Wops=0, #iq=0, topRop=null, topWop=null, toWrite=0, interested=0} to connect queue
[*]Jan 26, 2013 12:05:07 AM de.javakaffee.web.msm.RequestTrackingHostValve   
[*]INFO: Setting ignorePattern to .*\.(png|gif|jpg|css|js)$
[*]2013-01-26 00:05:07.896 INFO net.spy.memcached.MemcachedConnection:Connection state changed for sun.nio.ch.SelectionKeyImpl@701a27
[*]2013-01-26 00:05:07.897 INFO net.spy.memcached.MemcachedConnection:Connection state changed for sun.nio.ch.SelectionKeyImpl@1d5a0
[*]Jan 26, 2013 12:05:07 AM de.javakaffee.web.msm.MemcachedSessionService setLockingMode
[*]INFO: Setting lockingMode to null
[*]Jan 26, 2013 12:05:07 AM de.javakaffee.web.msm.MemcachedSessionService createTranscoderFactory
[*]INFO: Creating transcoder factory de.javakaffee.web.msm.serializer.javolution.JavolutionTranscoderFactory
[*]Jan 26, 2013 12:05:07 AM de.javakaffee.web.msm.MemcachedSessionService startInternal
[*]INFO: MemcachedSessionService finished initialization, sticky true, operation timeout 1000, with node ids and failover node ids
[*]Jan 26, 2013 12:05:08 AM org.apache.catalina.startup.HostConfig deployDirectory
[*]INFO: Deploying web application directory /home/shirdrn/servers/cluster/nginx_tomcat_memcached/tomcat-2/webapps/host-manager
[*]Jan 26, 2013 12:05:08 AM org.apache.catalina.startup.HostConfig deployDirectory
[*]INFO: Deploying web application directory /home/shirdrn/servers/cluster/nginx_tomcat_memcached/tomcat-2/webapps/manager
[*]Jan 26, 2013 12:05:08 AM org.apache.catalina.startup.HostConfig deployDirectory
[*]INFO: Deploying web application directory /home/shirdrn/servers/cluster/nginx_tomcat_memcached/tomcat-2/webapps/docs
[*]Jan 26, 2013 12:05:08 AM org.apache.catalina.startup.HostConfig deployDirectory
[*]INFO: Deploying web application directory /home/shirdrn/servers/cluster/nginx_tomcat_memcached/tomcat-2/webapps/examples
[*]Jan 26, 2013 12:05:08 AM org.apache.catalina.startup.HostConfig deployDirectory
[*]INFO: Deploying web application directory /home/shirdrn/servers/cluster/nginx_tomcat_memcached/tomcat-2/webapps/ROOT
[*]Jan 26, 2013 12:05:08 AM org.apache.coyote.AbstractProtocol start
[*]INFO: Starting ProtocolHandler ["http-bio-9088"]
[*]Jan 26, 2013 12:05:08 AM org.apache.coyote.AbstractProtocol start
[*]INFO: Starting ProtocolHandler ["ajp-bio-9089"]
[*]Jan 26, 2013 12:05:08 AM org.apache.catalina.startup.Catalina start
[*]INFO: Server startup in 34163 ms




已经成功启动了。



Nginx安装、配置、启动



在Ubuntu系统下,如果没有安装Nginx,可以执行如下命令进行安装:





view plaincopy

[*]sudo apt-get install nginx


默认情况下,Nginx使用默认的配置文件/etc/nginx/nginx.conf,该配置文件直接包含了虚拟目录配置文件/etc/nginx/sites-available/default,修改该文件的内容为,如下所示:





view plaincopy

[*]# You may add here your
[*]# server {
[*]#   ...
[*]# }
[*]# statements for each of your virtual hosts to this file
[*]
[*]##
[*]# You should look at the following URL's in order to grasp a solid understanding
[*]# of Nginx configuration files in order to fully unleash the power of Nginx.
[*]# http://wiki.nginx.org/Pitfalls
[*]# http://wiki.nginx.org/QuickStart
[*]# http://wiki.nginx.org/Configuration
[*]#
[*]# Generally, you will want to move this file somewhere, and start with a clean
[*]# file but keep this around for reference. Or just disable in sites-enabled.
[*]#
[*]# Please see /usr/share/doc/nginx-doc/examples/ for more detailed examples.
[*]##
[*]
[*]
[*]
[*]upstream dev.shirdrn.org {
[*]   server 192.168.1.104:8088 weight=1;
[*]   server 192.168.1.104:9088 weight=1;
[*]}
[*]
[*]
[*]server {
[*]   #listen   80; ## listen for ipv4; this line is default and implied
[*]   #listen   [::]:80 default ipv6only=on; ## listen for ipv6
[*]
[*]   root /usr/share/nginx/www/session;# session是我们配置的虚拟目录,实际直接指向Tomcat下名为session的web应用
[*]   index index.html index.htm;
[*]
[*]   # Make site accessible from http://localhost/
[*]#######   server_name localhost;
[*]   server_name dev.shirdrn.org; # Nginx服务所在主机
[*]   charset utf-8;
[*]
[*]   location / {
[*]          proxy_pass   http://dev.shirdrn.org; #直接代理tomcat集群
[*]          proxy_set_headerX-Real-IP$remote_addr;
[*]          client_max_body_size100m;
[*]
[*]
[*]
[*]          # First attempt to serve request as file, then
[*]          # as directory, then fall back to index.html
[*]#######          try_files $uri $uri/ /index.html;
[*]          # Uncomment to enable naxsi on this location
[*]          # include /etc/nginx/naxsi.rules
[*]   }
[*]
[*]   location ~ ^/(WEB-INF)/ { # 禁止访问Tomcat下web应用的WEB-INF目录下的资源
[*]          deny all;
[*]   }
[*]
[*]   location /doc/ {
[*]          alias /usr/share/doc/;
[*]          autoindex on;
[*]          allow 127.0.0.1;
[*]          deny all;
[*]   }
[*]
[*]   # Only for nginx-naxsi : process denied requests
[*]   #location /RequestDenied {
[*]          # For example, return an error code
[*]          #return 418;
[*]   #}
[*]
[*]   #error_page 404 /404.html;
[*]
[*]   # redirect server error pages to the static page /50x.html
[*]   #
[*]   #error_page 500 502 503 504 /50x.html;
[*]   #location = /50x.html {
[*]   #   root /usr/share/nginx/www;
[*]   #}
[*]
[*]   # pass the PHP scripts to FastCGI server listening on 127.0.0.1:9000
[*]   #
[*]   #location ~ \.php$ {
[*]   #   fastcgi_split_path_info ^(.+\.php)(/.+)$;
[*]   #   # NOTE: You should have "cgi.fix_pathinfo = 0;" in php.ini
[*]   #
[*]   #   # With php5-cgi alone:
[*]   #   fastcgi_pass 127.0.0.1:9000;
[*]   #   # With php5-fpm:
[*]   #   fastcgi_pass unix:/var/run/php5-fpm.sock;
[*]   #   fastcgi_index index.php;
[*]   #   include fastcgi_params;
[*]   #}
[*]
[*]   # deny access to .htaccess files, if Apache's document root
[*]   # concurs with nginx's one
[*]   #
[*]   #location ~ /\.ht {
[*]   #   deny all;
[*]   #}
[*]}
[*]
[*]
[*]# another virtual host using mix of IP-, name-, and port-based configuration
[*]#
[*]#server {
[*]#   listen 8000;
[*]#   listen somename:8080;
[*]#   server_name somename alias another.alias;
[*]#   root html;
[*]#   index index.html index.htm;
[*]#
[*]#   location / {
[*]#          try_files $uri $uri/ /index.html;
[*]#   }
[*]#}
[*]
[*]
[*]# HTTPS server
[*]#
[*]#server {
[*]#   listen 443;
[*]#   server_name localhost;
[*]#
[*]#   root html;
[*]#   index index.html index.htm;
[*]#
[*]#   ssl on;
[*]#   ssl_certificate cert.pem;
[*]#   ssl_certificate_key cert.key;
[*]#
[*]#   ssl_session_timeout 5m;
[*]#
[*]#   ssl_protocols SSLv3 TLSv1;
[*]#   ssl_ciphers ALL:!ADH:!EXPORT56:RC4+RSA:+HIGH:+MEDIUM:+LOW:+SSLv3:+EXP;
[*]#   ssl_prefer_server_ciphers on;
[*]#
[*]#   location / {
[*]#          try_files $uri $uri/ /index.html;
[*]#   }
[*]#}


上面配置中根目录为/usr/share/nginx/www,因为我们使用到了一个虚拟目录来对应Tomcat下部署的web应用的path,所以需要在/usr/share/nginx/www下面创建这个对应的虚拟目录:





view plaincopy

[*]shirdrn@dev:~/servers/cluster/nginx_tomcat_memcached$ sudo mkdir /usr/share/nginx/www/session


最后,可以启动Nginx服务器,执行如下命令:





view plaincopy

[*]shirdrn@dev:~$ sudo /etc/init.d/nginx start


查询Nginx服务启动状态:





view plaincopy

[*]shirdrn@dev:~/servers/cluster/nginx_tomcat_memcached$ sudo ps -ef | grep nginx
[*]root      3199   10 Jan25 ?      00:00:00 nginx: master process /usr/sbin/nginx
[*]www-data320031990 Jan25 ?      00:00:00 nginx: worker process
[*]www-data320131990 Jan25 ?      00:00:00 nginx: worker process
[*]www-data320231990 Jan25 ?      00:00:00 nginx: worker process
[*]www-data320331990 Jan25 ?      00:00:00 nginx: worker process
[*]shirdrn   332730950 00:01 pts/5    00:00:00 grep --color=auto nginx






这是就可以通过Nginx访问(代理)Web应用首页了。



集群验证



上面配置完成,服务都已经启动成功,可以直接请求链接http://dev.shirdrn.org/session,就会直接访问部署在Tomcat服务器上的Java Web应用,看到登录页面。

使用前面准备的Web应用session中账号数据进行模拟登录,就可以查看Memcached缓存的数据:



[*]telnet 192.168.1.104 11211


telnet到memcached服务端口,查看缓存统计数据。





view plaincopy

[*]shirdrn@dev:~$ telnet 192.168.1.104 11211
[*]Trying 192.168.1.104...
[*]Connected to 192.168.1.104.
[*]Escape character is '^]'.
[*]
[*]ERROR
[*]stats
[*]STAT pid 3300
[*]STAT uptime 2148
[*]STAT time 1359131741
[*]STAT version 1.4.13
[*]STAT libevent 2.0.16-stable
[*]STAT pointer_size 32
[*]STAT rusage_user 0.052003
[*]STAT rusage_system 0.084005
[*]STAT curr_connections 7
[*]STAT total_connections 9
[*]STAT connection_structures 8
[*]STAT reserved_fds 20
[*]STAT cmd_get 23
[*]STAT cmd_set 6
[*]STAT cmd_flush 0
[*]STAT cmd_touch 0
[*]STAT get_hits 2
[*]STAT get_misses 21
[*]STAT delete_misses 0
[*]STAT delete_hits 0
[*]STAT incr_misses 0
[*]STAT incr_hits 0
[*]STAT decr_misses 0
[*]STAT decr_hits 0
[*]STAT cas_misses 0
[*]STAT cas_hits 0
[*]STAT cas_badval 0
[*]STAT touch_hits 0
[*]STAT touch_misses 0
[*]STAT auth_cmds 0
[*]STAT auth_errors 0
[*]STAT bytes_read 2675
[*]STAT bytes_written 2835
[*]STAT limit_maxbytes 67108864
[*]STAT accepting_conns 1
[*]STAT listen_disabled_num 0
[*]STAT threads 4
[*]STAT conn_yields 0
[*]STAT hash_power_level 16
[*]STAT hash_bytes 262144
[*]STAT hash_is_expanding 0
[*]STAT expired_unfetched 0
[*]STAT evicted_unfetched 0
[*]STAT bytes 858
[*]STAT curr_items 2
[*]STAT total_items 6
[*]STAT evictions 0
[*]STAT reclaimed 0
[*]END




[*]telnet 192.168.1.104 11311






view plaincopy

[*]shirdrn@dev:~$ telnet 192.168.1.104 11311
[*]Trying 192.168.1.104...
[*]Connected to 192.168.1.104.
[*]Escape character is '^]'.
[*]stats
[*]STAT pid 3308
[*]STAT uptime 2398
[*]STAT time 1359132000
[*]STAT version 1.4.13
[*]STAT libevent 2.0.16-stable
[*]STAT pointer_size 32
[*]STAT rusage_user 0.004000
[*]STAT rusage_system 0.128008
[*]STAT curr_connections 7
[*]STAT total_connections 9
[*]STAT connection_structures 8
[*]STAT reserved_fds 20
[*]STAT cmd_get 1
[*]STAT cmd_set 2
[*]STAT cmd_flush 0
[*]STAT cmd_touch 0
[*]STAT get_hits 0
[*]STAT get_misses 1
[*]STAT delete_misses 1
[*]STAT delete_hits 0
[*]STAT incr_misses 0
[*]STAT incr_hits 0
[*]STAT decr_misses 0
[*]STAT decr_hits 0
[*]STAT cas_misses 0
[*]STAT cas_hits 0
[*]STAT cas_badval 0
[*]STAT touch_hits 0
[*]STAT touch_misses 0
[*]STAT auth_cmds 0
[*]STAT auth_errors 0
[*]STAT bytes_read 644
[*]STAT bytes_written 2109
[*]STAT limit_maxbytes 67108864
[*]STAT accepting_conns 1
[*]STAT listen_disabled_num 0
[*]STAT threads 4
[*]STAT conn_yields 0
[*]STAT hash_power_level 16
[*]STAT hash_bytes 262144
[*]STAT hash_is_expanding 0
[*]STAT expired_unfetched 1
[*]STAT evicted_unfetched 0
[*]STAT bytes 0
[*]STAT curr_items 0
[*]STAT total_items 2
[*]STAT evictions 0
[*]STAT reclaimed 1
[*]END







我们可以停掉某个Tomcat服务,然后观察Session会话数据的复制过程,能够保证当前集群中的会话数据不丢失,另一个结点仍然能提供基于该存在Session的服务。





参考链接






[*]https://code.google.com/p/memcached-session-manager
[*]https://code.google.com/p/memcached-session-manager/wiki/SetupAndConfiguration
[*]http://code.google.com/p/kryo/
[*]http://wiki.ubuntu.org.cn/Nginx
[*]http://www.iteye.com/topic/676347
页: [1]
查看完整版本: Nginx+Memcached+Tomcat集群配置实践(Sticky Session)