赵小黑 发表于 2017-2-4 11:51:39

线上tomcat 网络无法访问问题排查

线上某台机器tomcat启动无法访问,通过jstack报错。
Unable to open socket file: target process not responding or HotSpot VM not loaded
因为这个错误提示误导我以为JDK的问题,升级JDK到1.7问题依旧。
 
这个时候查看日志发现:
INFO: validateJarFile(/home/admin/tlog-proxy/target/tlog-proxy.war/WEB-INF/lib/jsp-api-2.1-glassfish-2.1.v20091210.jar) - jar not loaded. See Servlet Spec 3.0, section 10.7.2. Offending class: javax/el/Expression.class
2015-12-03 02:43:53,332 org.apache.catalina.loader.WebappClassLoader validateJarFile
INFO: validateJarFile(/home/admin/tlog-proxy/target/tlog-proxy.war/WEB-INF/lib/servlet-api-2.5-20081211.jar) - jar not loaded. See Servlet Spec 3.0, section 10.7.2. Offending class: javax/servlet/Servlet.class
2015-12-03 02:44:03,987 org.apache.catalina.startup.HostConfig deployDirectory
INFO: Deployment of web application directory /home/admin/tlog-proxy/.default/deploy/tlog-proxy.war has finished in 10,908 ms
2015-12-03 02:44:03,990 org.apache.coyote.AbstractProtocol start
INFO: Starting ProtocolHandler ["http-bio-7001"]
2015-12-03 02:44:04,051 org.apache.catalina.startup.Catalina start
INFO: Server startup in 11069 ms
2015-12-03 02:44:09,258 org.apache.tomcat.util.net.JIoEndpoint$Acceptor run
SEVERE: Socket accept failed
java.net.SocketException: 打开的文件过多
        at java.net.PlainSocketImpl.socketAccept(Native Method)
        at java.net.AbstractPlainSocketImpl.accept(AbstractPlainSocketImpl.java:398)
        at java.net.ServerSocket.implAccept(ServerSocket.java:530)
        at java.net.ServerSocket.accept(ServerSocket.java:498)
        at org.apache.tomcat.util.net.DefaultServerSocketFactory.acceptSocket(DefaultServerSocketFactory.java:60)
        at org.apache.tomcat.util.net.JIoEndpoint$Acceptor.run(JIoEndpoint.java:220)
        at java.lang.Thread.run(Thread.java:745)
 
lsof | grep java | wc -l发现超过1万,正常应该只有几百。然后查看发现是抓取任务太多导致。
 
事后排查抓取任务才几百个,为啥会创建出上万个连接呢,最后排查发现HttpClient没有调用 close
 
  public void close() {
  if (client != null) {
  client.getConnectionManager().closeIdleConnections(0, TimeUnit.SECONDS);
  }
  }
 
 
页: [1]
查看完整版本: 线上tomcat 网络无法访问问题排查