Oracle ORA-03137: TTC protocol internal error : [12333] 故障分析

富1682

　　

　　
　　
　　
　　程序通过JDBC 连接数据库异常，报 ORA-03137[12333]的错误。
　　
　　当前程序的JDBC 驱动版本：ojdbc16-11.2.0.1.0.jar
　　数据库版本： 11.2.0.3
　　
　　
一. Log 信息
　　
1.1 alert log
　　TNS-12637: Packet receive failed
　　ns secondary err code: 12532
　　nt main err code: 0
　　nt secondary err code: 0
　　nt OS err code: 0
　　opiodr aborting process unknown ospid(28518) as a result of ORA-609
　　Fri Aug 09 15:41:20 2013
　　Errors in file /u01/app/oracle/diag/rdbms/tserpdb/tserpdb/trace/tserpdb_ora_28329.trc (incident=60746):
　　ORA-03137: TTC protocolinternal error : [12333] [19] [3] [14] [] [] [] []
　　Incident details in:/u01/app/oracle/diag/rdbms/tserpdb/tserpdb/incident/incdir_60746/tserpdb_ora_28329_i60746.trc
　　Fri Aug 09 15:41:21 2013
　　Sweep [inc][60746]: completed
　　Sweep [inc2][60746]: completed
　　Fri Aug 09 15:41:21 2013
　　Dumping diagnostic data indirectory=[cdmp_20130809154121], requested by (instance=1, osid=28329),summary=[incident=60746].
　　Fri Aug 09 15:59:42 2013
　　
　　
1.2 trace 文件
　　
　　Dump file/u01/app/oracle/diag/rdbms/tserpdb/tserpdb/incident/incdir_60746/tserpdb_ora_28329_i60746.trc
　　Oracle Database 11g Enterprise EditionRelease 11.2.0.3.0 - 64bit Production
　　With the Partitioning, Automatic StorageManagement, OLAP, Data Mining
　　and Real Application Testing options
　　ORACLE_HOME =/u01/app/oracle/product/11.2.0/dbhome_1
　　System name: Linux
　　Node name: sh-tsiagent-003180
　　Release: 2.6.18-308.el5
　　Version: #1SMP Tue Feb 21 20:06:06 EST 2012
　　Machine: x86_64
　　Instance name: tserpdb
　　Redo thread mounted by this instance: 1
　　Oracle process number: 92
　　Unix process pid: 28329, image:oracle@sh-tsiagent-003180
　　
　　
　　*** 2013-08-09 15:41:20.138
　　*** SESSION ID:(1911.53519) 2013-08-0915:41:20.138
　　*** CLIENT ID:() 2013-08-09 15:41:20.138
　　*** SERVICE NAME:(tsdb3180.develop)2013-08-09 15:41:20.138
　　*** MODULE NAME:(JDBC Thin Client)2013-08-09 15:41:20.138
　　*** ACTION NAME:() 2013-08-09 15:41:20.138
　　
　　Dump continued from file:/u01/app/oracle/diag/rdbms/tserpdb/tserpdb/trace/tserpdb_ora_28329.trc
　　ORA-03137: TTC protocol internal error :[12333] [19] [3] [14] [] [] [] []
　　
　　========= Dump for incident 60746 (ORA 3137[12333]) ========
　　
　　*** 2013-08-09 15:41:20.139
　　dbkedDefDump(): Starting incident defaultdumps (flags=0x2, level=3, mask=0x0)
　　----- Current SQL Statement for thissession (sql_id=35tr4jfq0x3jw) -----
　　select mpzone0_.ZONEID as ZONEID83_0_,mpzone0_.AREACODE as AREACODE83_0_, mpzone0_.CARDTYPE as CARDTYPE83_0_,mpzone0_.city as city83_0_, mpzone0_.province as province83_0_ fromIAGENT.MPZONE mpzone0_ where mpzone0_.ZONEID=:1
　　
　　----- Call Stack Trace -----
　　calling call entry argument values in hex
　　location type point (? means dubious value)
　　-------------------- ---------------------------- ----------------------------
　　skdstdst()+36 callkgdsdst() 000000000 ?000000000 ?
　　7FFF91403248 ? 000000001 ?
　　000000001 ?000000002 ?
　　…
　　__libc_start_main() callmain() 000000002 ?7FFF91414458 ?
　　+244000000001 ? 000000000 ?
　　009B910C0 ? 000000000 ?
　　_start()+36 call __libc_start_main() 000A0AF38 ? 000000002 ?
　　7FFF91414448 ? 000000000 ?
　　009B910C0 ? 000000002 ?
　　…
　　
　　--------------------- Binary Stack Dump---------------------
　　
　　
　　========== FRAME [1] (skdstdst()+36 ->kgdsdst()) ==========
　　defined by frame pointers0x7fff91407b00 and 0x7fff91407af0
　　CALL TYPE: call ERROR SIGNALED: no COMPONENT: (null)
　　RDI 0000000000000000 RSI 0000000000000000RDX 00007FFF91403248
　　RCX 0000000000000001 R8 0000000000000001 R90000000000000002
　　RAX 0000000000000000 RBX 0000000000000003RBP 00007FFF91407B00
　　R10 00007FFF913FFA80 R11 0000000000000000R12 0000000000000003
　　R13 0000000000000002 R14 0000000000000000R15 0000000000000001
　　RSP 00007FFF91407B00 RIP 0000000006772B38
　　Dump of memory from 0x7fff91407af0 to0x7fff91407b00
　　7FFF91407AF0 91407B00 00007FFF 06772B3D00000000 [.{@.....=+w.....]
　　
　　========== FRAME [2] (ksedst1()+98 ->skdstdst()) ==========
　　defined by frame pointers0x7fff91407bb0 and 0x7fff91407b00
　　CALL TYPE: call ERROR SIGNALED: no COMPONENT: KSE
　　
1.3 Java 错误
　　
　　org.springframework.transaction.TransactionSystemException:Could not commit Hibernate transaction; nested exception isorg.hibernate.TransactionException: JDBC commit failed
　　
　　org.springframework.orm.hibernate3.HibernateTransactionManager.doCommit(HibernateTransactionManager.java:660)
　　
　　org.springframework.transaction.support.AbstractPlatformTransactionManager.processCommit(AbstractPlatformTransactionManager.java:754)
　　
　　org.springframework.transaction.support.AbstractPlatformTransactionManager.commit(AbstractPlatformTransactionManager.java:723)
　　
　　org.springframework.transaction.interceptor.TransactionAspectSupport.commitTransactionAfterReturning(TransactionAspectSupport.java:394)
　　
　　.
　　
　　.
　　
　　.
　　
　　java.sql.SQLRecoverableException:No more data to read from socket
　　oracle.jdbc.driver.T4CMAREngine.unmarshalUB1(T4CMAREngine.java:1200)
　　oracle.jdbc.driver.T4CMAREngine.unmarshalSB1(T4CMAREngine.java:1155)
　　oracle.jdbc.driver.T4CTTIfun.receive(T4CTTIfun.java:279)
　　oracle.jdbc.driver.T4CTTIfun.doRPC(T4CTTIfun.java:186)
　　oracle.jdbc.driver.T4C7Ocommoncall.doOCOMMIT(T4C7Ocommoncall.java:75)
　　oracle.jdbc.driver.T4CConnection.doCommit(T4CConnection.java:558)
　　oracle.jdbc.driver.PhysicalConnection.commit(PhysicalConnection.java:3674)
　　oracle.jdbc.driver.PhysicalConnection.commit(PhysicalConnection.java:3680)
　　org.apache.tomcat.dbcp.dbcp.DelegatingConnection.commit(DelegatingConnection.java:334)
　　org.apache.tomcat.dbcp.dbcp.PoolingDataSource$PoolGuardConnectionWrapper.commit(PoolingDataSource.java:211)
　　
　　org.hibernate.transaction.JDBCTransaction.commitAndResetAutoCommit(JDBCTransaction.java:166)
　　
　　
　　
二.解决方法
　　
　　在MOS上搜了一下，该问题与Bug9445675 非常像。
　　
　　
　　Bug 9445675 NO MOREDATA TO READ FROM SOCKET WHEN USING END-TO-END METRICS
　　This bug does affect the JDBC driver.This bug may be the cause when all of the following conditions are met:
　　1)You are using the 10.1.x.x orthe 11.2.0.1 JDBC driver; the bug does not affect 10.2.x.x, or 11.1.x.xversions of the driver, nor versions 11.2.0.2 or above
　　2)You are using end-to-endmetrics in your Java code
　　3)The server side ORA-3137[12333] error is accompanied by the client side Java exception "No moredata to read from socket"
　　This bug is fixed in the11.2.0.2 version of the JDBC driver and above.
　　
　　
　　而当前版本的JDBC Driver 也是：11.2.0.1.0。 所以将JDBC DRIVER升级到11.2.0.2 之后，故障解决。
　　
　　
　　参考MOS：
　　Understanding and Diagnosing ORA-00600[12333] ORA-3137 [12333] Errors (文档 ID 389713.1)
　　ORA-03137 TTC Protocol Internal Error [12333] Using JDBC Driver (文档 ID 752297.1)
　　Troubleshooting ORA-3137 [12333] ErrorsEncountered When Using Oracle JDBC Driver (文档 ID1361107.1)
　　
　　
　　
　　
　　
　　
　　
　　
　　
　　--------------------------------------------------------------------------------------------
　　版权所有，文章允许转载，但必须以链接方式注明源地址，否则追究法律责任!
　　QQ: 251097186
　　Skype: tianlesoftware
　　Email: tianlesoftware@gmail.com
　　Blog:  http://blog.csdn.net/tianlesoftware
　　Weibo: http://weibo.com/tianlesoftware
　　Twitter: http://twitter.com/tianlesoftware
　　Facebook: http://www.facebook.com/tianlesoftware
　　Linkedin: http://cn.linkedin.com/in/tianlesoftware