Oracle EBS的DB Server无法启动

962148150

　　

　　冷备份系统后，然后重启DB Listener和DB没报什么错误，再起通过命令adstrtal.sh apps/apps来启动Apps Server的时候报错
　　[oracle@bej301441 scripts]$ adstrtal.sh apps/apps
You are running adstrtal.sh version 120.15.12010000.3
adstrtal.sh: Database connection could not be established. Either the database is down or the APPS credentials supplied are wrong.
USAGE:  adstrtal.sh <appsusername/appspassword>
        adstrtal.sh <applications_username/applications_password> -secureapps
        adstrtal.sh -nodbchk
adstrtal.sh: exiting with status 1

如果DB有问题的话，那么./adstrtal.sh apps/apps 是不会启动的。

解决过程：
然后检查DB的情况，尝试用sqlplus来连instance.　　[oracle@bej301441 mc3yd213_bej301441]$ sqlplus apps/apps@mc3yd213
SQL*Plus: Release 10.1.0.5.0 - Production on Sat Jun 30 03:18:39 2012
Copyright (c) 1982, 2005, Oracle.  All rights reserved.
ERROR:
ORA-12537: TNS:connection closed
Enter user-name:

　　sqlplus不能连上instance，那么应该是数据库的问题。
　　

　　ping服务器的IP

　　C:\Documents and Settings\ptian>ping 10.182.74.181
Pinging 10.182.74.181 with 32 bytes of data:
Reply from 10.182.74.181: bytes=32 time=229ms TTL=57
Reply from 10.182.74.181: bytes=32 time=230ms TTL=57
Reply from 10.182.74.181: bytes=32 time=230ms TTL=57
Reply from 10.182.74.181: bytes=32 time=228ms TTL=57
Ping statistics for 10.182.74.181:
    Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
    Minimum = 228ms, Maximum = 230ms, Average = 229ms
C:\Documents and Settings\ptian>

　　能够ping通，说明网络没问题
　　

　　SELINUX=disabled，防火墙也是关闭的，不存在防火墙阻挡的问题。

　　[oracle@bej301441 mc3yd213_bej301441]$ vi /etc/selinux/config
# This file controls the state of SELinux on the system.
# SELINUX= can take one of these three values:
#       enforcing - SELinux security policy is enforced.
#       permissive - SELinux prints warnings instead of enforcing.
#       disabled - SELinux is fully disabled.
SELINUX=disabled
# SELINUXTYPE= type of policy in use. Possible values are:
#       targeted - Only targeted network daemons are protected.
#       strict - Full SELinux protection.
SELINUXTYPE=targeted

　　

　　tnsping这个instance
　　C:\Documents and Settings\ptian>tnsping mc3yd213
TNS Ping Utility for 32-bit Windows: Version 10.1.0.2.0 - Production on 30-6月 -2012 18:14:29
Copyright (c) 1997, 2003, Oracle.  All rights reserved.
Used parameter files:
D:\oracle\product\10.1.0\Db_1\NETWORK\ADMIN\sqlnet.ora
Used TNSNAMES adapter to resolve the alias
Attempting to contact (DESCRIPTION = (ADDRESS = (PROTOCOL = tcp) (HOST = bej301441.cn.oracle.com)(PORT = 1522)) (CONNECT_DATA = (SID = mc3yd213)))
OK (770 msec)
C:\Documents and Settings\ptian>

　　这是说明服务器端DB Listener已经启动了。
　　

　　接下来再次检查listener的状态

　　[oracle@bej301441 mc3yd213_bej301441]$ lsnrctl status mc3yd213
LSNRCTL for Linux: Version 10.1.0.5.0 - Production on 30-JUN-2012 03:50:09
Copyright (c) 1991, 2004, Oracle.  All rights reserved.
Connecting to (DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=bej301441)(PORT=1522))(CONNECT_DATA=(SID=mc3yd213)))
STATUS of the LISTENER
------------------------
Alias                     mc3yd213
Version                   TNSLSNR for Linux: Version 11.1.0.7.0 - Production
Start Date                30-JUN-2012 03:09:47
Uptime                    0 days 0 hr. 40 min. 21 sec
Trace Level               off
Security                  ON: Local OS Authentication
SNMP                      OFF
Listener Parameter File   /u01/oracle/mc3yd213/db/tech_st/11.1.0/network/admin/mc3yd213_bej301441/listener.ora
Listener Log File         /u01/oracle/mc3yd213/db/tech_st/11.1.0/log/diag/tnslsnr/bej301441/mc3yd213/alert/log.xml
Listening Endpoints Summary...
  (DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=bej301441.cn.oracle.com)(PORT=1522)))
Services Summary...
Service &quot;mc3yd213&quot; has 1 instance(s).
  Instance &quot;mc3yd213&quot;, status UNKNOWN, has 1 handler(s) for this service...
The command completed successfully

　　确认DB Listener以及启动

　　

　　检查端口
　　[oracle@bej301441 mc3yd213_bej301441]$ netstat -an | grep 1522
tcp        0      0 0.0.0.0:1522                0.0.0.0:*                   LISTEN
tcp        0      0 10.182.74.33:1522           10.182.74.33:50530          TIME_WAIT
tcp    11541      0 ::ffff:10.182.74.33:23775   ::ffff:10.182.74.33:1522    CLOSE_WAIT
tcp    22601      0 ::ffff:10.182.74.33:47311   ::ffff:10.182.74.33:1522    CLOSE_WAIT
tcp    10061      0 ::ffff:10.182.74.33:60382   ::ffff:10.182.74.33:1522    CLOSE_WAIT
tcp    10061      0 ::ffff:10.182.74.33:60390   ::ffff:10.182.74.33:1522    CLOSE_WAIT
tcp    23081      0 ::ffff:10.182.74.33:37178   ::ffff:10.182.74.33:1522    CLOSE_WAIT
tcp     5501      0 ::ffff:10.182.74.33:47700   ::ffff:10.182.74.33:1522    CLOSE_WAIT
tcp    21611      0 ::ffff:10.182.74.33:12133   ::ffff:10.182.74.33:1522    CLOSE_WAIT
tcp    22601      0 ::ffff:10.182.74.33:47304   ::ffff:10.182.74.33:1522    CLOSE_WAIT
tcp    10131      0 ::ffff:10.182.74.33:23782   ::ffff:10.182.74.33:1522    CLOSE_WAIT
tcp    10071      0 ::ffff:10.182.74.33:60285   ::ffff:10.182.74.33:1522    CLOSE_WAIT
tcp    10061      0 ::ffff:10.182.74.33:60379   ::ffff:10.182.74.33:1522    CLOSE_WAIT
tcp    23081      0 ::ffff:10.182.74.33:37175   ::ffff:10.182.74.33:1522    CLOSE_WAIT
tcp    10071      0 ::ffff:10.182.74.33:60282   ::ffff:10.182.74.33:1522    CLOSE_WAIT
tcp    10071      0 ::ffff:10.182.74.33:60284   ::ffff:10.182.74.33:1522    CLOSE_WAIT
tcp    23081      0 ::ffff:10.182.74.33:37171   ::ffff:10.182.74.33:1522    CLOSE_WAIT
tcp     5391      0 ::ffff:10.182.74.33:47697   ::ffff:10.182.74.33:1522    CLOSE_WAIT
tcp    10061      0 ::ffff:10.182.74.33:60375   ::ffff:10.182.74.33:1522    CLOSE_WAIT
tcp    22601      0 ::ffff:10.182.74.33:47306   ::ffff:10.182.74.33:1522    CLOSE_WAIT
tcp    10061      0 ::ffff:10.182.74.33:60377   ::ffff:10.182.74.33:1522    CLOSE_WAIT
tcp    10061      0 ::ffff:10.182.74.33:60385   ::ffff:10.182.74.33:1522    CLOSE_WAIT
tcp    21611      0 ::ffff:10.182.74.33:12131   ::ffff:10.182.74.33:1522    CLOSE_WAIT
tcp    21551      0 ::ffff:10.182.74.33:12135   ::ffff:10.182.74.33:1522    CLOSE_WAIT
tcp     4351      0 ::ffff:10.182.74.33:60392   ::ffff:10.182.74.33:1522    CLOSE_WAIT
tcp    22601      0 ::ffff:10.182.74.33:47308   ::ffff:10.182.74.33:1522    CLOSE_WAIT
tcp     5971      0 ::ffff:10.182.74.33:29442   ::ffff:10.182.74.33:1522    CLOSE_WAIT
tcp    22601      0 ::ffff:10.182.74.33:47310   ::ffff:10.182.74.33:1522    CLOSE_WAIT
tcp     4271      0 ::ffff:10.182.74.33:60399   ::ffff:10.182.74.33:1522    CLOSE_WAIT
tcp     9811      0 ::ffff:10.182.74.33:60397   ::ffff:10.182.74.33:1522    CLOSE_WAIT
tcp    10071      0 ::ffff:10.182.74.33:23779   ::ffff:10.182.74.33:1522    CLOSE_WAIT
tcp    19661      0 ::ffff:10.182.74.33:57255   ::ffff:10.182.74.33:1522    CLOSE_WAIT
tcp     5381      0 ::ffff:10.182.74.33:15048   ::ffff:10.182.74.33:1522    CLOSE_WAIT
tcp    10131      0 ::ffff:10.182.74.33:23781   ::ffff:10.182.74.33:1522    CLOSE_WAIT
tcp    10071      0 ::ffff:10.182.74.33:60278   ::ffff:10.182.74.33:1522    CLOSE_WAIT
tcp     5791      0 ::ffff:10.182.74.33:29440   ::ffff:10.182.74.33:1522    CLOSE_WAIT
tcp    10071      0 ::ffff:10.182.74.33:60280   ::ffff:10.182.74.33:1522    CLOSE_WAIT
tcp    21611      0 ::ffff:10.182.74.33:12138   ::ffff:10.182.74.33:1522    CLOSE_WAIT
tcp    10061      0 ::ffff:10.182.74.33:60383   ::ffff:10.182.74.33:1522    CLOSE_WAIT
tcp    23081      0 ::ffff:10.182.74.33:37169   ::ffff:10.182.74.33:1522    CLOSE_WAIT
tcp    10071      0 ::ffff:10.182.74.33:60287   ::ffff:10.182.74.33:1522    CLOSE_WAIT
tcp    21611      0 ::ffff:10.182.74.33:12137   ::ffff:10.182.74.33:1522    CLOSE_WAIT
tcp    21611      0 ::ffff:10.182.74.33:12140   ::ffff:10.182.74.33:1522    CLOSE_WAIT
tcp    22601      0 ::ffff:10.182.74.33:47313   ::ffff:10.182.74.33:1522    CLOSE_WAIT
tcp    11541      0 ::ffff:10.182.74.33:23777   ::ffff:10.182.74.33:1522    CLOSE_WAIT
tcp    23031      0 ::ffff:10.182.74.33:37176   ::ffff:10.182.74.33:1522    CLOSE_WAIT
tcp    11541      0 ::ffff:10.182.74.33:23784   ::ffff:10.182.74.33:1522    CLOSE_WAIT
tcp     5851      0 ::ffff:10.182.74.33:29462   ::ffff:10.182.74.33:1522    CLOSE_WAIT
tcp    23031      0 ::ffff:10.182.74.33:37173   ::ffff:10.182.74.33:1522    CLOSE_WAIT
tcp     4271      0 ::ffff:10.182.74.33:60394   ::ffff:10.182.74.33:1522    CLOSE_WAIT
tcp    22581      0 ::ffff:10.182.74.33:47691   ::ffff:10.182.74.33:1522    CLOSE_WAIT

　　

　　进程中的tns
　　[oracle@bej301441 mc3yd213_bej301441]$ ps -ef|grep tns
root        13     2  0 May24 ?        00:00:00 [netns]
oracle   13616     1  0 May27 ?        00:00:32 /u01/oracle/mc3yd213/apps/tech_st/10.1.2/bin/tnslsnr LISTENER -inherit
oracle   22183     1  0 02:55 ?        00:00:00 /u01/oracle/mc3yd213/apps/tech_st/10.1.2/bin/tnslsnr APPS_mc3yd213 -inherit
oracle   22486     1  0 03:09 ?        00:00:00 /u01/oracle/mc3yd213/db/tech_st/11.1.0/bin/tnslsnr mc3yd213 -inherit
oracle   28638 20816  0 03:51 pts/8    00:00:00 grep tns
[oracle@bej301441 mc3yd213_bej301441]$

　　

　　 /etc/hosts

　　[oracle@bej301441 mc3yd213_bej301441]$ cat /etc/hosts
# Do not remove the following line, or various programs
# that require network functionality will fail.
127.0.0.1               localhost.localdomain localhost
::1             localhost6.localdomain6 localhost6
10.182.74.33            bej301441.cn.oracle.com bej301441
10.182.74.181 bej301589.cn.oracle.com
138.1.191.58 rinoid06.us.oracle.com
138.1.191.59 rinoid07.us.oracle.com

　　

　　$TNS_ADMIN/listener.ora

　　[oracle@bej301441 mc3yd213_bej301441]$ cd $TNS_ADMIN
[oracle@bej301441 admin]$ cat listener.ora
# $Header: admk80ln_ux.sql 120.9 2007/03/09 04:31:05 sbandla ship $
# LISTENER.ORA For Oracle Applications
# This file is automatically generated
APPS_mc3yd213 =
  (ADDRESS_LIST =
    (ADDRESS= (PROTOCOL= TCP)(Host= bej301441)(Port= 1627))
  )
SID_LIST_APPS_mc3yd213 =
  (SID_LIST =
    ( SID_DESC = ( SID_NAME = FNDSM )
                 ( ORACLE_HOME = /u01/oracle/mc3yd213/apps/tech_st/10.1.2 )
                 ( PROGRAM = /u01/oracle/mc3yd213/apps/apps_st/appl/fnd/12.0.0/bin/FNDSM )
                 ( envs='MYAPPSORA=/u01/oracle/mc3yd213/apps/apps_st/appl/APPSmc3yd213_bej301441.env,PATH=/usr/bin:/usr/ccs/bin:/bin,FNDSM_SCRIPT=/u01/oracle/mc3yd213/inst/apps/mc3yd213_bej301441/admin/scripts/gsmstart.sh' )
    )
    ( SID_DESC = ( SID_NAME = FNDFS  )
                 ( ORACLE_HOME = /u01/oracle/mc3yd213/apps/tech_st/10.1.2 )
                 ( PROGRAM = /u01/oracle/mc3yd213/apps/apps_st/appl/fnd/12.0.0/bin/FNDFS )
                 ( envs='EPC_DISABLED=TRUE,NLS_LANG=American_America.AL32UTF8,LD_LIBRARY_PATH=/u01/oracle/mc3yd213/apps/tech_st/10.1.2/lib32:/u01/oracle/mc3yd213/apps/tech_st/10.1.2/lib:/usr/X11R6/lib:/usr/openwin/lib:/u01/oracle/mc3yd213/apps/tech_st/10.1.2/jdk/jre/lib/i386:/u01/oracle/mc3yd213/apps/tech_st/10.1.2/jdk/jre/lib/i386/server:/u01/oracle/mc3yd213/apps/tech_st/10.1.2/jdk/jre/lib/i386/native_threads:/u01/oracle/mc3yd213/apps/apps_st/appl/sht/12.0.0/lib,SHLIB_PATH=/u01/oracle/mc3yd213/apps/tech_st/10.1.2/lib32:/u01/oracle/mc3yd213/apps/tech_st/10.1.2/lib:/usr/X11R6/lib:/usr/openwin/lib:/u01/oracle/mc3yd213/apps/tech_st/10.1.2/jdk/jre/lib/i386:/u01/oracle/mc3yd213/apps/tech_st/10.1.2/jdk/jre/lib/i386/server:/u01/oracle/mc3yd213/apps/tech_st/10.1.2/jdk/jre/lib/i386/native_threads:/u01/oracle/mc3yd213/apps/apps_st/appl/sht/12.0.0/lib,LIBPATH=/u01/oracle/mc3yd213/apps/tech_st/10.1.2/lib32:/u01/oracle/mc3yd213/apps/tech_st/10.1.2/lib:/usr/X11R6/lib:/usr/openwin/lib:/u01/oracle/mc3yd213/apps/tech_st/10.1.2/jdk/jre/lib/i386:/u01/oracle/mc3yd213/apps/tech_st/10.1.2/jdk/jre/lib/i386/server:/u01/oracle/mc3yd213/apps/tech_st/10.1.2/jdk/jre/lib/i386/native_threads:/u01/oracle/mc3yd213/apps/apps_st/appl/sht/12.0.0/lib,APPLFSTT=mc3yd213_BALANCE;mc3yd213_FO;mc3yd213,APPLFSWD=/u01/oracle/mc3yd213/inst/apps/mc3yd213_bej301441/appl/admin;/u01/oracle/mc3yd213/inst/apps/mc3yd213_bej301441/appltmp;/u01/oracle/mc3yd213/apps/apps_st/comn/webapps/oacore/html/oam/nonUix/launchMode/restricted')
    )
  )
STARTUP_WAIT_TIME_APPS_mc3yd213 = 0
CONNECT_TIMEOUT_APPS_mc3yd213 = 10
TRACE_LEVEL_APPS_mc3yd213 = OFF
LOG_DIRECTORY_APPS_mc3yd213 = /u01/oracle/mc3yd213/inst/apps/mc3yd213_bej301441/logs/ora/10.1.2/network
LOG_FILE_APPS_mc3yd213 = APPS_mc3yd213
TRACE_DIRECTORY_APPS_mc3yd213 = /u01/oracle/mc3yd213/inst/apps/mc3yd213_bej301441/logs/ora/10.1.2/network
TRACE_FILE_APPS_mc3yd213 = APPS_mc3yd213
ADMIN_RESTRICTIONS_APPS_mc3yd213 = ON
IFILE = /u01/oracle/mc3yd213/inst/apps/mc3yd213_bej301441/ora/10.1.2/network/admin/mc3yd213_bej301441_listener_ifile.ora
SUBSCRIBE_FOR_NODE_DOWN_EVENT_APPS_mc3yd213 = OFF

[oracle@bej301441 admin]$

　　

　　使用addbctl.sh启动数据库
　　Use addbctl.sh to start DB server.It tell me server is running...
[oracle@bej301441 mc3yd213_bej301441]$ addbctl.sh start immediate
You are running addbctl.sh version 120.1
Starting the database mc3yd213 ...
SQL*Plus: Release 11.1.0.7.0 - Production on Sat Jun 30 07:33:53 2012
Copyright (c) 1982, 2008, Oracle. All rights reserved.
Connected to an idle instance.
ORA-01081: cannot start already-running ORACLE - shut it down first
Disconnected
addbctl.sh: exiting with status 9
[oracle@bej301441 mc3yd213_bej301441]$
提示信息说，数据库已经启动了

　　

　　尝试重启DB
　　addbctl.sh stop immediate >addbctl.sh start发现数据库还是连不上。
　　

　　解决方法

　　使用addbctl.sh stop abort来关闭数据库，再用addbctl.sh start启动。之前关闭DB的时候，没加abort项（用的是immediate），说明当时关闭数据库的时候，数据库并没有完全关闭，所以重启没有成功。

　　关闭项（NORMAL，IMMEDIATE，ABORT）：

　　SHUTDOWN NORMAL：不允许新的连接、等待会话结束、等待事务结束、做一个检查点并关闭数据文件。启动时不需要实例恢复。
SHUTDOWN IMMEDIATE：不允许新的连接、不等待会话结束、不等待事务结束、做一个检查点并关闭数据文件。没有结束的事务是自动rollback的。启动时不需要实例恢复。
SHUTDOWN ABORT：不允许新的连接、不等待会话结束、不等待事务结束、不做检查点且没有关闭数据文件。启动时自动进行实例恢复。

　　

　　ps:我在OTN上发的问题帖：https://forums.oracle.com/forums/thread.jspa?threadID=2408941&tstart=0

　　转载请注明出处：http://blog.iyunv.com/pan_tian/article/details/7705275

======EOF======