Oracle 11g Direct NFS

222e

NFS SERVER 10.10.10.22
NFS CLIENT 10.10.10.12
1、nfs server需要启动的服务：
#service portmap start
Starting portmap: [  OK  ]
#service nfs start
Starting NFS services:  [  OK  ]
Starting NFS quotas: [  OK  ]
Starting NFS daemon: [  OK  ]
Starting NFS mountd: [  OK  ]
Starting RPC idmapd: [  OK  ]
2、nfs client需要启动的服务
#service portmap start
Starting portmap: [  OK  ]

3、在NFS SERVICE端配置NFS
   root用户编辑文件/etc/exports内容如下：
/u01/nfs 10.10.10.12(rw,no_root_squash,insecure)
检查配置结果：
[iyunv@db11g-em10g nfs]# exportfs
/u01/nfs        10.10.10.12
[iyunv@db11g-em10g nfs]# exportfs -rv
exporting 10.10.10.12:/u01/nfs
4、在NFS Client端root用户挂载NFS文件系统
[iyunv@db11g oracle]# mount 10.10.10.22:/u01/nfs /oracle/nfs

5、配置NFS CLIENT数据库支持Direct NFS
   oracle用户编辑$ORACLE_HOME/dbs/oranfstab文件，内容如下
[oracle@db11g ~]$ cat $ORACLE_HOME/dbs/oranfstab
server:db11g-em10g
path:10.10.10.22
export:/u01/nfs mount:/oracle/nfs
   修改libodm11.so库文件的链接对象
$cd $ORACLE_HOME/lib
$mv libodm11.so libodm11.so.bak
$ln -s libnfsodm11.so libodm11.so
数据库启动的告警日志中有如下信息：
Oracle instance running with ODM: Oracle Direct NFS ODM Library Version 2.0

6、在NFS Client端创建表空间测试：
SQL> select * from v$dnfs_servers
        ID SVRNAME         DIRNAME                 MNTPORT    NFSPORT      WTMAX      RTMAX
---------- --------------- -------------------- ---------- ---------- ---------- ----------
         6 db11g-em10g  /u01/nfs                    718       2049          0          0

SQL> create tablespace dnfs datafile '/oracle/nfs/dnfs01.dbf' size 2M;
SQL> select * from v$dnfs_files
FILENAME                                   FILESIZE       PNUM     SVR_ID
---------------------------------------- ---------- ---------- ----------
/oracle/nfs/dnfs01.dbf                      2105344          9          5
7、检查NFS SERVER端口的链接情况：
[iyunv@db11g-em10g nfs]# netstat -an |grep 2049
tcp        0      0 0.0.0.0:2049                0.0.0.0:*                   LISTEN     
tcp        0      0 10.10.10.22:2049            10.10.10.12:23052           ESTABLISHED
tcp        0      0 10.10.10.22:2049            10.10.10.12:728             ESTABLISHED
tcp        0      0 10.10.10.22:2049            10.10.10.12:56522           ESTABLISHED
udp        0      0 0.0.0.0:2049                0.0.0.0:*                              

发现的问题：
1、NFS CLient端必须要启动portmap服务，否者在client端mount nfs文件出错：
[iyunv@db11g oracle]# mount 10.10.10.22:/u01/nfs /oracle/nfs
mount.nfs: Input/output error
[iyunv@db11g oracle]# service portmap start
Starting portmap: [  OK  ]
[iyunv@db11g oracle]# mount 10.10.10.22:/u01/nfs /oracle/nfs

2、NFS SERVER端的配置文件/etc/exports使用默认的内容：
[iyunv@db11g-em10g nfs]# cat /etc/exports
/u01/nfs 10.10.10.12(rw,sync,no_root_squash)
那么在Client端创建表空间时候出现如下的错误：
[oracle@db11g dbs]$ sqlplus / as sysdba
SQL*Plus: Release 11.1.0.7.0 - Production on Sat Dec 22 15:33:19 2012
Copyright (c) 1982, 2008, Oracle.  All rights reserved.

Connected to:
Oracle Database 11g Enterprise Edition Release 11.1.0.7.0 - Production
With the Partitioning and Real Application Testing options
SQL> create tablespace nfsdemo datafile '/oracle/nfs/nfsdemo01.dbf' size 2M;
create tablespace nfsdemo datafile '/oracle/nfs/nfsdemo01.dbf' size 2M
*
ERROR at line 1:
ORA-01119: error in creating database file '/oracle/nfs/nfsdemo01.dbf'
ORA-27086: unable to lock file - already in use
Linux Error: 37: No locks available
Additional information: 10
告警日志提示如下错误：
Sat Dec 22 15:38:18 2012
ORA-1119 signalled during: create tablespace dnfs datafile '/oracle/nfs/dnfs01.dbf' size 2M...
Sat Dec 22 15:46:52 2012
create tablespace dnfs datafile '/oracle/nfs/dnfs01.dbf' size 2M
Direct NFS: NFS3ERR 1 Not owner. path db11g-em10g mntport 718 nfsport 2049

Direct NFS Client
NAS storage devices use the Network File System (NFS) to access data. In Oracle Database 10g, NAS devices were accessed using operating system–specific kernel NFS drivers. This required that you tune many parameters, and the configuration varied across the platforms. The NFS clients tended to be inconsistent across the different operating system platforms. Manageability wasn’t easy because you had to contend with over 20 configuration parameters.
In Oracle Database 11g, the Oracle NFS implements the NFS Version 3 protocol in the Oracle RDBMS kernel. Implementing the Oracle Direct NFS offers the following benefits:
Avoids the bottlenecks and resource constraints by avoiding the kernel NFS layer.
Provides a common NFS interface for Oracle for use on all operating system platforms and supported NFS servers.
Provides load balancing across multiple connections to the NFS servers, thus improving performance.
Performance is predictable because the Oracle NFS implementation enables you to completely control the input/output path to the Network File Servers.
Easier management including simpler configuration and superior diagnosability.
Configuring Direct NFS
You don’t have to configure much to implement Oracle Direct NFS Client. Direct NFS searches for the mount point entries in the following order:
$ORCLE_HOME/dbs/oranfstab
/etc/orafnstab
/etc/mtab
The database uses the first match as the mount point. Oracle always requires that even when you use Direct NFS, the kernel NFS system must perform the mounting. For this reason, Oracle will always crosscheck the information about mount points in the oranfstab file with the operating system NFS mount points. If there’s a mismatch, Direct NFS can’t act as a client to the NFS server and stops serving the NFS server.
Enabling Direct NFS
You must follow these steps to enable Direct NFS:
1.You must mount all NFS mount points with your kernel NFS client. You must make sure you mount any file systems you plan on using through ODM NFS and make the file systems available to Oracle over regular NFS mounts.
2.If you want to specify Oracle-specific options to Direct NFS, you’ll need an oranfstab file. This is an optional step. The oranfstab file must have the following attributes so the database can access all NFS servers through Direct NFS:
server: Provides the NFS server name.
mount: Provides the local mount point for the NFS server.
export: Provides the exported path from the NFS server.
path: Provides the network path to the NFS server. You can specify up to four network paths with an IP address or by name. Using multiple network paths enables the Direct NFS client to use an alternate path if the current path fails. Multiple paths also enable the client to perform load balancing.
Mnt_timeout: Specifies (in seconds) the time Direct NFS Client should wait for a successful mount before timing out. This parameter is optional. The default timeout is 10 minutes (600).
Dontroute: Specifies that outgoing messages should not be routed by the operating system, but instead sent using the IP address to which they are bound. Note that this POSIX option sometimes does not work on Linux systems with multiple paths in the same subnet.
A typical oranfstab file looks similar to the following:
server: TestServer1
path: 130.33.34.11
export: /vol/oradata1
mount: /mnt/oradata1
In order to remove an NFS path that the database is using currently, you must restart the database.
3. You must replace the standard ODM library, libnfsodm10.so, with the ODM NFS library, as shown here:
$ cd $ORACLE_HOME/lib
$ cp libodmll.so libodmll.so_stub
$ ln -s libnfsodm11.so libodm11.so
You can disable the Direct NFS client by using any of the following three methods.
Delete the oranfstab file.
Replace the ODM NFS library with the stub libodm11.so file.
Modify the oranfstab file by deleting the specific NFS server or the network paths to the NFS server.
If the database can’t open the NFS server using Direct NFS, it will use the operating system kernel client instead.
Monitoring Direct NFS
You can query the following views in order to manage Direct NFS:
V$DNFS_STATS Shows performance statistics for Direct NFS.
V$DNFS_SERVERS Shows servers accessed by Direct NFS.
V$DNFS_FILES Shows files currently using Direct NFS.
V$DNFS_CHANNELS Shows the open network paths being used by Direct NFS.