RedHat 5.6_x86_64 + ASM + RAW+ Oracle 10g RAC （二）

三、安装配置
1.创建账户
在所有节点上创建所需账户
#groupadd –g 500 dba
#useradd -u 1001 -g oinstall -G dba oracle
#password oracle
nobody用户一般操作系统安装往后就存在，如果没有必须创建，该用户在安装完clusterware或RAC后执行一些扩展任务（extjob）
#id nobody
uid=99(nobody) gid=99(nobody) groups=99(nobody)
注意：
在所有节点上执行，并且所有节点上面的用户名、用户ID、组名、组ID必须相同。
2.创建目录
在所有节点上创建所需目录并赋予对应权限
#mkdir –p /home/oracle/orabase
#mkdir –p /home/oracle/orahome/product/10.2.0/db_1
#mkdir –p /home/oracle/orahome/product/10.2.0/crs
#mkdir –p /home/oracle/orahome/product/10.2.0/asm
#cd /home/oracle
#chmod –R 775 *
#chown –R oracle:oinstall *
3.配置环境变量
在JASON1机器上以oracle用户进入，编辑~/.bash_profile文件，添加如下内容：
export ORACLE_BASE=/home/oracle/orabase
export ORACLE_HOME=/home/oracle/orahome/product/10.2.0/db_1
export ORA_CRS_HOME=/home/oracle/orahome/product/10.2.0/crs
export ORA_ASM_HOME=/home/oracle/orahome/product/10.2.0/asm
export ORACLE_SID=JASON1
PATH=$PATH:$ORACLE_HOME/bin:$ORA_CRS_HOME/bin:$ORA_ASM_HOME/bin
export PATH
同样在在JASON2机器上以oracle用户进入，编辑~/.bash_profile文件，添加如下内容：
export ORACLE_BASE=/home/oracle/orabase
export ORACLE_HOME=/home/oracle/orahome/product/10.2.0/db_1
export ORA_CRS_HOME=/home/oracle/orahome/product/10.2.0/crs
export ORA_ASM_HOME=/home/oracle/orahome/product/10.2.0/asm
export ORACLE_SID=JASON2
PATH=$PATH:$ORACLE_HOME/bin:$ORA_CRS_HOME/bin:$ORA_ASM_HOME/bin
export PATH


4.配置hosts文件
在所有节点上，编辑/etc/hosts文件，最终内容如下：
JASON1节点
[oracle@jason1 ~]# vi /etc/hosts
# Do not remove the following line, or various programs
# that require network functionality will fail.
127.0.0.1   localhost.localdomain localhost
::1      localhost6.localdomain6 localhost6
192.168.12.241 jason1
192.168.12.242 jason2
10.10.10.241  jason1-priv
10.10.10.242  jason2-priv
192.168.12.243 jason1-vip
192.168.12.244 jason2-vip


JASON2节点
[oracle@jason2 ~]# vi /etc/hosts
# Do not remove the following line, or various programs
# that require network functionality will fail.
127.0.0.1   localhost.localdomain localhost
::1      localhost6.localdomain6 localhost6
192.168.12.241 jason1
192.168.12.242 jason2
10.10.10.241  jason1-priv
10.10.10.242  jason2-priv
192.168.12.243 jason1-vip
192.168.12.244 jason2-vip
5.SSH配置
在安装使用Oracle RAC之前，必须在所有节点上为oracle用户配置secure shell（SSH）。安装过程中，OUI程序会使用ssh和scp命令来执行远程命令和拷贝文件到其它节点上。要在安装过程中不输入oracle的用户密码，必须正确配置SSH。要建立用户等效性，请在两个节点上以 oracle 用户身份生成用户的公钥和私钥：
在jason1几点执行
[root@jason1~]# su - oracle
[oracle@jason1~]$ mkdir .ssh
[oracle@jason1~]$ chmod 700 .ssh
[oracle@jason1~]$ cd .ssh
[oracle@jason1.ssh]$ ssh-keygen -t rsa
Generatingpublic/private rsa key pair.
Enterfile in which to save the key (/home/oracle/.ssh/id_rsa):
Enterpassphrase (empty for no passphrase):
Entersame passphrase again:
Youridentification has been saved in /home/oracle/.ssh/id_rsa.
Yourpublic key has been saved in /home/oracle/.ssh/id_rsa.pub.
Thekey fingerprint is:
09:6c:a6:f7:28:6d:f9:19:7f:b7:cf:f4:0d:18:b3:eb oracle@jason1
[oracle@jason1.ssh]$ ssh-keygen -t dsa
Generatingpublic/private dsa key pair.
Enterfile in which to save the key (/home/oracle/.ssh/id_dsa):
Enterpassphrase (empty for no passphrase):
Entersame passphrase again:
Youridentification has been saved in /home/oracle/.ssh/id_dsa.
Yourpublic key has been saved in /home/oracle/.ssh/id_dsa.pub.
Thekey fingerprint is:
60:30:21:ad:88:b5:53:ea:5a:b8:06:13:42:3d:9d:09 oracle@jason1
[oracle@jason1.ssh]$


在jason2节点执行
[root@jason2~]# su - oracle
[oracle@jason2~]$ mkdir .ssh
[oracle@jason2~]$ chmod 700 .ssh
[oracle@jason2~]$ cd .ssh
[oracle@jason2.ssh]$ ssh-keygen -t rsa
Generatingpublic/private rsa key pair.
Enterfile in which to save the key (/home/oracle/.ssh/id_rsa):
Enterpassphrase (empty for no passphrase):
Entersame passphrase again:
Youridentification has been saved in /home/oracle/.ssh/id_rsa.
Yourpublic key has been saved in /home/oracle/.ssh/id_rsa.pub.
Thekey fingerprint is:
cd:9b:c3:e6:03:06:2c:81:da:fd:b3:0c:33:38:bc:27 oracle@jason2
[oracle@jason2.ssh]$ ssh-keygen -t dsa
Generatingpublic/private dsa key pair.
Enterfile in which to save the key (/home/oracle/.ssh/id_dsa):
Enterpassphrase (empty for no passphrase):
Entersame passphrase again:
Youridentification has been saved in /home/oracle/.ssh/id_dsa.
Yourpublic key has been saved in /home/oracle/.ssh/id_dsa.pub.
Thekey fingerprint is:
10:90:91:92:cd:65:7b:d7:dc:a6:65:f4:e7:f1:e2:f2oracle@jason2
[oracle@jason2.ssh]$


切回到jason1执行
[oracle@jason1.ssh]$ ssh jason2 cat ~/.ssh/id_rsa.pub >> authorized_keys
Theauthenticity of host 'jason2 (192.168.12.242)' can't be established.
RSAkey fingerprint is ac:40:85:c2:8d:50:01:e7:71:e6:93:90:8d:8e:d8:08.
Areyou sure you want to continue connecting (yes/no)? yes
Warning:Permanently added 'jason2,192.168.12.242' (RSA) to the list of known hosts.
oracle@jason2'spassword:
[oracle@jason1.ssh]$ ssh jason2 cat ~/.ssh/id_dsa.pub >> authorized_keys
oracle@jason2'spassword:
[oracle@jason1.ssh]$ scp authorized_keys jason2:/home/oracle/.ssh/
oracle@jason2'spassword:
authorized_keys                                                                            100% 1996   2.0KB/s  00:00  
[oracle@jason1.ssh]$


两台机器互相访问进行测试，最终达到如下效果为正常情况。
[oracle@jason1.ssh]$ ssh jason1-priv date
ThuNov 7 15:20:04 CST 2013
[oracle@jason1.ssh]$ ssh jason1 date
ThuNov 7 15:20:07 CST 2013
[oracle@jason1.ssh]$ ssh jason2-priv date
ThuNov 7 15:20:10 CST 2013
[oracle@jason1.ssh]$ ssh jason2 date
ThuNov 7 15:20:13 CST 2013
[oracle@jason1.ssh]$


[oracle@jason2.ssh]$ ssh jason2-priv date
ThuNov 7 15:20:43 CST 2013
[oracle@jason2.ssh]$ ssh jason2 date
ThuNov 7 15:20:44 CST 2013
[oracle@jason2.ssh]$ ssh jason1 date
ThuNov 7 15:20:47 CST 2013
[oracle@jason2.ssh]$ ssh jason1-priv date
ThuNov 7 15:20:50 CST 2013
[oracle@jason2.ssh]$
6.检查所需软件包
每个节点上都执行如下命令：
#rpm -q binutils compat-db control-center gcc gcc-c++glibc gnome-libs libstdc++ libstdc++-devel make openmotif21
如有个别包没有安装，请务必从光盘找到相关rpm包进行安装。
7.配置内核参数
在所有节点编辑sysctl.conf文件，添加如下内容：
#vi /etc/sysctl.conf
kernel.sem = 250 32000 100 128
kernel.shmmni = 4096
fs.file-max = 101365
net.ipv4.ip_local_port_range = 9000 65500
net.core.rmem_default = 1048576
net.core.rmem_max = 1048576
net.core.wmem_default = 262144
net.core.wmem_max = 262144
要使配置立即生效，执行sysctl –p
8.设置Shell对Oracle用户的限制
在每个节点上执行相同的操作，编辑/etc/security/limits.conf，添加如下内容：
#For Oracle
oracle     soft  nproc      2047
oracle     hard  nproc      16384
oracle     soft  nofile     1024
oracle     hard  nofile     65536
oracle     soft  memlock     3145728
oracle     hard  memlock     3145728
编辑/etc/pam.d/login，添加如下内容：
#For Oracle
session  required  /lib/security/pam_limits.so
9.配置用户资源限制
在每个节点上执行相同的操作,编辑/etc/profile增加以下内容：
if [ $USER = "oracle" ]; then
  if [ $SHELL ="/bin/ksh" ]; then
    ulimit -p16384
    ulimit -n65536
  else
    ulimit -u16384 -n 65536
  fi
  umask 022
fi


10.建立hangcheck模块加载
在所有节点上检测是否安装了hangcheck模块。
[root@jason1 ~]# find /lib/modules -name"hangcheck-timer.ko"
/lib/modules/2.6.18-238.el5/kernel/drivers/char/hangcheck-timer.ko
方法1：设置模块为自动加载
echo“options hangcheck-timer hangcheck_tick=30hangcheck_margin=180”>>/etc/modprobe.conf
echo “modprobe hangcheck” >> /etc/rc.local
方法2：在/etc/rc.local增加以下内容：
[root@jason1 ~]# insmod /lib/modules/2.6.18-238.el5/kernel/drivers/char/hangcheck-timer.ko hangcheck_tick=30 hangcheck_margin=180
检测模块是否运行
[root@jason1 ~]# lsmod |grep hangcheck
hangcheck_timer    37465 0
11.裸设备设置
在jason1上格式化分区，同时确保在jason2上也可以看到这些分区。同时在所有节点上执行1到5步骤配置裸设备。
[root@jason1 ~]# fdisk -l
Disk /dev/sda: 42.9 GB,42949672960 bytes
255 heads, 63 sectors/track,5221 cylinders
Units = cylinders of 16065 *512 = 8225280 bytes


  Device Boot   Start     End   Blocks Id System
/dev/sda1  *     1     13   104391 83 Linux
/dev/sda2       14     535  4192965  82 Linux swap / Solaris
/dev/sda3       536    5221  37640295  83 Linux


Disk /dev/sdb: 209 MB,209715200 bytes
64 heads, 32 sectors/track,200 cylinders
Units = cylinders of 2048 *512 = 1048576 bytes


  Device Boot   Start     End   Blocks Id System
/dev/sdb1        1     200   204784 83 Linux


Disk /dev/sdc: 209 MB,209715200 bytes
64 heads, 32 sectors/track,200 cylinders
Units = cylinders of 2048 *512 = 1048576 bytes


  Device Boot   Start     End   Blocks Id System
/dev/sdc1        1     200   204784 83 Linux


Disk /dev/sdd: 209 MB,209715200 bytes
64 heads, 32 sectors/track,200 cylinders
Units = cylinders of 2048 *512 = 1048576 bytes


  Device Boot   Start     End   Blocks Id System
/dev/sdd1        1     200   204784 83 Linux


Disk /dev/sde: 157 MB,157286400 bytes
64 heads, 32 sectors/track,150 cylinders
Units = cylinders of 2048 *512 = 1048576 bytes


  Device Boot   Start     End   Blocks Id System
/dev/sde1        1     150   153584 83 Linux


Disk /dev/sdf: 157 MB,157286400 bytes
64 heads, 32 sectors/track,150 cylinders
Units = cylinders of 2048 *512 = 1048576 bytes


  Device Boot   Start     End   Blocks Id System
/dev/sdf1        1     150   153584 83 Linux


Disk /dev/sdg: 10.7 GB,10737418240 bytes
255 heads, 63 sectors/track,1305 cylinders
Units = cylinders of 16065 *512 = 8225280 bytes


  Device Boot   Start     End   Blocks Id System
/dev/sdg1        1    1305  10482381  83 Linux


Disk /dev/sdh: 10.7 GB,10737418240 bytes
255 heads, 63 sectors/track,1305 cylinders
Units = cylinders of 16065 *512 = 8225280 bytes


  Device Boot   Start     End   Blocks Id System
/dev/sdh1        1    1305  10482381  83 Linux


Disk /dev/sdi: 10.7 GB,10737418240 bytes
255 heads, 63 sectors/track,1305 cylinders
Units = cylinders of 16065 *512 = 8225280 bytes


  Device Boot   Start     End   Blocks Id System
/dev/sdi1        1    1305  10482381  83 Linux
[root@jason1 ~]#


1)绑定裸设备
在/etc/sysconfig/rawdevices文件中添加以下内容，绑定裸设备：
/dev/raw/raw1 /dev/sdb1
/dev/raw/raw2 /dev/sdc1
/dev/raw/raw3 /dev/sdd1
/dev/raw/raw4 /dev/sde1
/dev/raw/raw5 /dev/sdf1
/dev/raw/raw6 /dev/sdg1
/dev/raw/raw7 /dev/sdh1
/dev/raw/raw8 /dev/sdi1
2)设置开机自启动裸设备服务
#chkconfig–level 345 rawdevices on
3)手工启动rawdevices服务
[root@jason1 ~]# service rawdevices start
Assigning devices:
     /dev/raw/raw1 -->  /dev/sdb1
/dev/raw/raw1: bound to major 8, minor 17
     /dev/raw/raw2 -->  /dev/sdc1
/dev/raw/raw2: bound to major 8, minor 33
     /dev/raw/raw3 -->  /dev/sdd1
/dev/raw/raw3: bound to major 8, minor 49
     /dev/raw/raw4 -->  /dev/sde1
/dev/raw/raw4: bound to major 8, minor 65
     /dev/raw/raw5 -->  /dev/sdf1
/dev/raw/raw5: bound to major 8, minor 81
     /dev/raw/raw6 -->  /dev/sdg1
/dev/raw/raw6: bound to major 8, minor 97
     /dev/raw/raw7 -->  /dev/sdh1
/dev/raw/raw7: bound to major 8, minor 113
     /dev/raw/raw8 -->  /dev/sdi1
/dev/raw/raw8: bound to major 8, minor 129
done
[root@jason1 ~]#
4)查看裸设备绑定情况
[root@jason1 ~]# raw -qa
/dev/raw/raw1: bound to major 8, minor 17
/dev/raw/raw2: bound to major 8, minor 33
/dev/raw/raw3: bound to major 8, minor 49
/dev/raw/raw4: bound to major 8, minor 65
/dev/raw/raw5: bound to major 8, minor 81
/dev/raw/raw6: bound to major 8, minor 97
/dev/raw/raw7: bound to major 8, minor 113
/dev/raw/raw8: bound to major 8, minor 129
[root@jason1 ~]#
5)裸设备权限配置
在/etc/rc.local中增加以下内容，以便重启后裸设备拥有合适权限
chown root:oinstall /dev/raw/raw1
chown root:oinstall /dev/raw/raw2
chown root:oinstall /dev/raw/raw3
chown root:oinstall /dev/raw/raw4
chown root:oinstall /dev/raw/raw5
chown oracle:oinstall /dev/raw/raw6
chown oracle:oinstall /dev/raw/raw7
chown oracle:oinstall /dev/raw/raw8


chmod 660 /dev/raw/raw1
chmod 660 /dev/raw/raw2
chmod 660 /dev/raw/raw3
chmod 660 /dev/raw/raw4
chmod 660 /dev/raw/raw5
chmod 644 /dev/raw/raw6
chmod 644 /dev/raw/raw7
chmod 644 /dev/raw/raw8
注：在jason2
12.ASM磁盘配置
ASMLIB下载地址
http://www.oracle.com/technetwork/server-storage/linux/downloads/rhel5-084877.html#oracleasm_rhel5_amd64
根据当前系统内核版本下载对应的ASM包,在所有节点进行安装。
1)、ASMlib安装
在所有节点进行安装ASMlib。
[root@jason1 mnt]# rpm -ivh oracleasm-support-2.1.8-1.el5.x86_64.rpm
warning: oracleasm-support-2.1.8-1.el5.x86_64.rpm:Header V3 DSA signature: NOKEY, key ID 1e5e0159
Preparing...        ###########################################[100%]
1:oracleasm-support   ########################################### [100%]
[root@jason1 mnt]# rpm -ivh oracleasm-2.6.18-238.el5-2.0.5-1.el5.x86_64.rpm
warning: oracleasm-2.6.18-238.el5-2.0.5-1.el5.x86_64.rpm:Header V3 DSA signature: NOKEY, key ID 1e5e0159
Preparing...        ########################################### [100%]
  1:oracleasm-2.6.18-238.el###########################################[100%]
[root@jason1 mnt]# rpm -ivh oracleasmlib-2.0.4-1.el5.x86_64.rpm
warning: oracleasmlib-2.0.4-1.el5.x86_64.rpm:Header V3 DSA signature: NOKEY, key ID 1e5e0159
Preparing...        ########################################### [100%]
1:oracleasmlib     ########################################### [100%]
[root@jason1 mnt]#
2)、ASM磁盘驱动配置
在所有节点执行如下配置。
[root@jason1 init.d]# /etc/init.d/oracleasm configure
Configuring the Oracle ASM library driver.


This will configure the on-boot properties of the OracleASM library
driver. Thefollowing questions will determine whether the driver is
loaded on boot and what permissions it will have. The current values
will be shown in brackets ('[]'). Hitting <ENTER> without typing an
answer will keep that current value. Ctrl-C will abort.


Default user to own the driver interface []: oracle
Default group to own the driver interface []: dba
Start Oracle ASM library driver on boot (y/n) [y]: y
Scan for Oracle ASM disks on boot (y/n) [y]: y
Writing Oracle ASM library driver configuration: done
Initializing the Oracle ASMLib driver:           [ OK ]
Scanning the system for Oracle ASMLib disks:        [ OK ]
[root@jason1 init.d]# /etc/init.d/oracleasm enable
Writing Oracle ASM library driver configuration: done
Initializing the Oracle ASMLib driver:           [ OK ]
Scanning the system for Oracle ASMLib disks:        [ OK ]
[root@jason1 init.d]#
3)、创建ASM磁盘
仅在其中一个节点上创建ASM磁盘
[root@jason1 init.d]# /etc/init.d/oracleasm creatediskVOL1 /dev/sdg1
Marking disk "VOL1" as an ASM disk:            [ OK ]
[root@jason1 init.d]# /etc/init.d/oracleasm creatediskVOL2 /dev/sdh1
Marking disk "VOL2" as an ASM disk:            [ OK ]
[root@jason1 init.d]# /etc/init.d/oracleasm creatediskVOL3 /dev/sdi1
Marking disk "VOL3" as an ASM disk:            [ OK ]
[root@jason1 init.d]

注意：用于asm的设备必须已经分区的设备（裸分区），不能直接是裸设备。
4)、扫描ASM磁盘
创建ASM磁盘只需要在一个节点执行，其余节点需要执行asm磁盘扫描才能找到磁盘。。
[root@jason2 init.d]# /etc/init.d/oracleasm scandisks
Scanning the system for Oracle ASMLib disks:        [ OK ]
[root@jason2 init.d]# /etc/init.d/oracleasm listdisks
VOL1
VOL2
VOL3
[root@jason2 init.d]#