oracle11.2.0.4 rac搭建中的crs-4000错误解析

fdwe2e

在系统环境为rhel6.5的服务器上，搭建数据库版本为oracle11.2.0.4的两节点的RAC，安装GRID时遭遇如下错误：（注：vote、data和闪回盘都在存储设备上）
ASM created and started successfully.

Disk Group VOTE mounted successfully.

clscfg: -install mode specified
clscfg: EXISTING configuration version 5 detected.
clscfg: version 5 is 11g Release 2.
Successfully accumulated necessary OCR keys.
clscfg: Arguments check out successfully.

NO KEYS WERE WRITTEN. Supply -force parameter to override.
-force is destructive and will destroy any previous cluster
configuration.
Failed to create voting files on disk group VOTE.
Change to configuration failed, but was successfully rolled back.
CRS-4000: Command Replace failed, or completed with errors.
Voting file add failed
Failed to add voting disks at /g01/11ggrid/app/11.2.0/grid/crs/install/crsconfig_lib.pm line 6930.
/g01/11ggrid/app/11.2.0/grid/perl/bin/perl -I/g01/11ggrid/app/11.2.0/grid/perl/lib -I/g01/11ggrid/app/11.2.0/grid/crs/install /g01/11ggrid/app/11.2.0/grid/crs/install/rootcrs.pl execution failed

这个错误是说无法对磁盘组VOTE创建投票文件。改变配置失败，但成功回滚。
投票文件添加失败，未能在/g01/11ggrid/app/11.2.0/grid/crs/install添加表决磁盘。

碰到这个错误还是头一遭，很是诡异。因为之前使用ASMLIB挂盘都没有出现任何问题，但安装却出现这个问题，没法继续进行下去。

检查ASM的 alert.log日志发现：
ERROR: Could not create voting files. It spans across 161 AUs (max supported is 64 AUs)
ERROR: Voting file allocation failed for group VOTE
Errors in file /g01/11ggrid/app/11.2.0/grid/diag/asm/+asm/+ASM1/trace/+ASM1_ora_7321.trc:
ORA-15303: Voting files could not be created in diskgroup CRSVOTE due to small Allocation Unit size

继续检查一下如下块的大小问题，发现一些信息：
# cd /sys/block/dm-11/queue
#  more *block_size
::::::::::::::
logical_block_size
::::::::::::::
512
::::::::::::::
physical_block_size
::::::::::::::
4096

究其原因是它表现出不同的逻辑和物理块大小。对于大多数的情况下，物理和逻辑块的大小是相同的“512”。

那么如何解决这个问题呢？查询ORACLE的官方网站，给出的解决方案是：
1. Oracle bug 11780656 is already fixed in 11.2.0.3 with compatible.asm = 11.2.0.3 but OUI does not allow to specify "compatible.asm" attribute to create OCRVOTE diskgroup.
   - Bug 11780656  - ASM MANAGED VOTING FILES CANNOT BE MORE THAN 64X AU SIZE

2. Oracle bug 13999609 indicates that ASMLib driver only works with the expectation that logical block size and physical block size are 512/512 bytes.   
   - Bug 13999609  - PHYSICAL BLOCK SIZE REPORTED CAN CAUSE ISSUES WITH 10G DATABASES


SOLUTION

1] Possible workaround is to use '/dev/oracleasm/disks/*' path instead of "ORCL:*" when creating OCRVOTE diskgroup in OUI.
OR
2] Install the new “oracleasm-support-2.1.8-1” ASMLIB RPM package (which contains the permanent fix) as described in note 1500460.1

根据以上提示，我们可以在安装oracleasm-support-2.1.8-1的包，或者也可以在选择ASM盘时，改用
/dev/oracleasm/disks/*的路径来解决此问题。

还可以进入/etc/sysconfig目录，修改其下的oracleasm文件，将其中的ORACLEASM_USE_LOGICAL_BLOCK_SIZE=false改为true
注：Oracle提供了配置参数来启用ASMLib继续使用正确的逻辑块的大小的功能来避免这个问题。


通过如上的处理，该问题得以解决，GRID最后安装成功。