11 配置共享磁盘
给双节点配置6块共享磁盘。
Note：
其中第1块大小为130M，用于存放Oracle 10g R2 RAC的OCR。
其中第2块大小为130M，用于存放Oracle 10g R2 RAC的Voting Disk。
其中第3块大小为3G，用于配置DATA磁盘组，存放数据库。
其中第4块大小为3G，用于配置FRA磁盘组，作为数据库的闪回恢复区，存放归档日志、RMAN备份等。
其中第5块大小为1G，用于将来升级到11gR2 RAC时，给Grid Infrastructure使用。
具体的配置过程，在本文档中就不再一一赘述，但是视频里会给出操作过程。具体可以通过
下述2中方式查看：
1 查看本人之前的系列视频之《黄伟老师Oracle RAC+DG系列视频+售后安心技术支持服务》第4讲RAC双节点格式化共享存储，配置ASM磁盘，配置ASM。
2 或查看本人系列文章之一步一步在Linux上安装Oracle 11gR2 RAC (3)第2.13 配置共享磁盘。
12 在共享磁盘上创建分区
这里在每个磁盘上分别创建1个分区，且把磁盘所有空间都用于这1个主分区。
Note:
对共享磁盘分区时，只需要以root用户在任意节点上创建即可，不需要在每个节点上执行！
分区后的信息如下：
Node1:
[root@node1 ~]# fdisk -l
Disk /dev/sda: 42.9 GB, 42949672960 bytes
255 heads, 63 sectors/track, 5221 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Device Boot Start End Blocks Id System
/dev/sda1 * 1 4711 37841076 83 Linux
/dev/sda2 4712 5221 4096575 82 Linux swap / Solaris
Disk /dev/sdb: 134 MB, 134217728 bytes
64 heads, 32 sectors/track, 128 cylinders
Units = cylinders of 2048 * 512 = 1048576 bytes
Device Boot Start End Blocks Id System
/dev/sdb1 1 128 131056 83 Linux
Disk /dev/sdc: 134 MB, 134217728 bytes
64 heads, 32 sectors/track, 128 cylinders
Units = cylinders of 2048 * 512 = 1048576 bytes
Device Boot Start End Blocks Id System
/dev/sdc1 1 128 131056 83 Linux
Disk /dev/sdd: 3221 MB, 3221225472 bytes
255 heads, 63 sectors/track, 391 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Device Boot Start End Blocks Id System
/dev/sdd1 1 391 3140676 83 Linux
Disk /dev/sde: 3221 MB, 3221225472 bytes
255 heads, 63 sectors/track, 391 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Device Boot Start End Blocks Id System
/dev/sde1 1 391 3140676 83 Linux
Disk /dev/sdf: 1073 MB, 1073741824 bytes
255 heads, 63 sectors/track, 130 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Device Boot Start End Blocks Id System
/dev/sdf1 1 130 1044193+ 83 Linux
[root@node1 ~]#
Node 2：
[root@node2 ~]# fdisk -l
Disk /dev/sda: 42.9 GB, 42949672960 bytes
255 heads, 63 sectors/track, 5221 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Device Boot Start End Blocks Id System
/dev/sda1 * 1 4463 35849016 83 Linux
/dev/sda2 4464 5221 6088635 82 Linux swap / Solaris
Disk /dev/sdb: 134 MB, 134217728 bytes
64 heads, 32 sectors/track, 128 cylinders
Units = cylinders of 2048 * 512 = 1048576 bytes
Device Boot Start End Blocks Id System
/dev/sdb1 1 128 131056 83 Linux
Disk /dev/sdc: 134 MB, 134217728 bytes
64 heads, 32 sectors/track, 128 cylinders
Units = cylinders of 2048 * 512 = 1048576 bytes
Device Boot Start End Blocks Id System
/dev/sdc1 1 128 131056 83 Linux
Disk /dev/sdd: 3221 MB, 3221225472 bytes
255 heads, 63 sectors/track, 391 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Device Boot Start End Blocks Id System
/dev/sdd1 1 391 3140676 83 Linux
Disk /dev/sde: 3221 MB, 3221225472 bytes
255 heads, 63 sectors/track, 391 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Device Boot Start End Blocks Id System
/dev/sde1 1 391 3140676 83 Linux
Disk /dev/sdf: 1073 MB, 1073741824 bytes
255 heads, 63 sectors/track, 130 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Device Boot Start End Blocks Id System
/dev/sdf1 1 130 1044193+ 83 Linux
[root@node2 ~]#
节点2上的磁盘分区信息除/dev/sda可能不同之外，其它共享磁盘分区信息应该完全同节点1。
13 配置裸设备以存放OCR、Voting Disk
由于在Oracle 10g版本里，OCR和Voting Disk还不支持存放在ASM磁盘里，所以，这里
需要将/dev/sdb1、/dev/sdc1这两个硬盘分区作为裸设备来分别存放OCR和Voting Disk。
我们将使用udev来管理/dev/sdb1、/dev/sdc1这2个设备：
编辑/etc/udev/rules.d/60-raw.rules 文件：
在其中加入下述内容：
ACTION==”add”, KERNEL==”/dev/sdb1″, RUN+=”/bin/raw /dev/raw/raw1 %N”
ACTION==”add”, ENV{MAJOR}==”8″, ENV{MINOR}==”17″, RUN+=”/bin/raw /dev/raw/raw1 %M %m”
ACTION==”add”, KERNEL==”/dev/sdc1″, RUN+=”/bin/raw /dev/raw/raw2 %N”
ACTION==”add”, ENV{MAJOR}==”8″, ENV{MINOR}==”33″, RUN+=”/bin/raw /dev/raw/raw2 %M %m”
KERNEL ==”raw[1-2]”, OWNER =”oracle”, GROUP =”oinstall”, MODE =”640″
编辑后：
[root@node1 rules.d]# pwd
/etc/udev/rules.d
[root@node1 rules.d]# ll
total 152
-rw-r–r– 1 root root 515 Jul 3 2009 05-udev-early.rules
-rw-r–r– 1 root root 900 Mar 31 2010 40-multipath.rules
-rw-r–r– 1 root root 15647 Jul 3 2009 50-udev.rules
-rw-r–r– 1 root root 471 Jul 3 2009 51-hotplug.rules
-rw-r–r– 1 root root 143 Jul 4 2009 60-net.rules
-rw-r–r– 1 root root 1088 Jun 6 2007 60-pcmcia.rules
-rw-r–r– 1 root root 628 Dec 19 21:23 60-raw.rules
-rw-r–r– 1 root root 9014 Apr 1 2010 60-wacom.rules
-rw-r–r– 1 root root 129 Apr 1 2010 61-uinput-stddev.rules
-rw-r–r– 1 root root 214 Apr 1 2010 61-uinput-wacom.rules
-rw-r–r– 1 root root 1823 Nov 5 2008 85-pcscd_ccid.rules
-rw-r–r– 1 root root 114 Nov 5 2008 90-alsa.rules
-rw-r–r– 1 root root 61 Jul 3 2009 90-dm.rules
-rw-r–r– 1 root root 82 Apr 1 2010 90-hal.rules
-rw-r–r– 1 root root 107 Jul 3 2009 95-pam-console.rules
-rw-r–r– 1 root root 292 Apr 1 2010 98-kexec.rules
-rw-r–r– 1 root root 2319 Jul 15 2008 bluetooth.rules
[root@node1 rules.d]# cat 60-raw.rules
# Enter raw device bindings here.
#
# An example would be:
# ACTION==”add”, KERNEL==”sda”, RUN+=”/bin/raw /dev/raw/raw1 %N”
# to bind /dev/raw/raw1 to /dev/sda, or
# ACTION==”add”, ENV{MAJOR}==”8″, ENV{MINOR}==”1″, RUN+=”/bin/raw /dev/raw/raw2 %M %m”
# to bind /dev/raw/raw2 to the device with major 8, minor 1.
ACTION==”add”, KERNEL==”/dev/sdb1″, RUN+=”/bin/raw /dev/raw/raw1 %N”
ACTION==”add”, ENV{MAJOR}==”8″, ENV{MINOR}==”17″, RUN+=”/bin/raw /dev/raw/raw1 %M %m”
ACTION==”add”, KERNEL==”/dev/sdc1″, RUN+=”/bin/raw /dev/raw/raw2 %N”
ACTION==”add”, ENV{MAJOR}==”8″, ENV{MINOR}==”33″, RUN+=”/bin/raw /dev/raw/raw2 %M %m”
KERNEL ==”raw[1-2]”, OWNER =”oracle”, GROUP =”oinstall”, MODE =”640″
[root@node1 rules.d]#
关于使用udev来管理Linux下的设备文件，具体可以查看链接：
http://www.ibm.com/developerworks/cn/linux/l-cn-udev/
该部分内容，不是本课程内容，不做过多阐述。
其中的ENV{MAJOR}和ENV{MINOR}可以通过下述命令确定：
[root@node1 rules.d]# ll /dev/sd*
brw-r—– 1 root disk 8, 0 Dec 19 16:33 /dev/sda
brw-r—– 1 root disk 8, 1 Dec 19 16:34 /dev/sda1
brw-r—– 1 root disk 8, 2 Dec 19 16:33 /dev/sda2
brw-r—– 1 root disk 8, 16 Dec 19 16:33 /dev/sdb
brw-r—– 1 root disk 8, 17 Dec 19 21:20 /dev/sdb1
brw-r—– 1 root disk 8, 32 Dec 19 16:33 /dev/sdc
brw-r—– 1 root disk 8, 33 Dec 19 21:20 /dev/sdc1
brw-r—– 1 root disk 8, 48 Dec 19 16:33 /dev/sdd
brw-r—– 1 root disk 8, 49 Dec 19 21:20 /dev/sdd1
brw-r—– 1 root disk 8, 64 Dec 19 16:33 /dev/sde
brw-r—– 1 root disk 8, 65 Dec 19 21:20 /dev/sde1
brw-r—– 1 root disk 8, 80 Dec 19 16:33 /dev/sdf
brw-r—– 1 root disk 8, 81 Dec 19 21:20 /dev/sdf1
[root@node1 rules.d]#
然后，启动udev:
[root@node1 rules.d]# /sbin/udevcontrol reload_rules
[root@node1 rules.d]# /sbin/start_udev
Starting udev: [ OK ]
[root@node1 rules.d]#
之后，可以看到/dev/raw下的裸设备文件：
[root@node1 rules.d]# ll /dev/raw/*
crw-r—– 1 oracle oinstall 162, 1 Dec 19 21:31 /dev/raw/raw1
crw-r—– 1 oracle oinstall 162, 2 Dec 19 21:31 /dev/raw/raw2
[root@node1 rules.d]#
Note：
上述操作需要在双节点执行！且确保在双节点均可以看到裸设备文件，以及oracle用户具有对裸设备的权限！
14 安装asm软件包、创建ASM磁盘
选择与OS内核对应的软件包：
[root@node1 asm]# uname -rm
2.6.18-194.el5 x86_64
[root@node1 asm]# ll
total 244
-rw-r–r– 1 root root 137486 Dec 19 13:50 oracleasm-2.6.18-194.el5-2.0.5-1.el5.x86_64.rpm
-rw-r–r– 1 root root 13929 Dec 19 13:50 oracleasmlib-2.0.4-1.el5.i386.rpm
-rw-r–r– 1 root root 85687 Dec 19 13:50 oracleasm-support-2.1.7-1.el5.i386.rpm
[root@node1 asm]#
安装：
[root@node1 asm]# rpm -ivh oracleasm-support-2.1.7-1.el5.i386.rpm
warning: oracleasm-support-2.1.7-1.el5.i386.rpm: Header V3 DSA signature: NOKEY, key ID 1e5e0159
Preparing… ########################################### [100%]
1:oracleasm-support ########################################### [100%]
[root@node1 asm]# rpm -ivh oracleasm-2.6.18-194.el5-2.0.5-1.el5.x86_64.rpm
warning: oracleasm-2.6.18-194.el5-2.0.5-1.el5.x86_64.rpm: Header V3 DSA signature: NOKEY, key ID 1e5e0159
Preparing… ########################################### [100%]
1:oracleasm-2.6.18-194.el########################################### [100%]
[root@node1 asm]# rpm -ivh oracleasmlib-2.0.4-1.el5.i386.rpm
warning: oracleasmlib-2.0.4-1.el5.i386.rpm: Header V3 DSA signature: NOKEY, key ID 1e5e0159
Preparing… ########################################### [100%]
1:oracleasmlib ########################################### [100%]
[root@node1 asm]#
Note：
1 注意安装顺序，先安装oracleasm-support-2.1.7-1.el5.i386.rpm ，然后oracleasm-2.6.18-194.el5-2.0.5-1.el5.x86_64.rpm ，最后oracleasmlib-2.0.4-1.el5.i386.rpm。
2 ASM软件包一定要选择与OS内核对应的软件包。
配置ASM驱动：
[root@node1 asm]# /etc/init.d/oracleasm configure
Configuring the Oracle ASM library driver.
This will configure the on-boot properties of the Oracle ASM library
driver. The following questions will determine whether the driver is
loaded on boot and what permissions it will have. The current values
will be shown in brackets (‘[]’). Hitting <ENTER> without typing an
answer will keep that current value. Ctrl-C will abort.
Default user to own the driver interface []: oracle
Default group to own the driver interface []: oinstall
Start Oracle ASM library driver on boot (y/n) [n]: y
Scan for Oracle ASM disks on boot (y/n) [y]: y
Writing Oracle ASM library driver configuration: done
Initializing the Oracle ASMLib driver: [ OK ]
Scanning the system for Oracle ASMLib disks: [ OK ]
[root@node1 asm]# /etc/init.d/oracleasm status
Checking if ASM is loaded: yes
Checking if /dev/oracleasm is mounted: yes
[root@node1 asm]#
Note：
1 上述操作需要在双节点上完成！
2 在配置oracleasm时，要选择正确的用户、用户组，这里是oracle及oinstall组。
创建ASM磁盘：
这里选择将/dev/sdd1和/dev/sdf1分别创建为ASMDISK1、ASMDISK2。
[root@node1 asm]# /etc/init.d/oracleasm createdisk ASMDISK1 /dev/sdd1
Marking disk “ASMDISK1” as an ASM disk: [ OK ]
[root@node1 asm]# /etc/init.d/oracleasm createdisk ASMDISK2 /dev/sde1
Marking disk “ASMDISK2” as an ASM disk: [ OK ]
[root@node1 asm]# /etc/init.d/oracleasm listdisks
ASMDISK1
ASMDISK2
[root@node1 asm]#
创建ASM磁盘操作，只需在任意节点上执行即可。然后，再另一节点上执行scandisks:
[root@node2 asm]# /etc/init.d/oracleasm scandisks
Scanning the system for Oracle ASMLib disks: [ OK ]
[root@node2 asm]# /etc/init.d/oracleasm listdisks
ASMDISK1
ASMDISK2
[root@node2 asm]#
升级后，双节点上的CRS都从10.2.0.1.0升级到10.2.0.5.0。
