2012年 – 第2页 – Oracle and Linux

8月工作小记二三事

9月 3,2012 留下评论 Experience, Oracle, RAC

上个月，应该算是一个较为忙碌的工作月。其中：

1 公事的一DBA同事离职，所有的数据库这块全部压到我1个人的身上，要看的库多了，任务重了，有些库也只能做到粗犷式管理了；

2 出差2次，月初1次到上海，月末1次同样到上海；2次差旅虽说都是现场实施Oracle数据库环境的搭建，但都不是很顺利，究其原因：①项目前期同客户需求、准备工作都很有问题。如去之前，问项目组人员是实施RAC的数据库还是配置单实例的库，告知于我是单实例的库，结果一到客户现场方知是实施RAC的数据库；②个人的技术经验匮乏的短板也暴露的很明显了，尤其是存储方面的知识；

3 快速记录这2次出差上海解决问题的方法：

① 月初去上海某客户遇到的问题，客户环境采用Dell R910单台服务器，存储采用Dell MD 3200。在阵列的管理界面将存储映射到主机时，发现找不到主机，原因：犯的是低级错误，Dell MD 3200 支持级联扩展存储，误将主机的SAS卡插到阵列上的SAS OUT扩展口上，而应该将其连接到阵列上的SAS IN口。其次，在做好阵列之后，重启主机、阵列之后，发现主机依然无法找到对应的LUN，解决方法：注释掉/etc/udev/rules.d/90-dm.rules配置文件。

②第2次去上海另1客户实施RAC时，同样碰到存储的问题。只是这次的主机是2台IBM X3650M3，存储是IBM System Storage DS3500磁盘阵列柜。在阵列上做好LUN并映射到主机后，还是不能正常发现存储设备，解决办法是安装IBM提供的RDAC补丁包。下述，记录安装RDAC软件包的过程，该软件包需要到IBM官方网站下载对应的操作系统版本和阵列型号的补丁。

–

[root@node1 ~]# cd linuxrdac-09.03.0C05.0638/
[root@node1 linuxrdac-09.03.0C05.0638]# ll
total 1780
-r-xr-xr-x 1 root root     73 Apr 18 04:31 bootMpp26p.sh
drwxr-xr-x 2 root root   4096 Apr 18 04:31 dracutsetup
-r-xr-xr-x 1 root root   3808 Apr 18 04:31 genfileattributes
-r--r--r-- 1 root root   2603 Apr 18 04:31 genuniqueid.c
-r--r--r-- 1 root root   2603 Apr 18 04:31 hbaCheck
-r--r--r-- 1 root root  15136 Apr 18 04:31 License.txt
-r-xr-xr-x 1 root root   6592 Apr 18 04:31 lsvdev
-r--r--r-- 1 root root  22991 Apr 18 04:31 Makefile
-r--r--r-- 1 root root 150075 Apr 18 04:31 mppCmn_s2tos3.c
-r--r--r-- 1 root root  21321 Apr 18 04:31 mppCmn_SysInterface.c
-r--r--r-- 1 root root    814 Apr 18 04:31 mpp.conf
-r--r--r-- 1 root root 520094 Apr 18 04:31 MPP_hba.c
-r-xr-xr-x 1 root root   5155 Apr 18 04:31 mppiscsi_umountall
drwxr-xr-x 2 root root   4096 Apr 18 04:31 mpp_linux_headers
drwxr-xr-x 2 root root   4096 Apr 18 04:31 mpp_linux_sys_headers
-r--r--r-- 1 root root  57111 Apr 18 04:31 mppLnx26p_sysdep.c
-r--r--r-- 1 root root 175028 Apr 18 04:31 mppLnx26p_upper.c
-r--r--r-- 1 root root 206555 Apr 18 04:31 mppLnx26p_vhba.c
-r--r--r-- 1 root root 185100 Apr 18 04:31 mppLnx26p_vhbaio.c
-r--r--r-- 1 root root  95675 Apr 18 04:31 mppLnx26p_vhbalib.c
-r--r--r-- 1 root root  21610 Apr 18 04:31 mppLnx26p_vhbamisc.c
-r--r--r-- 1 root root  33709 Apr 18 04:31 mppLnx26p_vhbaproc.c
-r--r--r-- 1 root root 120597 Apr 18 04:31 mppLnx26p_vhbatask.c
-r--r--r-- 1 root root   1805 Apr 18 04:31 mpp_rcscript.REDHAT
-r--r--r-- 1 root root   1903 Apr 18 04:31 mpp_rcscript.SUSE
-r-xr-xr-x 1 root root  28522 Apr 18 04:31 mppSupport
-r--r--r-- 1 root root  24245 Apr 18 04:31 Readme.txt
-rw-r--r-- 1 root root   5559 Apr 18 04:31 setupDriver.REDHAT
-r--r--r-- 1 root root   8984 Apr 18 04:31 setupDriver.SUSE
-r--r--r-- 1 root root    261 Apr 18 04:31 setupMpp26p.sh
drwxr-xr-x 2 root root   4096 Apr 18 04:31 utility
[root@node1 linuxrdac-09.03.0C05.0638]# make
make[1]: Entering directory `/usr/src/kernels/2.6.18-194.el5-x86_64'
  CC [M]  /root/linuxrdac-09.03.0C05.0638/MPP_hba.o
  CC [M]  /root/linuxrdac-09.03.0C05.0638/mppLnx26p_upper.o
  CC [M]  /root/linuxrdac-09.03.0C05.0638/mppLnx26p_sysdep.o
  CC [M]  /root/linuxrdac-09.03.0C05.0638/mppCmn_s2tos3.o
  CC [M]  /root/linuxrdac-09.03.0C05.0638/mppCmn_SysInterface.o
  CC [M]  /root/linuxrdac-09.03.0C05.0638/mppLnx26p_vhbamisc.o
  CC [M]  /root/linuxrdac-09.03.0C05.0638/mppLnx26p_vhbatask.o
  CC [M]  /root/linuxrdac-09.03.0C05.0638/mppLnx26p_vhba.o
  CC [M]  /root/linuxrdac-09.03.0C05.0638/mppLnx26p_vhbaproc.o
  CC [M]  /root/linuxrdac-09.03.0C05.0638/mppLnx26p_vhbalib.o
  CC [M]  /root/linuxrdac-09.03.0C05.0638/mppLnx26p_vhbaio.o
/root/linuxrdac-09.03.0C05.0638/mppLnx26p_vhbaio.c: In function a€?__mppLnx_scsi_donea€?:
/root/linuxrdac-09.03.0C05.0638/mppLnx26p_vhbaio.c:3691: warning: label a€?donea€? defined but not used
  LD [M]  /root/linuxrdac-09.03.0C05.0638/mppUpper.o
  LD [M]  /root/linuxrdac-09.03.0C05.0638/mppVhba.o
  Building modules, stage 2.
  MODPOST
  CC      /root/linuxrdac-09.03.0C05.0638/mppUpper.mod.o
  LD [M]  /root/linuxrdac-09.03.0C05.0638/mppUpper.ko
  CC      /root/linuxrdac-09.03.0C05.0638/mppVhba.mod.o
  LD [M]  /root/linuxrdac-09.03.0C05.0638/mppVhba.ko
make[1]: Leaving directory `/usr/src/kernels/2.6.18-194.el5-x86_64'
make[1]: Entering directory `/usr/src/kernels/2.6.18-194.el5-x86_64'
  Building modules, stage 2.
  MODPOST
make[1]: Leaving directory `/usr/src/kernels/2.6.18-194.el5-x86_64'
gcc  -I/root/linuxrdac-09.03.0C05.0638 -I/root/linuxrdac-09.03.0C05.0638/mpp_linux_headers/ -I/root/linuxrdac-09.03.0C05.0638/mpp_linux_sys_headers/ -c ./utility/mppUtil.c  -o mppUtil.o
/bin/bash ./genfileattributes bld
gcc  -I/root/linuxrdac-09.03.0C05.0638 -I/root/linuxrdac-09.03.0C05.0638/mpp_linux_headers/ -I/root/linuxrdac-09.03.0C05.0638/mpp_linux_sys_headers/ -c ./utility/mppUtil26p_sysdep.c -o mppUtilSysdep.o
gcc mppUtil.o mppUtilSysdep.o -o mppUtil
gcc -o genuniqueid genuniqueid.c
[root@node1 linuxrdac-09.03.0C05.0638]# make install
make[1]: Entering directory `/root/linuxrdac-09.03.0C05.0638'
make[2]: Entering directory `/usr/src/kernels/2.6.18-194.el5-x86_64'
  Building modules, stage 2.
  MODPOST
make[2]: Leaving directory `/usr/src/kernels/2.6.18-194.el5-x86_64'
make[2]: Entering directory `/usr/src/kernels/2.6.18-194.el5-x86_64'
  Building modules, stage 2.
  MODPOST
make[2]: Leaving directory `/usr/src/kernels/2.6.18-194.el5-x86_64'
/bin/bash ./genfileattributes bld
gcc  -I/root/linuxrdac-09.03.0C05.0638 -I/root/linuxrdac-09.03.0C05.0638/mpp_linux_headers/ -I/root/linuxrdac-09.03.0C05.0638/mpp_linux_sys_headers/ -c ./utility/mppUtil26p_sysdep.c -o mppUtilSysdep.o
gcc mppUtil.o mppUtilSysdep.o -o mppUtil
Checking Host Adapter Configuration...
Detected 1 Emulex Host Adapter Port(s) on the system
Please wait while we modify the system configuration files.
Your kernel version is 2.6.18-194.el5
Preparing to install MPP driver against this kernel version...
Generating module dependencies...
Creating new MPP initrd image...
        You must now edit your boot loader configuration file, /boot/grub/menu.lst, to 
        add a new boot menu, which uses mpp-2.6.18-194.el5.img as the initrd image.
        Now Reboot the system for MPP to take effect.
        The new boot menu entry should look something like this (note that it may 
        vary with different system configuration):

        ...

                title Red Hat Linux (2.6.18-194.el5) with MPP support
                root (hd0,5)
                kernel /vmlinuz-2.6.18-194.el5 ro root=LABEL=RH9
                initrd /mpp-2.6.18-194.el5.img
        ...
MPP driver package has been successfully installed on your system.
make[1]: Leaving directory `/root/linuxrdac-09.03.0C05.0638'
[root@node1 linuxrdac-09.03.0C05.0638]#

4 问题得以解决，也深知自己存在的问题！送给自己一句话以自勉：革命尚未成功，同志仍需努力！

遇到ORA-600 [kmgs_parameter_update_timeout_1] [1565]的错误续

7月 27,2012 留下评论 Experience, Oracle, RAC

刚发完上篇文章之后，又从老杨的一篇文章“http://blog.itpub.net/post/468/450451?SelectActiveLayout=a”中看到一些有所帮助的信息，而这个案例中在alert里看到，在这个600的错误抛出之前一条错误信息中看到：

Wed Jul 25 09:56:46  2012
Thread 1 advanced to log sequence 580 (LGWR switch)
  Current log# 1 seq# 580 mem# 0: +DATA/zhfr8db/onlinelog/group_1.271.783424599
  Current log# 1 seq# 580 mem# 1: +FLASH/zhfr8db/onlinelog/group_1.256.783424601
Wed Jul 25 10:49:39  2012
Unexpected communication failure with ASM instance:
 error 21561 (ORA-21561: 生成 OID 失败
)

的错误信息。

从后续的alert日志里，尝试关闭数据库服务器的时候抛出的错误中也有类似错误：

Wed Jul 25 10:55:03  2012
Trace dumping is performing id=[cdmp_20120725105503]
Wed Jul 25 10:56:06  2012
Restarting dead background process MMON
MMON started with pid=33, OS id=6756
Wed Jul 25 12:46:20  2012
Unexpected communication failure with ASM instance:
 error 21561 (ORA-21561: 生成 OID 失败
)
NOTE: ASMB process state dumped to trace file c:\oracle\product\10.2.0\admin\zhfr8db\udump\zhfr8db1_ora_5552.trc
Wed Jul 25 12:47:03  2012
Unexpected communication failure with ASM instance:
 error 21561 (ORA-21561: 生成 OID 失败
)
NOTE: ASMB process state dumped to trace file c:\oracle\product\10.2.0\admin\zhfr8db\udump\zhfr8db1_ora_2008.trc
Wed Jul 25 12:50:01  2012
Unexpected communication failure with ASM instance:
 error 21561 (ORA-21561: 生成 OID 失败
)

再从c:\oracle\product\10.2.0\admin\zhfr8db\udump\zhfr8db1_ora_5552.trc文件跟踪看到：

*** 2012-07-25 12:46:20.268
*** CLIENT ID:() 2012-07-25 12:46:20.268
      ----------------------------------------
      SO: 000000047111DEF0, type: 2, owner: 0000000000000000, flag: INIT/-/-/0x00
      (process) Oracle pid=31, calls cur/top: 000000047E15DC20/000000047E15DC20, flag: (6) SYSTEM
                int error: 0, call error: 0, sess error: 0, txn error 0
  (post info) last post received: 0 0 33
              last post received-location: ksrpublish
              last process to post me: 7e11e6f8 1 6
              last post sent: 849 0 4
              last post sent-location: kslpsr
              last process posted by me: 7312aef8 1 6
        (latch info) wait_event=0 bits=0
        Process Group: DEFAULT, pseudo proc: 00000004731384B8
        O/S info: user: SYSTEM, term: DATACENTER01, ospid: 4828 
        OSD pid info: Windows thread id: 4828, image: ORACLE.EXE (ASMB)
        Short stack dump: 
ksdxfstk+42<-ksdxcb+1630<-ssthreadsrgruncallback+589<-OracleOradebugThreadStart+975<-0000000077D6B71A
<-0000000077EF047A<-0000000077DA79F3<-0000000008653328<-000000000865190C<-0000000005F564A9
<-0000000005F0CF64<-0000000005EE5D88<-0000000005EE57F9<-0000000005EA5ECB<-ttcdrv+14881
<-0000000005EAAA6D<-xupirtrc+1335<-xupirtr+216<-upirtr+23<-kpurcs+45
<-OCIKDispatch+32<-kfnOpExecute+146<-kfnbRun+1062<-ksbrdp+988<-opirip+700
<-opidrv+856<-sou2o+52<-opimai_real+268<-opimai+96<-BackgroundThreadStart+637<-0000000077D6B71A
        ----------------------------------------
        SO: 000000047114E1A0, type: 4, owner: 000000047111DEF0, flag: INIT/-/-/0x00
        (session) sid: 189 trans: 0000000000000000, creator: 000000047111DEF0, flag: (51) USR/- BSY/-/-/-/-/-
                  DID: 0001-001F-00000003, short-term DID: 0000-0000-00000000
                  txn branch: 0000000000000000
                  oct: 0, prv: 0, sql: 0000000000000000, psql: 0000000000000000, user: 0/SYS
        service name: SYS$BACKGROUND
        waiting for 'ASM background timer' wait_time=0, seconds since wait started=420165
                    =0, =0, =0
                    blocking sess=0x0000000000000000 seq=31
        Dumping Session Wait History
         for 'ASM background timer' count=1 wait_time=4.999949 sec
                    =0, =0, =0
         for 'ASM background timer' count=1 wait_time=4.999893 sec
                    =0, =0, =0
         for 'ASM background timer' count=1 wait_time=5.000022 sec
                    =0, =0, =0
         for 'ASM background timer' count=1 wait_time=4.999948 sec
                    =0, =0, =0
         for 'ASM background timer' count=1 wait_time=4.999924 sec
                    =0, =0, =0
         for 'ASM background timer' count=1 wait_time=5.000012 sec
                    =0, =0, =0
         for 'ASM background timer' count=1 wait_time=4.999948 sec
                    =0, =0, =0
         for 'ASM background timer' count=1 wait_time=4.999858 sec
                    =0, =0, =0
         for 'ASM background timer' count=1 wait_time=4.999991 sec
                    =0, =0, =0
         for 'ASM background timer' count=1 wait_time=5.000000 sec
                    =0, =0, =0
        Sampled Session History of session 189 serial 1
        ---------------------------------------------------
        The sampled session history is constructed by sampling
        the target session every 1 second. The sampling process
        captures at each sample if the session is in a non-idle wait,
        an idle wait, or not in a wait. If the session is in a
        non-idle wait then one interval is shown for all the samples
        the session was in the same non-idle wait. If the
        session is in an idle wait or not in a wait for
        consecutive samples then one interval is shown for all
        the consecutive samples. Though we display these consecutive
        samples  in a single interval the session may NOT be continuously
        idle or not in a wait (the sampling process does not know).
 
        The history is displayed in reverse chronological order.
 
        sample interval: 1 sec, max history 120 sec
        ---------------------------------------------------
          [121 samples,                                    12:44:20 - 12:46:20]
            idle wait at each sample
        temporary object counter: 0
          KTU Session Commit Cache Dump for IDLs: 
          KTU Session Commit Cache Dump for Non-IDLs: 
          ----------------------------------------
          UOL used : 0 locks(used=0, free=0)
          KGX Atomic Operation Log 000000047AECC840
           Mutex 0000000000000000(0, 0) idn 0 oper NONE
           Cursor Pin uid 189 efd 3 whr 11 slp 0
          KGX Atomic Operation Log 000000047AECC888
           Mutex 0000000000000000(0, 0) idn 0 oper NONE
           Library Cache uid 189 efd 0 whr 0 slp 0
          KGX Atomic Operation Log 000000047AECC8D0
           Mutex 0000000000000000(0, 0) idn 0 oper NONE
           Library Cache uid 189 efd 0 whr 0 slp 0
          ----------------------------------------
          SO: 000000045A233D80, type: 41, owner: 000000047114E1A0, flag: INIT/-/-/0x00
          (dummy) nxc=0, nlb=0   
        ----------------------------------------
        SO: 0000000472172A40, type: 11, owner: 000000047111DEF0, flag: INIT/-/-/0x00
        (broadcast handle) flag: (2) ACTIVE SUBSCRIBER, owner: 000000047111DEF0,
                           event: 31, last message event: 31,
                           last message waited event: 31,                            next message: 0000000476225BC8(0), messages read: 0
                           channel: (00000004711640E0) system events broadcast channel
                                    scope: 2, event: 30690, last mesage event: 7413,
                                    publishers/subscribers: 1/45,
                                    messages published: 3
                                    oldest msg (?): 0000000476225BB8 id: 1 pub: 000000047E11F768
                                    heuristic msg queue length: 3
        ----------------------------------------
        SO: 0000000472270FA0, type: 19, owner: 000000047111DEF0, flag: INIT/-/-/0x00
         GES MSG BUFFERS: st=emp chunk=0x0000000000000000 hdr=0x0000000000000000 lnk=0x0000000000000000 flags=0x0 inc=0
          outq=0 sndq=0 opid=0 prmb=0x0 
          mbg[i]=(0 0) mbg[b]=(0 0) mbg[r]=(0 0)
          fmq[i]=(0 0) fmq[b]=(0 0) fmq[r]=(0 0)
          mop[s]=0 mop[q]=0 pendq=0 zmbq=0
          nonksxp_recvs=0
        ------------process 0x0000000472270FA0--------------------
        proc version      : 0
        Local node        : 0
        pid               : 4828
        lkp_node          : 0
        svr_mode          : 0
        proc state        : KJP_FROZEN
        Last drm hb acked : 0
        Total accesses    : 3
        Imm.  accesses    : 0
        Locks on ASTQ     : 0
        Locks Pending AST : 0
        Granted locks     : 0
        AST_Q: 
        PENDING_Q: 
        GRANTED_Q: 
        ----------------------------------------
        SO: 000000047E15DC20, type: 3, owner: 000000047111DEF0, flag: INIT/-/-/0x00
        (call) sess: cur 7114e1a0, rec 0, usr 7114e1a0; depth: 0
          ----------------------------------------
          SO: 000000045ED23770, type: 84, owner: 000000047E15DC20, flag: INIT/-/-/0x00
          (kfgso) flags: 00000000 clt: 3 err: 0 hint: 0
          (kfgpn) rpi: 1 itrn:0000000000000000 gst:0000000000000000 usrp:0000000000000000
          busy: 0 rep: 0 grp: 5d60b840 check: 0/0 glink: 5d60b888 5d60b888
            kfgrp:  number: 0/0 type: 0 compat: 0.0.0.0.0 dbcompat:0.0.0.0.0
            timestamp: 0 state: 0 flags: 2 gpnlist: 5ed237f0 5ed237f0
            KFGPN at 5ed23770 in dependent chain
        ----------------------------------------
        SO: 000000045AEEDD48, type: 16, owner: 000000047111DEF0, flag: INIT/-/-/0x00
        (osp req holder)
PSO child state object changes :
Dump of memory from 0x0000000474167DC0 to 0x0000000474167FC8
474167DC0 00000005 00000000 5AEEDD48 00000004  [........H..Z....]
474167DD0 00000010 000313F4 7E15DC20 00000004  [........ ..~....]
474167DE0 00000003 000313F4 72270FA0 00000004  [..........'r....]
474167DF0 00000013 000312CB 72172A40 00000004  [........@*.r....]
474167E00 0000000B 000313F4 7114E1A0 00000004  [...........q....]
474167E10 00000004 000312CB 00000000 00000000  [................]
474167E20 00000000 00000000 00000000 00000000  [................]
        Repeat 25 times
474167FC0 00000000 00000000                    [........]        
*** 2012-07-25 12:46:37.393
*** CLIENT ID:() 2012-07-25 12:46:37.393
WARNING:Could not lower the asynch I/O limit to 256 for SQL direct I/O. It is set to -1 
WARNING:Could not lower the asynch I/O limit to 320 for SQL direct I/O. It is set to -1 
WARNING:Could not lower the asynch I/O limit to 256 for SQL direct I/O. It is set to -1 
WARNING:Could not lower the asynch I/O limit to 256 for SQL direct I/O. It is set to -1 
WARNING:Could not lower the asynch I/O limit to 288 for SQL direct I/O. It is set to -1 
WARNING:Could not lower the asynch I/O limit to 224 for SQL direct I/O. It is set to -1 
WARNING:Could not lower the asynch I/O limit to 256 for SQL direct I/O. It is set to -1 
WARNING:Could not lower the asynch I/O limit to 192 for SQL direct I/O. It is set to -1 
WARNING:Could not lower the asynch I/O limit to 160 for SQL direct I/O. It is set to -1 
*** 2012-07-25 12:47:20.314
*** CLIENT ID:() 2012-07-25 12:47:20.314
WARNING:Could not lower the asynch I/O limit to 256 for SQL direct I/O. It is set to -1

难道问题是：

WARNING:Could not lower the asynch I/O limit to 256 for SQL direct I/O. It is set to -1 
WARNING:Could not lower the asynch I/O limit to 320 for SQL direct I/O. It is set to -1 
WARNING:Could not lower the asynch I/O limit to 256 for SQL direct I/O. It is set to -1 
WARNING:Could not lower the asynch I/O limit to 256 for SQL direct I/O. It is set to -1 
WARNING:Could not lower the asynch I/O limit to 288 for SQL direct I/O. It is set to -1 
WARNING:Could not lower the asynch I/O limit to 224 for SQL direct I/O. It is set to -1 
WARNING:Could not lower the asynch I/O limit to 256 for SQL direct I/O. It is set to -1 
WARNING:Could not lower the asynch I/O limit to 192 for SQL direct I/O. It is set to -1 
WARNING:Could not lower the asynch I/O limit to 160 for SQL direct I/O. It is set to -1

导致的，这些WARNING又说明什么呢？

遇到ORA-600 [kmgs_parameter_update_timeout_1] [1565]的错误

7月 27,2012 留下评论 Experience, Oracle, RAC

今天上午，在一套Windows 2003 64位的双节点10.2.0.5.0的64位RAC数据库上，遇到一则600的错误，ORA-00600: 内部错误代码, 参数: [kmgs_parameter_update_timeout_1], [1565], [], [], [], [], [], []。从(节点1的)alert日志里获取的具体错误信息如下：

Wed Jul 25 10:49:39  2012
Unexpected communication failure with ASM instance:
 error 21561 (ORA-21561: 生成 OID 失败
)
NOTE: ASMB process state dumped to trace file c:\oracle\product\10.2.0\admin\zhfr8db\bdump\zhfr8db1_mmon_4624.trc
System State dumped to trace file c:\oracle\product\10.2.0\admin\zhfr8db\bdump\zhfr8db1_mmon_4624.trc
Wed Jul 25 10:55:02  2012
Errors in file c:\oracle\product\10.2.0\admin\zhfr8db\bdump\zhfr8db1_mmon_4624.trc:
ORA-00600: 内部错误代码, 参数: [kmgs_parameter_update_timeout_1], [1565], [], [], [], [], [], []
ORA-01565: 标识文件 '+DATA/zhfr8db/spfilezhfr8db.ora' 时出错
ORA-17503: ksfdopn: 2 未能打开文件 +DATA/zhfr8db/spfilezhfr8db.ora
ORA-21561: 生成 OID 失败

Wed Jul 25 10:55:03  2012
Trace dumping is performing id=[cdmp_20120725105503]
Wed Jul 25 10:56:06  2012
Restarting dead background process MMON
MMON started with pid=33, OS id=6756

当时的表现情况是，客户端无法通过应用系统访问数据库，客户端通过tnsping service_name的返回结果也是，有时通，返回10毫秒，而有时挂死了。

登录到其中的一个节点服务器(节点1)上，执行lsnrctl status，查看监听状态也挂死，而到另外一个节点(节点2)上，执行lsnrctl status一切正常。并且，分别在两个节点上，通过SQL*PLUS连接数据库均正常，执行crs_stat -t返回的结果也都正常。

接下来，分别在两个节点上作了一个AWR报告，分析了AWR发现并没有发现数据库的异常现象。

从告警日志里看到跟SPFILE相关的错误，于是在SQL*PLUS里尝试create pfile from spfile,以及使用RMAN工具backup spfile时，都报错了，当时匆忙，具体错误号没有记录下来。

没辙了，就到节点1上，尝试通过srvctl stop database -d db_unique_name关闭数据库时，挂死了，丝毫没有反应；通过SQL*PLUS在节点1上，尝试shutdown immediate来手工关闭实例，也未果。

当时，从网络上找到老杨以及惜分飞的文章，不过，貌似都和我遇到的这个情况不太类似。

然后，在没有更好的办法的前提下，直接重启节点1的Windows服务器，毕竟当时应用已经不可用了，再者是RAC架构，最起码还留着一个实例。重启之后，数据库一切正常了。

最后，从MetaLink上找到这样的一篇文章：ORA-600 [kmgs_parameter_update_timeout_1], [1565] While Accessing Spfile Managed By ASM [ID 553915.1]该文档描述，该错误影响到10.2及以上版本的数据库，原因是：

This is due to unpublished bug 5399699 where ORA-600 [kmgs_parameter_update_timeout_1] or similar errors can occur in MMON when ASM is being used.

In 10g MMON manages all memory re-size activity by modifying related parameters. If MMON is not running DBW0 will handle this task. The parameter update activity is triggered by a timeout. Basically this error indicates that the MMON process is not able to write to the SPFILE to store some settings required for dynamic SGA parameter adjustments.

也就是说：这个错误是oracle还未发布的一个bug,bug号是5399699。这个错误是说在10g的数据库里，MMON进程动态的管理内存，MMON（Memory Monitor）是10g数据库的新进程。从10g开始，数据库支持自动调整SGA内存，当需要调整（动态增大或减小）的时候，MMON进程会自动完成，MMON会把这个改变的信息，写入到SPFILE里。

结合最开始的alert日志文件的内容，是由于当时MMON进程无法把这个信息写入到SPFILE里导致的，导致后来MMON后台进程僵死了，在Wed Jul 25 10:56:06 2012的时候，MMON进程又被重启了。然后数据库一直处于“假死”的状态。

Oracle Metalink上这篇文章给的解决方案：

Solution

1. Upgrade to the 10.2.0.4.4 PSU Patch:9352164 or higher where this bug is fixed.

OR

2. Check if Patch:5399699 is available for your RDBMS release and platform.

OR

3. Use one of the following workarounds:

Relocate the spfile either to some other diskgroup on which the archive logs are not being managed.
Move the spfile to the file system

显然，第1个与当前环境下的数据库版本不一致；而第2个方案中，又没有查到Windows 2003 X64平台下的补丁；第3个方案，我当前是RAC的库，如果把SPFILE迁移到文件系统上的话，又不太合适，除非给每个实例单独配置PFILE。

最后，这个问题，对于我这个Oracle菜鸟而言，依然头痛，没有更好的解决方案，如果大家有遇到过类似的案例的话，请不吝赐教！

记录一次在IBM P750小机上给Oracle动态扩展存储

7月 24,2012 留下评论 AIX, Oracle

本文详细描述一次在IBM P750的小机上动态给Oracle数据库扩展存储空间的操作。

背景描述：一套2台IBM P750的小机通过HACMP做的HA，上面跑的是Oracle 11gR2的单实例数据库，除小机自带两块300G本地存储之外，共享存储采用的是IBM DS 5100，做RAID 10之后，可用空间2.1TB。目前该机器上有两个VG：rootvg，datavg。其中，rootvg存放AIX操作系统，由本机自带两块盘提供物理卷，datavg给oracle数据库用，物理卷是阵列上的磁盘。其中，datavg下的逻辑卷/oradata用于存放数据库的数据文件、联机日志文件、控制文件等；/oraflash主要用于存放归档日志和RMAN备份。

1 添加之前，查看当前文件系统使用信息：

$ df -g
Filesystem    GB blocks      Free %Used    Iused %Iused Mounted on
/dev/hd4           1.00      0.78   23%    10542     6% /
/dev/hd2          10.00      7.48   26%    52002     3% /usr
/dev/hd9var        5.00      4.45   12%     8742     1% /var
/dev/hd3          10.00      7.22   28%      729     1% /tmp
/dev/hd1           0.50      0.49    2%      135     1% /home
/dev/hd11admin      0.50      0.50    1%        5     1% /admin
/proc                 -         -    -         -     -  /proc
/dev/hd10opt       0.50      0.23   55%    10267    16% /opt
/dev/livedump      0.50      0.50    1%        4     1% /var/adm/ras/livedump
/dev/oracle      100.00     80.71   20%   357020     2% /u01
/dev/oradata     500.00    135.74   73%       39     1% /oradata
/dev/oraflash    500.00     58.62   89%     1186     1% /oraflash
$

从上看到，挂载在/oraflash下的/dev/oraflash文件系统大小是500G，可用空间剩余58G，挂载在/oradata下的/dev/oradata文件系统大小是500G，剩余空间是135G，因业务量迅速增长，现需要扩充存储空间。接下来准备扩充文件系统/oradata和/oraflash，准备分别扩充300G。

2 扩之前，查看VG信息：

$ lsvg -o
datavg
rootvg
$

看到，当前varyon的卷组是rootvg和datavg。

3 查看datavg的物理卷信息：

$ lsvg -p datavg
datavg:
PV_NAME           PV STATE          TOTAL PPs   FREE PPs    FREE DISTRIBUTION
hdisk2            active            1599        0           00..00..00..00..00
hdisk3            active            1599        0           00..00..00..00..00
hdisk4            active            1199        397         00..00..00..157..240
hdisk5            active            1599        1598        320..319..319..320..320
hdisk6            active            1599        1599        320..320..319..320..320
hdisk7            active            1599        0           00..00..00..00..00
hdisk8            active            1599        0           00..00..00..00..00
hdisk9            active            1599        797         157..00..00..320..320
hdisk10           active            1599        1599        320..320..319..320..320
hdisk11           active            1599        1599        320..320..319..320..320
$

4 查看datavg下的逻辑卷信息：

$ lsvg -l datavg
datavg:
LV NAME             TYPE       LPs     PPs     PVs  LV STATE      MOUNT POINT
oradata             jfs2       4000    4000    3    open/syncd    /oradata
loglv01             jfs2log    1       1       1    open/syncd    N/A
oraflash            jfs2       4000    4000    3    open/syncd    /oraflash
$

从上，看到oradata，oraflash两个逻辑卷都位于datavg卷组下。

5 接下来，查看datavg的详细信息：

$ lsvg datavg
VOLUME GROUP:       datavg                   VG IDENTIFIER:  00f64a5100004c000000012d5e49cf72
VG STATE:           active                   PP SIZE:        128 megabyte(s)
VG PERMISSION:      read/write               TOTAL PPs:      15590 (1995520 megabytes)
MAX LVs:            256                      FREE PPs:       7589 (971392 megabytes)
LVs:                3                        USED PPs:       8001 (1024128 megabytes)
OPEN LVs:           3                        QUORUM:         6 (Enabled)
TOTAL PVs:          10                       VG DESCRIPTORS: 10
STALE PVs:          0                        STALE PPs:      0
ACTIVE PVs:         10                       AUTO ON:        no
MAX PPs per VG:     32768                    MAX PVs:        1024
LTG size (Dynamic): 256 kilobyte(s)          AUTO SYNC:      no
HOT SPARE:          no                       BB POLICY:      relocatable
PV RESTRICTION:     none
$

从上看到，datavg卷组的PP size=128M，Totol PPs=15590，意味着卷组的总大小=128*15590M=1948G，已用8001个PPs(1000G)，可用PPs 7589个(948G)。说明，卷组上还有空间可供使用。

6 查看逻辑卷oraflash的信息：

$ lslv oraflash
LOGICAL VOLUME:     oraflash               VOLUME GROUP:   datavg
LV IDENTIFIER:      00f64a5100004c000000012d5e49cf72.3 PERMISSION:     read/write
VG STATE:           active/complete        LV STATE:       opened/syncd
TYPE:               jfs2                   WRITE VERIFY:   off
MAX LPs:            4000                   PP SIZE:        128 megabyte(s)
COPIES:             1                      SCHED POLICY:   parallel
LPs:                4000                   PPs:            4000
STALE PPs:          0                      BB POLICY:      relocatable
INTER-POLICY:       minimum                RELOCATABLE:    yes
INTRA-POLICY:       middle                 UPPER BOUND:    1024
MOUNT POINT:        /oraflash              LABEL:          /oraflash
MIRROR WRITE CONSISTENCY: on/ACTIVE
EACH LP COPY ON A SEPARATE PV ?: yes
Serialize IO ?:     NO
DEVICESUBTYPE : DS_LVZ
COPY 1 MIRROR POOL: None
COPY 2 MIRROR POOL: None
COPY 3 MIRROR POOL: None
$

这里，看到oraflash逻辑卷MAX LPs、LPs、PPs都是4000，说明如果要直接扩充文件系统的话，扩充之后的文件系统Max LPs不能超过4000，否则就得先扩逻辑卷oraflash了。oraflash文件系统类型是jfs2。

7 顺便查看oraflash占用哪几个物理卷的信息：

$ lslv -l oraflash
oraflash:/oraflash
PV                COPIES        IN BAND       DISTRIBUTION
hdisk7            1599:000:000  20%           320:320:319:320:320
hdisk8            1599:000:000  20%           320:320:319:320:320
hdisk9            802:000:000   39%           163:320:319:000:000
$

8 尝试直接使用smitty直接扩充oraflash文件系统，尝试增加100G，即从现有的500G扩充到600G：

从上看到，oraflash文件系统类型是jfs2，直接以root执行smitty chjfs2命令，进入smitty界面:

然后，选择/oraflash,回车，进入下一操作界面:

将Unit Size选择为G，Number of units输入600，表示600G大小，即该文件系统的扩充目标大小。回车，进入下图：

发现报错，提示逻辑卷oraflash的超出最大4000个Lps。扩充失败。看来得先修改逻辑卷oraflash的属性了。

9 接下来，先修改逻辑卷oraflash的最大Lps，这里准备增加2400个，从目前的4000个Lps增加到6400个。2400*128=300G，这个需要事先计算好。

root用户执行smitty chlv,进入下述操作界面：

选择第一项，Change a Logical Volume，然后选择对应的oraflash,如下图：

回车，进入下一界面：

然后，修改MAXIMUM NUMBER of LOGICAL PARTITIONS值为6400，改完之后，直接回车，进入下一界面：

提示成功，状态OK。执行Esc+0退出。

10 再次对/oraflash文件系统进行扩充。依次以root执行smitty chjfs2命令，选择/oraflash文件系统，同样将Unit Size选择为G，Number of units输入800，表示800G大小，即该文件系统的扩充目标大小。这里，因为上述我们已经将逻辑卷oraflash的最大Lps，增加2400个Lps，2400Lps*128M/Lps=300G，所以我们的目标大小是800G。如下图所示：

修改之后，直接回车。发现最后扩充成功了。

如法炮制，通过smit chlv修改oradata逻辑卷的MAX LPs为6400个之后，然后，执行smit chjfs2选择修改/oradata文件系统，扩展到800G。

最终，修改之后，文件系统使用信息如下：

# df -g
Filesystem    GB blocks      Free %Used    Iused %Iused Mounted on
/dev/hd4           1.00      0.78   23%    10542     6% /
/dev/hd2          10.00      7.48   26%    52002     3% /usr
/dev/hd9var        5.00      4.45   12%     8742     1% /var
/dev/hd3          10.00      7.22   28%      729     1% /tmp
/dev/hd1           0.50      0.49    2%      135     1% /home
/dev/hd11admin      0.50      0.50    1%        5     1% /admin
/proc                 -         -    -         -     -  /proc
/dev/hd10opt       0.50      0.23   55%    10267    16% /opt
/dev/livedump      0.50      0.50    1%        4     1% /var/adm/ras/livedump
/dev/oracle      100.00     80.71   20%   357030     2% /u01
/dev/oradata     800.00    435.70   46%       39     1% /oradata
/dev/oraflash    800.00    358.54   56%     1187     1% /oraflash
#

看到/oradata和/oraflash已从500G扩充到800G。同时，看到datavg的FREE PPS变少了，从之前的7589减少到2789,减少了7589-2789=4800个，正好是分别往oradata和oraflash上加的2400个：

# lsvg datavg
VOLUME GROUP:       datavg                   VG IDENTIFIER:  00f64a5100004c000000012d5e49cf72
VG STATE:           active                   PP SIZE:        128 megabyte(s)
VG PERMISSION:      read/write               TOTAL PPs:      15590 (1995520 megabytes)
MAX LVs:            256                      FREE PPs:       2789 (356992 megabytes)
LVs:                3                        USED PPs:       12801 (1638528 megabytes)
OPEN LVs:           3                        QUORUM:         6 (Enabled)
TOTAL PVs:          10                       VG DESCRIPTORS: 10
STALE PVs:          0                        STALE PPs:      0
ACTIVE PVs:         10                       AUTO ON:        no
MAX PPs per VG:     32768                    MAX PVs:        1024
LTG size (Dynamic): 256 kilobyte(s)          AUTO SYNC:      no
HOT SPARE:          no                       BB POLICY:      relocatable
PV RESTRICTION:     none
#

并且，oraflash和oradata的PPs都从4000增加到6400：

# lsvg -l datavg
datavg:
LV NAME             TYPE       LPs     PPs     PVs  LV STATE      MOUNT POINT
oradata             jfs2       6400    6400    5    open/syncd    /oradata
loglv01             jfs2log    1       1       1    open/syncd    N/A
oraflash            jfs2       6400    6400    5    open/syncd    /oraflash
#

而此时，oraflash、oradata占用物理卷信息也改变了：

# lslv -l oradata
oradata:/oradata
PV                COPIES        IN BAND       DISTRIBUTION
hdisk2            1599:000:000  20%           320:320:319:320:320
hdisk3            1599:000:000  20%           320:320:319:320:320
hdisk4            1199:000:000  20%           240:240:239:240:240
hdisk11           1599:000:000  20%           320:320:319:320:320
hdisk5            404:000:000   78%           000:319:085:000:000
# lslv -l oraflash
oraflash:/oraflash
PV                COPIES        IN BAND       DISTRIBUTION
hdisk7            1599:000:000  20%           320:320:319:320:320
hdisk8            1599:000:000  20%           320:320:319:320:320
hdisk9            1599:000:000  20%           320:320:319:320:320
hdisk6            1599:000:000  20%           320:320:319:320:320
hdisk10           004:000:000   100%          000:004:000:000:000
#

最终，完成在IBM P750小机上在Oracle数据库正常运行的前提下，动态给Oracle添加存储。

解决VARCHAR2和NVARCHAR2隐式数据类型转换导致的性能问题

7月 17,2012 留下评论 Performance Tuning, SQL

前段时间，接到公司一项目组的数据库优化需求：一个计算费用的存储过程执行起来非常之慢，需要优化。

拿到存储过程之后，快速看了下代码，然后通过PL/SQL Developer去查看SQL代码的执行计划。其中，看到下面的一条简单的SQL语句的执行计划有点儿问题，导致主表CA_SE_MANIFEST走的全表扫描，而该表的数据量很大，而TMP_COM_ID2又是一张临时表。可是主表CA_SE_MANIFEST的BL_NO_ID字段是主键字段，有唯一索引，却没有走索引，这样的话，执行效率应该不会好到哪儿去！

SQL语句：

DELETE FROM TMP_COM_ID2 WHERE EXISTS(SELECT 1 FROM CA_SE_MANIFEST WHERE BL_NO_ID = C_ID AND DOCUMENT_TYPE IN ('1','2'))

执行计划：

经过分析，发现表TMP_COM_ID2的结构信息如下，只有1个字段，其中C_ID字段的数据类型是NVARCHAR2：

SQL> desc tmp_com_id2
Name Type          Nullable Default Comments 
---- ------------- -------- ------- -------- 
C_ID NVARCHAR2(20)                           
 
SQL>

而主表CA_SE_MANIFEST的BL_NO_ID字段数据类型却是VARCHAR2：

SQL> desc ca_se_manifest
Name                      Type         Nullable Default Comments                                                                                                                                                                                                           
------------------ ------------------- -------- ------- -------------------------------- 
BL_NO_ID           VARCHAR2(20 BYTE)                    出口提单序号                                                                                                                                                                                                       
BL_NO              VARCHAR2(40 BYTE)                    同一船名、航次下、分公司的提单号不重复                                                                                                                                             
CARRIER_BL_NO      VARCHAR2(20 BYTE)   Y                船公司提单号 
...
...

基本上定位到了原因，是由于VARCHAR2和NVARCHAR2数据类型不一致导致的隐式数据类型转换，进而导致主表CA_SE_MANIFEST的主键字段BL_NO_ID上的索引失效，最终产生了全表扫描。

经过分析，可以将临时表TMP_COM_ID2的C_ID字段数据类型改成VARCHAR2，而主表CA_SE_MANIFEST的定义信息显然不可轻易修改。修改完临时表的字段数据类型之后，该条SQL的执行计划已经不再是CA_SE_MANIFEST的全表扫描了：

最终，这个小问题得以解决。

补充：① VARCHAR2 可以认为是VARCHAR的变种，它是一个变长的字符串数据类型；

②NVARCHAR2 是包含UNICODE编码格式数据的变长字符串。

一步一步在Linux上安装Oracle 11gR2 RAC (8–完结)

6月 21,2012 有16条评论 oracle 11g, RAC

7 脚本附录

7.1 preusers.sh脚本

#!/bin/bash
#Purpose:Create 6 groups named 'oinstall','dba','asmadmin','asmdba','asmoper','oper', plus 2 users named 'oracle','grid'.
#Also setting the Environment
#variable for oracle user.
#variable for grid user.
#Usage:Log on as the superuser('root'),and then execute the command:#./1preusers.sh
#Author:Asher Huang

echo "Now create 6 groups named 'oinstall','dba','asmadmin','asmdba','asmoper','oper'"
echo "Plus 2 users named 'oracle','grid',Also setting the Environment"


groupadd -g 1000 oinstall 
groupadd -g 1200 asmadmin 
groupadd -g 1201 asmdba 
groupadd -g 1202 asmoper 
useradd -u 1100 -g oinstall -G asmadmin,asmdba,asmoper -d /home/grid -s /bin/bash -c "grid Infrastructure Owner" grid 
echo "grid" | passwd --stdin grid

echo 'export PS1="`/bin/hostname -s`-> "'>> /home/grid/.bash_profile 
echo "export TMP=/tmp">> /home/grid/.bash_profile  
echo 'export TMPDIR=$TMP'>>/home/grid/.bash_profile 
echo "export ORACLE_SID=+ASM1">> /home/grid/.bash_profile 
echo "export ORACLE_BASE=/u01/app/grid">> /home/grid/.bash_profile
echo "export ORACLE_HOME=/u01/app/11.2.0/grid">> /home/grid/.bash_profile
echo "export ORACLE_TERM=xterm">> /home/grid/.bash_profile
echo "export NLS_DATE_FORMAT='yyyy/mm/dd hh24:mi:ss'" >> /home/grid/.bash_profile
echo 'export TNS_ADMIN=$ORACLE_HOME/network/admin'  >> /home/grid/.bash_profile
echo 'export PATH=/usr/sbin:$PATH'>> /home/grid/.bash_profile
echo 'export PATH=$ORACLE_HOME/bin:$PATH'>> /home/grid/.bash_profile
echo 'export LD_LIBRARY_PATH=$ORACLE_HOME/lib:/lib:/usr/lib'>> /home/grid/.bash_profile
echo 'export CLASSPATH=$ORACLE_HOME/JRE:$ORACLE_HOME/jlib:$ORACLE_HOME/rdbms/jlib'>> /home/grid/.bash_profile
echo "export EDITOR=vi" >> /home/grid/.bash_profile
echo "export LANG=en_US" >> /home/grid/.bash_profile
echo "export NLS_LANG=american_america.AL32UTF8" >> /home/grid/.bash_profile
echo "umask 022">> /home/grid/.bash_profile

groupadd -g 1300 dba 
groupadd -g 1301 oper 
useradd -u 1101 -g oinstall -G dba,oper,asmdba -d /home/oracle -s /bin/bash -c "Oracle Software Owner" oracle 
echo "oracle" | passwd --stdin oracle

echo 'export PS1="`/bin/hostname -s`-> "'>> /home/oracle/.bash_profile 
echo "export TMP=/tmp">> /home/oracle/.bash_profile  
echo 'export TMPDIR=$TMP'>>/home/oracle/.bash_profile 
echo "export ORACLE_HOSTNAME=node1.localdomain">> /home/oracle/.bash_profile 
echo "export ORACLE_SID=devdb1">> /home/oracle/.bash_profile 
echo "export ORACLE_BASE=/u01/app/oracle">> /home/oracle/.bash_profile
echo 'export ORACLE_HOME=$ORACLE_BASE/product/11.2.0/db_1'>> /home/oracle/.bash_profile
echo "export ORACLE_UNQNAME=devdb">> /home/oracle/.bash_profile
echo 'export TNS_ADMIN=$ORACLE_HOME/network/admin'  >> /home/oracle/.bash_profile
echo "export ORACLE_TERM=xterm">> /home/oracle/.bash_profile
echo 'export PATH=/usr/sbin:$PATH'>> /home/oracle/.bash_profile
echo 'export PATH=$ORACLE_HOME/bin:$PATH'>> /home/oracle/.bash_profile
echo 'export LD_LIBRARY_PATH=$ORACLE_HOME/lib:/lib:/usr/lib'>> /home/oracle/.bash_profile
echo 'export CLASSPATH=$ORACLE_HOME/JRE:$ORACLE_HOME/jlib:$ORACLE_HOME/rdbms/jlib'>> /home/oracle/.bash_profile
echo "export EDITOR=vi" >> /home/oracle/.bash_profile
echo "export LANG=en_US" >> /home/oracle/.bash_profile
echo "export NLS_LANG=american_america.AL32UTF8" >> /home/oracle/.bash_profile
echo "export NLS_DATE_FORMAT='yyyy/mm/dd hh24:mi:ss'" >> /home/oracle/.bash_profile
echo "umask 022">> /home/oracle/.bash_profile

echo "The Groups and users has been created"
echo "The Environment for grid,oracle also has been set successfully"

7.2 predir.sh脚本

#!/bin/bash
#Purpose:Create the necessary directory for oracle,grid users and change the authention to oracle,grid users.
#Usage:Log on as the superuser('root'),and then execute the command:#./2predir.sh
#Author:Asher Huang

echo "Now create the necessary directory for oracle,grid users and change the authention to oracle,grid users..."
mkdir -p /u01/app/grid 
mkdir -p /u01/app/11.2.0/grid 
mkdir -p /u01/app/oracle 
chown -R oracle:oinstall /u01
chown -R grid:oinstall /u01/app/grid 
chown -R grid:oinstall /u01/app/11.2.0
chmod -R 775 /u01
echo "The necessary directory for oracle,grid users and change the authention to oracle,grid users has been finished"

7.3 prelimits.sh脚本

#!/bin/bash
#Purpose:Change the /etc/security/limits.conf.
#Usage:Log on as the superuser('root'),and then execute the command:#./3prelimits.sh
#Author:Asher Huang

echo "Now modify the /etc/security/limits.conf,but backup it named /etc/security/limits.conf.bak before"
cp /etc/security/limits.conf /etc/security/limits.conf.bak
echo "oracle soft nproc 2047" >>/etc/security/limits.conf
echo "oracle hard nproc 16384" >>/etc/security/limits.conf
echo "oracle soft nofile 1024" >>/etc/security/limits.conf
echo "oracle hard nofile 65536" >>/etc/security/limits.conf
echo "grid soft nproc 2047" >>/etc/security/limits.conf
echo "grid hard nproc 16384" >>/etc/security/limits.conf
echo "grid soft nofile 1024" >>/etc/security/limits.conf
echo "grid hard nofile 65536" >>/etc/security/limits.conf
echo "Modifing the /etc/security/limits.conf has been succeed."

7.4 prelogin.sh脚本

#!/bin/bash
#Purpose:Modify the /etc/pam.d/login.
#Usage:Log on as the superuser('root'),and then execute the command:#./4prelimits.sh
#Author:Asher Huang

echo "Now modify the /etc/pam.d/login,but with a backup named /etc/pam.d/login.bak"
cp /etc/pam.d/login /etc/pam.d/login.bak

echo "session required /lib/security/pam_limits.so" >>/etc/pam.d/login
echo "session required pam_limits.so" >>/etc/pam.d/login

echo "Modifing the /etc/pam.d/login has been succeed."

7.5 preprofile.sh脚本

#!/bin/bash
#Purpose:Modify the /etc/profile.
#Usage:Log on as the superuser('root'),and then execute the command:#./5preprofile.sh
#Author:Asher Huang

echo "Now modify the  /etc/profile,but with a backup named  /etc/profile.bak"
cp /etc/profile /etc/profile.bak
echo 'if [ $USER = "oracle" ]||[ $USER = "grid" ]; then' >>  /etc/profile
echo 'if [ $SHELL = "/bin/ksh" ]; then' >> /etc/profile
echo 'ulimit -p 16384' >> /etc/profile
echo 'ulimit -n 65536' >> /etc/profile
echo 'else' >> /etc/profile
echo 'ulimit -u 16384 -n 65536' >> /etc/profile
echo 'fi' >> /etc/profile
echo 'fi' >> /etc/profile
echo "Modifing the /etc/profile has been succeed."

7.6 presysctl.sh脚本

#!/bin/bash
#Purpose:Modify the /etc/sysctl.conf.
#Usage:Log on as the superuser('root'),and then execute the command:#./6presysctl.sh
#Author:Asher Huang

echo "Now modify the /etc/sysctl.conf,but with a backup named /etc/sysctl.bak"
cp /etc/sysctl.conf /etc/sysctl.conf.bak

echo "fs.aio-max-nr = 1048576" >> /etc/sysctl.conf
echo "fs.file-max = 6815744" >> /etc/sysctl.conf
echo "kernel.shmall = 2097152" >> /etc/sysctl.conf
echo "kernel.shmmax = 1054472192" >> /etc/sysctl.conf
echo "kernel.shmmni = 4096" >> /etc/sysctl.conf
echo "kernel.sem = 250 32000 100 128" >> /etc/sysctl.conf
echo "net.ipv4.ip_local_port_range = 9000 65500" >> /etc/sysctl.conf
echo "net.core.rmem_default = 262144" >> /etc/sysctl.conf
echo "net.core.rmem_max = 4194304" >> /etc/sysctl.conf
echo "net.core.wmem_default = 262144" >> /etc/sysctl.conf
echo "net.core.wmem_max = 1048586" >> /etc/sysctl.conf
echo "net.ipv4.tcp_wmem = 262144 262144 262144" >> /etc/sysctl.conf
echo "net.ipv4.tcp_rmem = 4194304 4194304 4194304" >> /etc/sysctl.conf

echo "Modifing the /etc/sysctl.conf has been succeed."
echo "Now make the changes take effect....."
sysctl -p

一步一步在Linux上安装Oracle 11gR2 RAC (7)

6月 20,2012 有3条评论 oracle 11g, RAC

5 创建ASM磁盘组

以grid用户创建ASM磁盘组，创建的ASM磁盘组为下一步创建数据库提供存储。

① grid用户登录图形界面，执行asmca命令来创建磁盘组：

② 进入ASMCA配置界面后，单击Create，创建新的磁盘组：

③ 输入磁盘组名 DATA，冗余策略选择External，磁盘选择ORCL:VOL3，单击OK：

④ DATA磁盘组创建完成，单击OK：

⑤ 继续创建磁盘组，磁盘组名FLASH，冗余策略选择External，磁盘选择ORCL:VOL4：

⑥ 最后，完成DATA、FLASH磁盘组的创建，Exit推出ASMCA图形配置界面：

至此，利用ASMCA创建好DATA、FLASH磁盘组。且，可以看到连同之前创建的GRIDDG 3个磁盘组均已经被RAC双节点MOUNT。

6 创建RAC数据库

接下来，使用DBCA来创建RAC数据库。

① 以oracle用户登录图形界面，执行dbca，进入DBCA的图形界面，选择第1项，创建RAC数据库：

② 选择创建数据库选项，Next：

③ 选择创建通用数据库，Next：

④ 配置类型选择Admin-Managed，输入数据库名devdb，选择双节点，Next：

⑤ 选择默认，配置OEM、启用数据库自动维护任务，Next：

⑥ 选择数据库用户使用同一口令，Next：

⑦ 数据库存储选择ASM，使用OMF，数据区选择之前创建的DATA磁盘组，Next：

⑧ 指定数据库闪回区，选择之前创建好的FLASH磁盘组，Next：

⑨ 选择创建数据库自带Sample Schema，Next：

⑩ 选择数据库字符集，AL32UTF8，Next：

⑪ 选择默认数据库存储信息，直接Next：

⑫ 单击，Finish，开始创建数据库，Next：

创建数据库可能持续时间稍长：

⑬ 完成创建数据库。

至此，我们完成创建RAC数据库！！！

一步一步在Linux上安装Oracle 11gR2 RAC (6)

6月 19,2012 有3条评论 oracle 11g, RAC

4安装Oracle 软件

从此步骤开始，我们正式安装oracle软件：

① 以oracle用户登录图形界面，执行/home/oracle/database/runInstaller,进入OUI的图形安装界面：

② 进入OUI安装界面后，选择第3项，跳过软件更新，Next：

③ 选择第2项，只安装oracle软件，Next：

④ 选择第2项，安装oracle RAC，选择所有节点，Next：

⑤ 语言选择默认，English，Next：

⑥ 选择第1项，安装企业版软件，Next：

⑦ 选择oracle软件的安装路径，其中ORACLE_BASE,ORACLE_HOME均选择之前已经配置好的，可参照2.5节中的配置信息。Next：

⑧ 选择oracle用户组，Next：

⑨ 执行安装前的预检查，Next：

⑩ 安装概要信息，Install：

⑪ 根据提示以root用户分别在两个节点上执行脚本，Next：

执行/u01/app/oracle/product/11.2.0/db_1/root.sh 脚本：

node1：

[root@node1 ~]# /u01/app/oracle/product/11.2.0/db_1/root.sh

Performing root user operation for Oracle 11g

The following environment variables are set as:

ORACLE_OWNER= oracle

ORACLE_HOME= /u01/app/oracle/product/11.2.0/db_1

Enter the full pathname of the local bin directory: [/usr/local/bin]:

The contents of "dbhome" have not changed. No need to overwrite.

The contents of "oraenv" have not changed. No need to overwrite.

The contents of "coraenv" have not changed. No need to overwrite.

Entries will be added to the /etc/oratab file as needed by

Database Configuration Assistant when a database is created

Finished running generic part of root script.

Now product-specific root actions will be performed.

Finished product-specific root actions.

[root@node1 ~]#

node2：

[root@node2 ~]# /u01/app/oracle/product/11.2.0/db_1/root.sh

Performing root user operation for Oracle 11g

The following environment variables are set as:

ORACLE_OWNER= oracle

ORACLE_HOME= /u01/app/oracle/product/11.2.0/db_1

Enter the full pathname of the local bin directory: [/usr/local/bin]:

The contents of "dbhome" have not changed. No need to overwrite.

The contents of "oraenv" have not changed. No need to overwrite.

The contents of "coraenv" have not changed. No need to overwrite.

Entries will be added to the /etc/oratab file as needed by

Database Configuration Assistant when a database is created

Finished running generic part of root script.

Now product-specific root actions will be performed.

Finished product-specific root actions.

[root@node2 ~]#

⑫ 最后，单击close，完成oracle软件在双节点上的安装。

至此，我们在RAC双节点上完成oracle软件的安装！！！

一步一步在Linux上安装Oracle 11gR2 RAC (5)

6月 18,2012 有3条评论 oracle 11g, RAC

3 安装Grid Infrastructure

从此步骤开始，我们正式安装Grid软件：

① 以grid用户登录图形界面，执行/home/grid/grid/runInstaller,进入OUI的图形安装界面：

② 进入OUI安装界面后，选择第3项，跳过软件更新，Next：

③ 选择集群的Grid Infrastructure，Next：

④ 选择 advanced Installation，Next：

⑤ 语言选择默认，English，Next：

⑥ 去掉Configure GNS选项，按照之前2.1节中的表格输入Cluster Name:scan-cluster,SCAN Name：scan-cluster.localdomain。Next：

⑦ 单击Add，添加第2个节点，Next：

⑧ 确认网络接口，Next：

⑨ 选择ASM，作为存储，Next：

⑩ 输入ASM磁盘组名，这里命名为GRIDDG，冗余策略选择External外部，AU大小选择默认1M，ASM磁盘选择VOL1，VOL2。Next：

⑪ 选择给ASM的SYS、ASMSNMP用户配置为相同的口令，并输入口令，Next：

⑫ 选择不使用IPMI，Next：

⑬ 给ASM指定不同的组，Next：

⑭ 选择GRID软件的安装路径，其中ORACLE_BASE,ORACLE_HOME均选择之前已经配置好的，可参照2.5节中的配置信息。这里需要注意GRID软件的ORACLE_HOME不能是ORACLE_BASE的子目录。

⑮ 选择默认的Inventory，Next：

⑯ 检查出现告警，提示在所有节点上缺失cvuqdisk-1.0.9-1软件包。

可以选择忽略，直接进入下一步安装。也可以从grid安装文件的rpm目录下获取该RPM包，然后进行安装。

node1：

[root@node1 rpm]# pwd

/home/grid/grid/rpm

[root@node1 rpm]# ll

total 12

-rwxr-xr-x 1 root root 8551 Sep 22 2011 cvuqdisk-1.0.9-1.rpm

[root@node1 rpm]# rpm -ivh cvuqdisk-1.0.9-1.rpm

Preparing… ########################################### [100%]

Using default group oinstall to install package

1:cvuqdisk ########################################### [100%]

[root@node1 rpm]#

node2：

[root@node2 ~]# ll

total 96

-rw——- 1 root root 1371 Apr 20 14:48 anaconda-ks.cfg

drwxr-xr-x 2 root root 4096 Apr 26 11:20 asm_rpm

-rwxr-xr-x 1 root root 8551 Apr 27 09:27 cvuqdisk-1.0.9-1.rpm

-rw-r–r– 1 root root 51256 Apr 20 14:48 install.log

-rw-r–r– 1 root root 4077 Apr 20 14:48 install.log.syslog

drwxr-xr-x 2 root root 4096 Apr 24 10:45 shell

[root@node2 ~]# export CVUQDISK_GRP=oinstall

[root@node2 ~]# rpm -ivh cvuqdisk-1.0.9-1.rpm

Preparing… ########################################### [100%]

1:cvuqdisk ########################################### [100%]

[root@node2 ~]#

在所有节点上安装完cvuqdisk-1.0.9-1软件后，重新执行预检查，不再有警告信息。

⑰ 进入安装GRID安装之前的概要信息，Install进行安装：

⑱ 根据提示以root用户分别在两个节点上执行脚本：

执行/u01/app/oraInventory/orainstRoot.sh脚本：

node1：

[root@node1 ~]# /u01/app/oraInventory/orainstRoot.sh

Changing permissions of /u01/app/oraInventory.

Adding read,write permissions for group.

Removing read,write,execute permissions for world.

Changing groupname of /u01/app/oraInventory to oinstall.

The execution of the script is complete.

[root@node1 ~]#

node2：

[root@node2 ~]# /u01/app/oraInventory/orainstRoot.sh

Changing permissions of /u01/app/oraInventory.

Adding read,write permissions for group.

Removing read,write,execute permissions for world.

Changing groupname of /u01/app/oraInventory to oinstall.

The execution of the script is complete.

[root@node2 ~]#

执行/u01/app/11.2.0/grid/root.sh脚本：

node1：

[root@node1 ~]# /u01/app/11.2.0/grid/root

root.sh rootupgrade.sh

[root@node1 ~]# /u01/app/11.2.0/grid/root.sh

Performing root user operation for Oracle 11g

The following environment variables are set as:

ORACLE_OWNER= grid

ORACLE_HOME= /u01/app/11.2.0/grid

Enter the full pathname of the local bin directory: [/usr/local/bin]:

Copying dbhome to /usr/local/bin …

Copying oraenv to /usr/local/bin …

Copying coraenv to /usr/local/bin …

Creating /etc/oratab file…

Entries will be added to the /etc/oratab file as needed by

Database Configuration Assistant when a database is created

Finished running generic part of root script.

Now product-specific root actions will be performed.

Using configuration parameter file: /u01/app/11.2.0/grid/crs/install/crsconfig_params

Creating trace directory

OLR initialization – successful

root wallet

root wallet cert

root cert export

peer wallet

profile reader wallet

pa wallet

peer wallet keys

pa wallet keys

peer cert request

pa cert request

peer cert

pa cert

peer root cert TP

profile reader root cert TP

pa root cert TP

peer pa cert TP

pa peer cert TP

profile reader pa cert TP

profile reader peer cert TP

peer user cert

pa user cert

Adding Clusterware entries to inittab

CRS-2672: Attempting to start 'ora.mdnsd' on 'node1'

CRS-2676: Start of 'ora.mdnsd' on 'node1' succeeded

CRS-2672: Attempting to start 'ora.gpnpd' on 'node1'

CRS-2676: Start of 'ora.gpnpd' on 'node1' succeeded

CRS-2672: Attempting to start 'ora.cssdmonitor' on 'node1'

CRS-2672: Attempting to start 'ora.gipcd' on 'node1'

CRS-2676: Start of 'ora.gipcd' on 'node1' succeeded

CRS-2676: Start of 'ora.cssdmonitor' on 'node1' succeeded

CRS-2672: Attempting to start 'ora.cssd' on 'node1'

CRS-2672: Attempting to start 'ora.diskmon' on 'node1'

CRS-2676: Start of 'ora.diskmon' on 'node1' succeeded

CRS-2676: Start of 'ora.cssd' on 'node1' succeeded

ASM created and started successfully.

Disk Group GRIDDG created successfully.

clscfg: -install mode specified

Successfully accumulated necessary OCR keys.

Creating OCR keys for user 'root', privgrp 'root'..

Operation successful.

CRS-4256: Updating the profile

Successful addition of voting disk 9516d145c0254f9ebf50064a6a916182.

Successfully replaced voting disk group with +GRIDDG.

CRS-4256: Updating the profile

CRS-4266: Voting file(s) successfully replaced

## STATE File Universal Id File Name Disk group

— —– —————– ——— ———

1. ONLINE 9516d145c0254f9ebf50064a6a916182 (ORCL:VOL1) [GRIDDG]

Located 1 voting disk(s).

CRS-2672: Attempting to start 'ora.asm' on 'node1'

CRS-2676: Start of 'ora.asm' on 'node1' succeeded

CRS-2672: Attempting to start 'ora.GRIDDG.dg' on 'node1'

CRS-2676: Start of 'ora.GRIDDG.dg' on 'node1' succeeded

CRS-2672: Attempting to start 'ora.registry.acfs' on 'node1'

CRS-2676: Start of 'ora.registry.acfs' on 'node1' succeeded

Configure Oracle Grid Infrastructure for a Cluster … succeeded

[root@node1 ~]#

node2：

[root@node2 ~]# /u01/app/11.2.0/grid/root.sh

Performing root user operation for Oracle 11g

The following environment variables are set as:

ORACLE_OWNER= grid

ORACLE_HOME= /u01/app/11.2.0/grid

Enter the full pathname of the local bin directory: [/usr/local/bin]:

Copying dbhome to /usr/local/bin …

Copying oraenv to /usr/local/bin …

Copying coraenv to /usr/local/bin …

Creating /etc/oratab file…

Entries will be added to the /etc/oratab file as needed by

Database Configuration Assistant when a database is created

Finished running generic part of root script.

Now product-specific root actions will be performed.

Using configuration parameter file: /u01/app/11.2.0/grid/crs/install/crsconfig_params

Creating trace directory

OLR initialization – successful

Adding Clusterware entries to inittab

CRS-4402: The CSS daemon was started in exclusive mode but found an active CSS daemon on node node1,

number 1, and is terminating

An active cluster was found during exclusive startup, restarting to join the cluster

Configure Oracle Grid Infrastructure for a Cluster … succeeded

[root@node2 ~]#

此时，集群件相关的服务已经启动。当然，ASM实例也将在两个节点上启动。

[root@node1 ~]# su – grid

node1-> crs_stat -t

Name Type Target State Host

————————————————————

ora.GRIDDG.dg ora….up.type ONLINE ONLINE node1

ora….N1.lsnr ora….er.type ONLINE ONLINE node1

ora.asm ora.asm.type ONLINE ONLINE node1

ora.cvu ora.cvu.type ONLINE ONLINE node1

ora.gsd ora.gsd.type OFFLINE OFFLINE

ora….network ora….rk.type ONLINE ONLINE node1

ora….SM1.asm application ONLINE ONLINE node1

ora.node1.gsd application OFFLINE OFFLINE

ora.node1.ons application ONLINE ONLINE node1

ora.node1.vip ora….t1.type ONLINE ONLINE node1

ora….SM2.asm application ONLINE ONLINE node2

ora.node2.gsd application OFFLINE OFFLINE

ora.node2.ons application ONLINE ONLINE node2

ora.node2.vip ora….t1.type ONLINE ONLINE node2

ora.oc4j ora.oc4j.type ONLINE ONLINE node1

ora.ons ora.ons.type ONLINE ONLINE node1

ora….ry.acfs ora….fs.type ONLINE ONLINE node1

ora.scan1.vip ora….ip.type ONLINE ONLINE node1

node1->

⑲ 执行完上述脚本之后，单击OK，Next，进入下一步。

⑳ 最后，单击close，完成GRID软件在双节点上的安装。

至此，GRID集群件安装成功！！！

一步一步在Linux上安装Oracle 11gR2 RAC (4)

6月 14,2012 有9条评论 oracle 11g, RAC

2.14 配置ASM磁盘

在上一步2.13节中，我们已经对RAC双节点已经配置好了共享磁盘，接下来需要将这些共享磁盘格式化、然后用asmlib将其配置为ASM磁盘，用于将来存放OCR、Voting Disk和数据库用。

注意：只需在其中1个节点上格式化就可以，接下来我们选择在node1节点上格式化。

这里我们以asmlib软件来创建ASM磁盘，而不使用raw disk，而且从11gR2开始，OUI的图形界面已经不再支持raw disk。

2.14.1 格式化共享磁盘

① 以root用户分别在两个节点上执行fdisk命令，查看现有硬盘分区信息：

node1：

[root@node1 ~]# fdisk -l

Disk /dev/sda: 21.4 GB, 21474836480 bytes

255 heads, 63 sectors/track, 2610 cylinders

Units = cylinders of 16065 * 512 = 8225280 bytes

Device Boot Start End Blocks Id System

/dev/sda1 * 1 2163 17374266 83 Linux

/dev/sda2 2164 2609 3582495 82 Linux swap / Solaris

Disk /dev/sdb: 524 MB, 524288000 bytes

64 heads, 32 sectors/track, 500 cylinders

Units = cylinders of 2048 * 512 = 1048576 bytes

Disk /dev/sdb doesn't contain a valid partition table

Disk /dev/sdc: 524 MB, 524288000 bytes

64 heads, 32 sectors/track, 500 cylinders

Units = cylinders of 2048 * 512 = 1048576 bytes

Disk /dev/sdc doesn't contain a valid partition table

Disk /dev/sdd: 3221 MB, 3221225472 bytes

255 heads, 63 sectors/track, 391 cylinders

Units = cylinders of 16065 * 512 = 8225280 bytes

Disk /dev/sdd doesn't contain a valid partition table

Disk /dev/sde: 3221 MB, 3221225472 bytes

255 heads, 63 sectors/track, 391 cylinders

Units = cylinders of 16065 * 512 = 8225280 bytes

Disk /dev/sde doesn't contain a valid partition table

[root@node1 ~]#

node2：

[root@node2 ~]# fdisk -l

Disk /dev/sda: 21.4 GB, 21474836480 bytes

255 heads, 63 sectors/track, 2610 cylinders

Units = cylinders of 16065 * 512 = 8225280 bytes

Device Boot Start End Blocks Id System

/dev/sda1 * 1 2163 17374266 83 Linux

/dev/sda2 2164 2609 3582495 82 Linux swap / Solaris

Disk /dev/sdb: 524 MB, 524288000 bytes

64 heads, 32 sectors/track, 500 cylinders

Units = cylinders of 2048 * 512 = 1048576 bytes

Disk /dev/sdb doesn't contain a valid partition table

Disk /dev/sdc: 524 MB, 524288000 bytes

64 heads, 32 sectors/track, 500 cylinders

Units = cylinders of 2048 * 512 = 1048576 bytes

Disk /dev/sdc doesn't contain a valid partition table

Disk /dev/sdd: 3221 MB, 3221225472 bytes

255 heads, 63 sectors/track, 391 cylinders

Units = cylinders of 16065 * 512 = 8225280 bytes

Disk /dev/sdd doesn't contain a valid partition table

Disk /dev/sde: 3221 MB, 3221225472 bytes

255 heads, 63 sectors/track, 391 cylinders

Units = cylinders of 16065 * 512 = 8225280 bytes

Disk /dev/sde doesn't contain a valid partition table

[root@node2 ~]#

从上，我们可以看到目前两个节点上的分区信息一致：其中/dev/sda是用于存放操作系统的，/dev/sdb、/dev/sdc、/dev/sdd、/dev/sde这4块盘都没有分区信息，这是我们在上一步2.13节中配置的4块共享磁盘。

② root用户在node1上格式化/dev/sdb、/dev/sdc、/dev/sdd、/dev/sde这4块盘

[root@node1 ~]# fdisk /dev/sdb

Device contains neither a valid DOS partition table, nor Sun, SGI or OSF disklabel

Building a new DOS disklabel. Changes will remain in memory only,

until you decide to write them. After that, of course, the previous

content won't be recoverable.

Warning: invalid flag 0x0000 of partition table 4 will be corrected by w(rite)

Command (m for help): n

Command action

e extended

p primary partition (1-4)

p

Partition number (1-4): 1

First cylinder (1-500, default 1):

Using default value 1

Last cylinder or +size or +sizeM or +sizeK (1-500, default 500):

Using default value 500

Command (m for help): w

The partition table has been altered!

Calling ioctl() to re-read partition table.

Syncing disks.

[root@node1 ~]#

说明：fdisk /dev/sdb表示要对/dev/sdb磁盘进行格式化，其中，输入的命令分别表示：

n表示新建1个分区；

p表示分区类型选择为primary partition 主分区；

1表示分区编号从1开始；

起始、终止柱面选择默认值，即1和500；

w表示将新建的分区信息写入硬盘分区表。

③ 重复上述步骤②，以root用户在node1上分别格式化其余3块磁盘：

④ 格式化完毕之后，在node1，node2节点上分别看到下述信息：

node1：

[root@node1 ~]# fdisk -l

Disk /dev/sda: 21.4 GB, 21474836480 bytes

255 heads, 63 sectors/track, 2610 cylinders

Units = cylinders of 16065 * 512 = 8225280 bytes

Device Boot Start End Blocks Id System

/dev/sda1 * 1 2163 17374266 83 Linux

/dev/sda2 2164 2609 3582495 82 Linux swap / Solaris

Disk /dev/sdb: 524 MB, 524288000 bytes

64 heads, 32 sectors/track, 500 cylinders

Units = cylinders of 2048 * 512 = 1048576 bytes

Device Boot Start End Blocks Id System

/dev/sdb1 1 500 511984 83 Linux

Disk /dev/sdc: 524 MB, 524288000 bytes

64 heads, 32 sectors/track, 500 cylinders

Units = cylinders of 2048 * 512 = 1048576 bytes

Device Boot Start End Blocks Id System

/dev/sdc1 1 500 511984 83 Linux

Disk /dev/sdd: 3221 MB, 3221225472 bytes

255 heads, 63 sectors/track, 391 cylinders

Units = cylinders of 16065 * 512 = 8225280 bytes

Device Boot Start End Blocks Id System

/dev/sdd1 1 391 3140676 83 Linux

Disk /dev/sde: 3221 MB, 3221225472 bytes

255 heads, 63 sectors/track, 391 cylinders

Units = cylinders of 16065 * 512 = 8225280 bytes

Device Boot Start End Blocks Id System

/dev/sde1 1 391 3140676 83 Linux

[root@node1 ~]#

node2：

[root@node2 ~]# fdisk -l

Disk /dev/sda: 21.4 GB, 21474836480 bytes

255 heads, 63 sectors/track, 2610 cylinders

Units = cylinders of 16065 * 512 = 8225280 bytes

Device Boot Start End Blocks Id System

/dev/sda1 * 1 2163 17374266 83 Linux

/dev/sda2 2164 2609 3582495 82 Linux swap / Solaris

Disk /dev/sdb: 524 MB, 524288000 bytes

64 heads, 32 sectors/track, 500 cylinders

Units = cylinders of 2048 * 512 = 1048576 bytes

Device Boot Start End Blocks Id System

/dev/sdb1 1 500 511984 83 Linux

Disk /dev/sdc: 524 MB, 524288000 bytes

64 heads, 32 sectors/track, 500 cylinders

Units = cylinders of 2048 * 512 = 1048576 bytes

Device Boot Start End Blocks Id System

/dev/sdc1 1 500 511984 83 Linux

Disk /dev/sdd: 3221 MB, 3221225472 bytes

255 heads, 63 sectors/track, 391 cylinders

Units = cylinders of 16065 * 512 = 8225280 bytes

Device Boot Start End Blocks Id System

/dev/sdd1 1 391 3140676 83 Linux

Disk /dev/sde: 3221 MB, 3221225472 bytes

255 heads, 63 sectors/track, 391 cylinders

Units = cylinders of 16065 * 512 = 8225280 bytes

Device Boot Start End Blocks Id System

/dev/sde1 1 391 3140676 83 Linux

[root@node2 ~]#

至此，格式化共享磁盘完毕。

2.14.2 在两个节点上安装ASM RPM软件包

在安装ASM软件包时，要注意选择的软件包要与操作系统平台、内核版本选择一致。ASM软件包可以到Oracle官网下载。

node1安装：

[root@node1 ~]# rpm -qa|grep asm

用上述命令，并未发现节点1上安装任何asm软件包。

[root@node1 ~]# cd asm_rpm/

[root@node1 asm_rpm]# ll

total 136

-rw-r–r– 1 root root 25977 Apr 26 11:19 oracleasm-2.6.18-194.el5-2.0.5-1.el5.x86_64.rpm

-rw-r–r– 1 root root 14176 Apr 26 11:19 oracleasmlib-2.0.4-1.el5.x86_64.rpm

-rw-r–r– 1 root root 89027 Apr 26 11:19 oracleasm-support-2.1.3-1.el5.x86_64.rpm

[root@node1 asm_rpm]# rpm -ivh oracleasm-support-2.1.3-1.el5.x86_64.rpm

warning: oracleasm-support-2.1.3-1.el5.x86_64.rpm: Header V3 DSA signature: NOKEY, key ID 1e5e0159

Preparing… ########################################### [100%]

1:oracleasm-support ########################################### [100%]

[root@node1 asm_rpm]# rpm -ivh oracleasm-2.6.18-194.el5-2.0.5-1.el5.x86_64.rpm

warning: oracleasm-2.6.18-194.el5-2.0.5-1.el5.x86_64.rpm: Header V3 DSA signature: NOKEY, key ID 1e5e0159

Preparing… ########################################### [100%]

1:oracleasm-2.6.18-194.el########################################### [100%]

[root@node1 asm_rpm]# rpm -ivh oracleasmlib-2.0.4-1.el5.x86_64.rpm

warning: oracleasmlib-2.0.4-1.el5.x86_64.rpm: Header V3 DSA signature: NOKEY, key ID 1e5e0159

Preparing… ########################################### [100%]

1:oracleasmlib ########################################### [100%]

[root@node1 asm_rpm]# rpm -qa|grep asm

oracleasm-2.6.18-194.el5-2.0.5-1.el5

oracleasm-support-2.1.3-1.el5

oracleasmlib-2.0.4-1.el5

[root@node1 asm_rpm]#

node2 安装：

[root@node2 asm_rpm]# ll

total 136

-rw-r–r– 1 root root 25977 Apr 26 11:20 oracleasm-2.6.18-194.el5-2.0.5-1.el5.x86_64.rpm

-rw-r–r– 1 root root 14176 Apr 26 11:20 oracleasmlib-2.0.4-1.el5.x86_64.rpm

-rw-r–r– 1 root root 89027 Apr 26 11:20 oracleasm-support-2.1.3-1.el5.x86_64.rpm

[root@node2 asm_rpm]# rpm -qa|grep asm

[root@node2 asm_rpm]# rpm -ivh oracleasm-support-2.1.3-1.el5.x86_64.rpm

warning: oracleasm-support-2.1.3-1.el5.x86_64.rpm: Header V3 DSA signature: NOKEY, key ID 1e5e0159

Preparing… ########################################### [100%]

1:oracleasm-support ########################################### [100%]

[root@node2 asm_rpm]# rpm -ivh oracleasm-2.6.18-194.el5-2.0.5-1.el5.x86_64.rpm

warning: oracleasm-2.6.18-194.el5-2.0.5-1.el5.x86_64.rpm: Header V3 DSA signature: NOKEY, key ID 1e5e0159

Preparing… ########################################### [100%]

1:oracleasm-2.6.18-194.el########################################### [100%]

[root@node2 asm_rpm]# rpm -ivh oracleasmlib-2.0.4-1.el5.x86_64.rpm

warning: oracleasmlib-2.0.4-1.el5.x86_64.rpm: Header V3 DSA signature: NOKEY, key ID 1e5e0159

Preparing… ########################################### [100%]

1:oracleasmlib ########################################### [100%]

[root@node2 asm_rpm]# rpm -qa|grep asm

oracleasmlib-2.0.4-1.el5

oracleasm-support-2.1.3-1.el5

oracleasm-2.6.18-194.el5-2.0.5-1.el5

[root@node2 asm_rpm]#

说明：安装上述3个ASM RPM软件包时要先安装oracleasm-support-2.1.3-1.el5软件包，其次安装oracleasm-2.6.18-194.el5-2.0.5-1.el5软件包，最后安装oracleasmlib-2.0.4-1.el5软件包。

安装完毕后，执行 rpm -qa|grep asm确认是否安装成功。

2.14.3 配置ASM driver服务

在node1上以root用户进行配置。在安装完上述步骤2.14.2节中的3个ASM RPM软件包之后，可以通过执行/usr/sbin/oracleasm命令来进行配置，也可以通过执行/etc/init.d/oracleasm命令来进行配置，后者命令是Oracle 10g中进行ASM配置的命令，Oracle推荐执行前者命令，不过后者命令保留使用。

① 查看ASM服务状态：

[root@node1 ~]# /usr/sbin/oracleasm status

Checking if ASM is loaded: no

Checking if /dev/oracleasm is mounted: no

[root@node1 ~]#

看到，默认情况下ASM服务并未开启。具体命令和相关参数可以直接执行下述命令来获取：

[root@node1 ~]# /usr/sbin/oracleasm -h

Usage: oracleasm [–exec-path=<exec_path>] <command> [ <args> ]

oracleasm –exec-path

oracleasm -h

oracleasm -V

The basic oracleasm commands are:

configure Configure the Oracle Linux ASMLib driver

init Load and initialize the ASMLib driver

exit Stop the ASMLib driver

scandisks Scan the system for Oracle ASMLib disks

status Display the status of the Oracle ASMLib driver

listdisks List known Oracle ASMLib disks

querydisk Determine if a disk belongs to Oracle ASMlib

createdisk Allocate a device for Oracle ASMLib use

deletedisk Return a device to the operating system

renamedisk Change the label of an Oracle ASMlib disk

update-driver Download the latest ASMLib driver

[root@node1 ~]#

② 配置ASM服务：

[root@node1 ~]# /usr/sbin/oracleasm configure -i

Configuring the Oracle ASM library driver.

This will configure the on-boot properties of the Oracle ASM library

driver. The following questions will determine whether the driver is

loaded on boot and what permissions it will have. The current values

will be shown in brackets ('[]'). Hitting <ENTER> without typing an

answer will keep that current value. Ctrl-C will abort.

Default user to own the driver interface []: grid

Default group to own the driver interface []: asmadmin

Start Oracle ASM library driver on boot (y/n) [n]: y

Scan for Oracle ASM disks on boot (y/n) [y]: y

Writing Oracle ASM library driver configuration: done

[root@node1 ~]# /usr/sbin/oracleasm status

Checking if ASM is loaded: no

Checking if /dev/oracleasm is mounted: no

[root@node1 ~]# /usr/sbin/oracleasm init

Loading module "oracleasm": oracleasm

Mounting ASMlib driver filesystem: /dev/oracleasm

[root@node1 ~]# /usr/sbin/oracleasm configure

ORACLEASM_ENABLED=true

ORACLEASM_UID=grid

ORACLEASM_GID=asmadmin

ORACLEASM_SCANBOOT=true

ORACLEASM_SCANORDER=""

ORACLEASM_SCANEXCLUDE=""

[root@node1 ~]#

说明：/usr/sbin/oracleasm configure -i命令进行配置时，用户配置为grid，组为asmadmin，启动ASM library driver驱动服务，并且将其配置为随着操作系统的启动而自动启动。

配置完成后，记得执行 /usr/sbin/oracleasm init命令来加载oracleasm内核模块。

③ 在node2上执行上述步骤②，完成ASM服务配置。

2.14.4 配置ASM磁盘

我们安装ASM RPM软件包，配置ASM 驱动服务的最终目的是要创建ASM磁盘，为将来安装grid软件、创建Oracle数据库提供存储。

说明：只需在一个节点上创建ASM磁盘即可！创建完之后，在其它节点上执行/usr/sbin/oracleasm scandisks之后，就可看到ASM磁盘。

接下来，开始创建ASM磁盘：

① 执行/usr/sbin/oracleasm createdisk来创建ASM磁盘

[root@node1 ~]# /usr/sbin/oracleasm listdisks

[root@node1 ~]# /usr/sbin/oracleasm createdisk -h

Usage: oracleasm-createdisk [-l <manager>] [-v] <label> <device>

[root@node1 ~]# /usr/sbin/oracleasm createdisk VOL1 /dev/sdb1

Writing disk header: done

Instantiating disk: done

[root@node1 ~]# /usr/sbin/oracleasm createdisk VOL2 /dev/sdc1

Writing disk header: done

Instantiating disk: done

[root@node1 ~]# /usr/sbin/oracleasm createdisk VOL3 /dev/sdd1

Writing disk header: done

Instantiating disk: done

[root@node1 ~]# /usr/sbin/oracleasm createdisk VOL4 /dev/sde1

Writing disk header: done

Instantiating disk: done

[root@node1 ~]# /usr/sbin/oracleasm listdisks

VOL1

VOL2

VOL3

VOL4

[root@node1 ~]#

从上看到，创建出来4块ASM磁盘。此时，node2上还看不到刚创建的ASM磁盘。

② node2 执行/usr/sbin/oracleasm scandisks扫描磁盘

[root@node2 ~]# /usr/sbin/oracleasm listdisks

[root@node2 ~]# /usr/sbin/oracleasm scandisks

Reloading disk partitions: done

Cleaning any stale ASM disks…

Scanning system for ASM disks…

Instantiating disk "VOL1"

Instantiating disk "VOL2"

Instantiating disk "VOL3"

Instantiating disk "VOL4"

[root@node2 ~]# /usr/sbin/oracleasm listdisks

VOL1

VOL2

VOL3

VOL4

[root@node2 ~]#

③ 如何确定ASM磁盘同物理磁盘之间的对应关系？

[root@node1 ~]# /usr/sbin/oracleasm querydisk /dev/sd*

Device "/dev/sda" is not marked as an ASM disk

Device "/dev/sda1" is not marked as an ASM disk

Device "/dev/sda2" is not marked as an ASM disk

Device "/dev/sdb" is not marked as an ASM disk

Device "/dev/sdb1" is marked an ASM disk with the label "VOL1"

Device "/dev/sdc" is not marked as an ASM disk

Device "/dev/sdc1" is marked an ASM disk with the label "VOL2"

Device "/dev/sdd" is not marked as an ASM disk

Device "/dev/sdd1" is marked an ASM disk with the label "VOL3"

Device "/dev/sde" is not marked as an ASM disk

Device "/dev/sde1" is marked an ASM disk with the label "VOL4"

[root@node1 ~]#

至此，ASM磁盘准备工作已经完成！

2.15 解压安装介质

在获取开篇1.2节中提到的安装介质如下：

[root@node1 ~]# ls -l

total 3401724

-rw——- 1 root root 1376 Apr 20 14:05 anaconda-ks.cfg

drwxr-xr-x 2 root root 4096 Apr 26 11:19 asm_rpm

-rw-r–r– 1 root root 51217 Apr 20 14:05 install.log

-rw-r–r– 1 root root 4077 Apr 20 14:05 install.log.syslog

-rw-r–r– 1 root root 1358454646 Apr 20 16:22 p10404530_112030_Linux-x86-64_1of7.zip

-rw-r–r– 1 root root 1142195302 Apr 20 16:29 p10404530_112030_Linux-x86-64_2of7.zip

-rw-r–r– 1 root root 979195792 Apr 20 17:07 p10404530_112030_Linux-x86-64_3of7.zip

drwxr-xr-x 2 root root 4096 Apr 24 10:17 shell

[root@node1 ~]#

其中：

p10404530_112030_Linux-x86-64_1of7.zip和

p10404530_112030_Linux-x86-64_2of7.zip

是Oracle软件的安装介质。

p10404530_112030_Linux-x86-64_3of7.zip是GRID软件的安装介质。

注意：这里的3个软件包均是来源于MetaLink网站，其版本均是目前Oracle 11g的最新版本，11.2.0.3.0。如果没有MetaLink账号的话，也可以从oracle官方网站免费获取11.2.0.1.0的版本软件来进行安装和配置。

我们通过下述命令来解压上述3个压缩软件包：

[root@node1 ~]# unzip p10404530_112030_Linux-x86-64_1of7.zip

[root@node1 ~]# unzip p10404530_112030_Linux-x86-64_2of7.zip

[root@node1 ~]# unzip p10404530_112030_Linux-x86-64_3of7.zip

解压之后，信息如下：

[root@node1 ~]# ls -l

total 3401724

-rw——- 1 root root 1376 Apr 20 14:05 anaconda-ks.cfg

drwxr-xr-x 2 root root 4096 Apr 26 11:19 asm_rpm

drwxr-xr-x 8 root root 4096 Sep 22 2011 database

drwxr-xr-x 8 root root 4096 Sep 22 2011 grid

-rw-r–r– 1 root root 51217 Apr 20 14:05 install.log

-rw-r–r– 1 root root 4077 Apr 20 14:05 install.log.syslog

-rw-r–r– 1 root root 1358454646 Apr 20 16:22 p10404530_112030_Linux-x86-64_1of7.zip

-rw-r–r– 1 root root 1142195302 Apr 20 16:29 p10404530_112030_Linux-x86-64_2of7.zip

-rw-r–r– 1 root root 979195792 Apr 20 17:07 p10404530_112030_Linux-x86-64_3of7.zip

drwxr-xr-x 2 root root 4096 Apr 24 10:17 shell

[root@node1 ~]# du -sh database/

2.5G database/

[root@node1 ~]# du -sh grid/

1.1G grid/

[root@node1 ~]#

可以看到，数据库的安装文件2.5G大小，GRID软件的安装1.1GB。

为便于将来安装软件，分别将其move到oracle用户和grid用户的家目录：

[root@node1 ~]# mv database/ /home/oracle/

[root@node1 ~]# mv grid/ /home/grid/

[root@node1 ~]#

2.16 安装前预检查配置信息

在安装 GRID之前，建议先利用CVU(Cluster Verification Utility)检查 CRS的安装前环境。

① 使用 CVU 检查CRS的安装前环境：

[root@node1 ~]# su – grid

node1-> pwd

/home/grid

node1-> ls

Desktop grid

node1-> cd grid/

node1-> ll

total 72

drwxr-xr-x 9 root root 4096 Sep 22 2011 doc

drwxr-xr-x 4 root root 4096 Sep 22 2011 install

-rwxr-xr-x 1 root root 28122 Sep 22 2011 readme.html

drwxr-xr-x 2 root root 4096 Sep 22 2011 response

drwxr-xr-x 2 root root 4096 Sep 22 2011 rpm

-rwxr-xr-x 1 root root 4878 Sep 22 2011 runcluvfy.sh

-rwxr-xr-x 1 root root 3227 Sep 22 2011 runInstaller

drwxr-xr-x 2 root root 4096 Sep 22 2011 sshsetup

drwxr-xr-x 14 root root 4096 Sep 22 2011 stage

-rwxr-xr-x 1 root root 4326 Sep 2 2011 welcome.html

node1-> ./runcluvfy.sh stage -pre crsinst -n node1,node2 -fixup -verbose

Performing pre-checks for cluster services setup

Checking node reachability…

Check: Node reachability from node "node1"

Destination Node Reachable?

———————————— ————————

node1 yes

node2 yes

Result: Node reachability check passed from node "node1"

Checking user equivalence…

Check: User equivalence for user "grid"

Node Name Status

———————————— ————————

node2 passed

node1 passed

Result: User equivalence check passed for user "grid"

Checking node connectivity…

Checking hosts config file…

Node Name Status

———————————— ————————

node2 passed

node1 passed

Verification of the hosts config file successful

Interface information for node "node2"

Name IP Address Subnet Gateway Def. Gateway HW Address MTU

—— ————— ————— ————— ————— —————– ——

eth0 172.16.0.192 172.16.0.0 0.0.0.0 172.16.15.254 00:0C:29:00:42:89 1500

eth1 192.168.94.12 192.168.94.0 0.0.0.0 172.16.15.254 00:0C:29:00:42:93 1500

Interface information for node "node1"

Name IP Address Subnet Gateway Def. Gateway HW Address MTU

—— ————— ————— ————— ————— —————– ——

eth0 172.16.0.191 172.16.0.0 0.0.0.0 172.16.15.254 00:0C:29:A2:AE:1F 1500

eth1 192.168.94.11 192.168.94.0 0.0.0.0 172.16.15.254 00:0C:29:A2:AE:29 1500

Check: Node connectivity of subnet "172.16.0.0"

Source Destination Connected?

—————————— —————————— —————-

node2[172.16.0.192] node1[172.16.0.191] yes

Result: Node connectivity passed for subnet "172.16.0.0" with node(s) node2,node1

Check: TCP connectivity of subnet "172.16.0.0"

Source Destination Connected?

—————————— —————————— —————-

node1:172.16.0.191 node2:172.16.0.192 passed

Result: TCP connectivity check passed for subnet "172.16.0.0"

Check: Node connectivity of subnet "192.168.94.0"

Source Destination Connected?

—————————— —————————— —————-

node2[192.168.94.12] node1[192.168.94.11] yes

Result: Node connectivity passed for subnet "192.168.94.0" with node(s) node2,node1

Check: TCP connectivity of subnet "192.168.94.0"

Source Destination Connected?

—————————— —————————— —————-

node1:192.168.94.11 node2:192.168.94.12 passed

Result: TCP connectivity check passed for subnet "192.168.94.0"

Interfaces found on subnet "172.16.0.0" that are likely candidates for VIP are:

node2 eth0:172.16.0.192

node1 eth0:172.16.0.191

Interfaces found on subnet "192.168.94.0" that are likely candidates for a private interconnect are:

node2 eth1:192.168.94.12

node1 eth1:192.168.94.11

Checking subnet mask consistency…

Subnet mask consistency check passed for subnet "172.16.0.0".

Subnet mask consistency check passed for subnet "192.168.94.0".

Subnet mask consistency check passed.

Result: Node connectivity check passed

Checking multicast communication…

Checking subnet "172.16.0.0" for multicast communication with multicast group "230.0.1.0"…

Check of subnet "172.16.0.0" for multicast communication with multicast group "230.0.1.0" passed.

Checking subnet "192.168.94.0" for multicast communication with multicast group "230.0.1.0"…

Check of subnet "192.168.94.0" for multicast communication with multicast group "230.0.1.0" passed.

Check of multicast communication passed.

Checking ASMLib configuration.

Node Name Status

———————————— ————————

node2 passed

node1 passed

Result: Check for ASMLib configuration passed.

Check: Total memory

Node Name Available Required Status

———— ———————— ———————— ———-

node2 1.9641GB (2059516.0KB) 1.5GB (1572864.0KB) passed

node1 1.9641GB (2059516.0KB) 1.5GB (1572864.0KB) passed

Result: Total memory check passed

Check: Available memory

Node Name Available Required Status

———— ———————— ———————— ———-

node2 1.8744GB (1965456.0KB) 50MB (51200.0KB) passed

node1 1.7501GB (1835088.0KB) 50MB (51200.0KB) passed

Result: Available memory check passed

Check: Swap space

Node Name Available Required Status

———— ———————— ———————— ———-

node2 3.4165GB (3582484.0KB) 2.9462GB (3089274.0KB) passed

node1 3.4165GB (3582484.0KB) 2.9462GB (3089274.0KB) passed

Result: Swap space check passed

Check: Free disk space for "node2:/tmp"

Path Node Name Mount point Available Required Status

—————- ———— ———— ———— ———— ————

/tmp node2 / 13.0361GB 1GB passed

Result: Free disk space check passed for "node2:/tmp"

Check: Free disk space for "node1:/tmp"

Path Node Name Mount point Available Required Status

—————- ———— ———— ———— ———— ————

/tmp node1 / 5.874GB 1GB passed

Result: Free disk space check passed for "node1:/tmp"

Check: User existence for "grid"

Node Name Status Comment

———— ———————— ————————

node2 passed exists(1100)

node1 passed exists(1100)

Checking for multiple users with UID value 1100

Result: Check for multiple users with UID value 1100 passed

Result: User existence check passed for "grid"

Check: Group existence for "oinstall"

Node Name Status Comment

———— ———————— ————————

node2 passed exists

node1 passed exists

Result: Group existence check passed for "oinstall"

Check: Group existence for "dba"

Node Name Status Comment

———— ———————— ————————

node2 passed exists

node1 passed exists

Result: Group existence check passed for "dba"

Check: Membership of user "grid" in group "oinstall" [as Primary]

Node Name User Exists Group Exists User in Group Primary Status

—————- ———— ———— ———— ———— ————

node2 yes yes yes yes passed

node1 yes yes yes yes passed

Result: Membership check for user "grid" in group "oinstall" [as Primary] passed

Check: Membership of user "grid" in group "dba"

Node Name User Exists Group Exists User in Group Status

—————- ———— ———— ———— —————-

node2 yes yes no failed

node1 yes yes no failed

Result: Membership check for user "grid" in group "dba" failed

Check: Run level

Node Name run level Required Status

———— ———————— ———————— ———-

node2 5 3,5 passed

node1 5 3,5 passed

Result: Run level check passed

Check: Hard limits for "maximum open file descriptors"

Node Name Type Available Required Status

—————- ———— ———— ———— —————-

node2 hard 65536 65536 passed

node1 hard 65536 65536 passed

Result: Hard limits check passed for "maximum open file descriptors"

Check: Soft limits for "maximum open file descriptors"

Node Name Type Available Required Status

—————- ———— ———— ———— —————-

node2 soft 1024 1024 passed

node1 soft 1024 1024 passed

Result: Soft limits check passed for "maximum open file descriptors"

Check: Hard limits for "maximum user processes"

Node Name Type Available Required Status

—————- ———— ———— ———— —————-

node2 hard 16384 16384 passed

node1 hard 16384 16384 passed

Result: Hard limits check passed for "maximum user processes"

Check: Soft limits for "maximum user processes"

Node Name Type Available Required Status

—————- ———— ———— ———— —————-

node2 soft 2047 2047 passed

node1 soft 2047 2047 passed

Result: Soft limits check passed for "maximum user processes"

Check: System architecture

Node Name Available Required Status

———— ———————— ———————— ———-

node2 x86_64 x86_64 passed

node1 x86_64 x86_64 passed

Result: System architecture check passed

Check: Kernel version

Node Name Available Required Status

———— ———————— ———————— ———-

node2 2.6.18-194.el5 2.6.18 passed

node1 2.6.18-194.el5 2.6.18 passed

Result: Kernel version check passed

Check: Kernel parameter for "semmsl"

Node Name Current Configured Required Status Comment

—————- ———— ———— ———— ———— ————

node2 250 250 250 passed

node1 250 250 250 passed

Result: Kernel parameter check passed for "semmsl"

Check: Kernel parameter for "semmns"

Node Name Current Configured Required Status Comment

—————- ———— ———— ———— ———— ————

node2 32000 32000 32000 passed

node1 32000 32000 32000 passed

Result: Kernel parameter check passed for "semmns"

Check: Kernel parameter for "semopm"

Node Name Current Configured Required Status Comment

—————- ———— ———— ———— ———— ————

node2 100 100 100 passed

node1 100 100 100 passed

Result: Kernel parameter check passed for "semopm"

Check: Kernel parameter for "semmni"

Node Name Current Configured Required Status Comment

—————- ———— ———— ———— ———— ————

node2 128 128 128 passed

node1 128 128 128 passed

Result: Kernel parameter check passed for "semmni"

Check: Kernel parameter for "shmmax"

Node Name Current Configured Required Status Comment

—————- ———— ———— ———— ———— ————

node2 1054472192 1054472192 1054472192 passed

node1 1054472192 1054472192 1054472192 passed

Result: Kernel parameter check passed for "shmmax"

Check: Kernel parameter for "shmmni"

Node Name Current Configured Required Status Comment

—————- ———— ———— ———— ———— ————

node2 4096 4096 4096 passed

node1 4096 4096 4096 passed

Result: Kernel parameter check passed for "shmmni"

Check: Kernel parameter for "shmall"

Node Name Current Configured Required Status Comment

—————- ———— ———— ———— ———— ————

node2 2097152 2097152 2097152 passed

node1 2097152 2097152 2097152 passed

Result: Kernel parameter check passed for "shmall"

Check: Kernel parameter for "file-max"

Node Name Current Configured Required Status Comment

—————- ———— ———— ———— ———— ————

node2 6815744 6815744 6815744 passed

node1 6815744 6815744 6815744 passed

Result: Kernel parameter check passed for "file-max"

Check: Kernel parameter for "ip_local_port_range"

Node Name Current Configured Required Status Comment

—————- ———— ———— ———— ———— ————

node2 between 9000.0 & 65500.0 between 9000.0 & 65500.0 between 9000.0 & 65500.0 passed

node1 between 9000.0 & 65500.0 between 9000.0 & 65500.0 between 9000.0 & 65500.0 passed

Result: Kernel parameter check passed for "ip_local_port_range"

Check: Kernel parameter for "rmem_default"

Node Name Current Configured Required Status Comment

—————- ———— ———— ———— ———— ————

node2 262144 262144 262144 passed

node1 262144 262144 262144 passed

Result: Kernel parameter check passed for "rmem_default"

Check: Kernel parameter for "rmem_max"

Node Name Current Configured Required Status Comment

—————- ———— ———— ———— ———— ————

node2 4194304 4194304 4194304 passed

node1 4194304 4194304 4194304 passed

Result: Kernel parameter check passed for "rmem_max"

Check: Kernel parameter for "wmem_default"

Node Name Current Configured Required Status Comment

—————- ———— ———— ———— ———— ————

node2 262144 262144 262144 passed

node1 262144 262144 262144 passed

Result: Kernel parameter check passed for "wmem_default"

Check: Kernel parameter for "wmem_max"

Node Name Current Configured Required Status Comment

—————- ———— ———— ———— ———— ————

node2 1048586 1048586 1048576 passed

node1 1048586 1048586 1048576 passed

Result: Kernel parameter check passed for "wmem_max"

Check: Kernel parameter for "aio-max-nr"

Node Name Current Configured Required Status Comment

—————- ———— ———— ———— ———— ————

node2 1048576 1048576 1048576 passed

node1 1048576 1048576 1048576 passed

Result: Kernel parameter check passed for "aio-max-nr"

Check: Package existence for "make"

Node Name Available Required Status

———— ———————— ———————— ———-

node2 make-3.81-3.el5 make-3.81 passed

node1 make-3.81-3.el5 make-3.81 passed

Result: Package existence check passed for "make"

Check: Package existence for "binutils"

Node Name Available Required Status

———— ———————— ———————— ———-

node2 binutils-2.17.50.0.6-14.el5 binutils-2.17.50.0.6 passed

node1 binutils-2.17.50.0.6-14.el5 binutils-2.17.50.0.6 passed

Result: Package existence check passed for "binutils"

Check: Package existence for "gcc(x86_64)"

Node Name Available Required Status

———— ———————— ———————— ———-

node2 gcc(x86_64)-4.1.2-48.el5 gcc(x86_64)-4.1.2 passed

node1 gcc(x86_64)-4.1.2-48.el5 gcc(x86_64)-4.1.2 passed

Result: Package existence check passed for "gcc(x86_64)"

Check: Package existence for "libaio(x86_64)"

Node Name Available Required Status

———— ———————— ———————— ———-

node2 libaio(x86_64)-0.3.106-5 libaio(x86_64)-0.3.106 passed

node1 libaio(x86_64)-0.3.106-5 libaio(x86_64)-0.3.106 passed

Result: Package existence check passed for "libaio(x86_64)"

Check: Package existence for "glibc(x86_64)"

Node Name Available Required Status

———— ———————— ———————— ———-

node2 glibc(x86_64)-2.5-49 glibc(x86_64)-2.5-24 passed

node1 glibc(x86_64)-2.5-49 glibc(x86_64)-2.5-24 passed

Result: Package existence check passed for "glibc(x86_64)"

Check: Package existence for "compat-libstdc++-33(x86_64)"

Node Name Available Required Status

———— ———————— ———————— ———-

node2 compat-libstdc++-33(x86_64)-3.2.3-61 compat-libstdc++-33(x86_64)-3.2.3 passed

node1 compat-libstdc++-33(x86_64)-3.2.3-61 compat-libstdc++-33(x86_64)-3.2.3 passed

Result: Package existence check passed for "compat-libstdc++-33(x86_64)"

Check: Package existence for "elfutils-libelf(x86_64)"

Node Name Available Required Status

———— ———————— ———————— ———-

node2 elfutils-libelf(x86_64)-0.137-3.el5 elfutils-libelf(x86_64)-0.125 passed

node1 elfutils-libelf(x86_64)-0.137-3.el5 elfutils-libelf(x86_64)-0.125 passed

Result: Package existence check passed for "elfutils-libelf(x86_64)"

Check: Package existence for "elfutils-libelf-devel"

Node Name Available Required Status

———— ———————— ———————— ———-

node2 elfutils-libelf-devel-0.137-3.el5 elfutils-libelf-devel-0.125 passed

node1 elfutils-libelf-devel-0.137-3.el5 elfutils-libelf-devel-0.125 passed

Result: Package existence check passed for "elfutils-libelf-devel"

Check: Package existence for "glibc-common"

Node Name Available Required Status

———— ———————— ———————— ———-

node2 glibc-common-2.5-49 glibc-common-2.5 passed

node1 glibc-common-2.5-49 glibc-common-2.5 passed

Result: Package existence check passed for "glibc-common"

Check: Package existence for "glibc-devel(x86_64)"

Node Name Available Required Status

———— ———————— ———————— ———-

node2 glibc-devel(x86_64)-2.5-49 glibc-devel(x86_64)-2.5 passed

node1 glibc-devel(x86_64)-2.5-49 glibc-devel(x86_64)-2.5 passed

Result: Package existence check passed for "glibc-devel(x86_64)"

Check: Package existence for "glibc-headers"

Node Name Available Required Status

———— ———————— ———————— ———-

node2 glibc-headers-2.5-49 glibc-headers-2.5 passed

node1 glibc-headers-2.5-49 glibc-headers-2.5 passed

Result: Package existence check passed for "glibc-headers"

Check: Package existence for "gcc-c++(x86_64)"

Node Name Available Required Status

———— ———————— ———————— ———-

node2 gcc-c++(x86_64)-4.1.2-48.el5 gcc-c++(x86_64)-4.1.2 passed

node1 gcc-c++(x86_64)-4.1.2-48.el5 gcc-c++(x86_64)-4.1.2 passed

Result: Package existence check passed for "gcc-c++(x86_64)"

Check: Package existence for "libaio-devel(x86_64)"

Node Name Available Required Status

———— ———————— ———————— ———-

node2 libaio-devel(x86_64)-0.3.106-5 libaio-devel(x86_64)-0.3.106 passed

node1 libaio-devel(x86_64)-0.3.106-5 libaio-devel(x86_64)-0.3.106 passed

Result: Package existence check passed for "libaio-devel(x86_64)"

Check: Package existence for "libgcc(x86_64)"

Node Name Available Required Status

———— ———————— ———————— ———-

node2 libgcc(x86_64)-4.1.2-48.el5 libgcc(x86_64)-4.1.2 passed

node1 libgcc(x86_64)-4.1.2-48.el5 libgcc(x86_64)-4.1.2 passed

Result: Package existence check passed for "libgcc(x86_64)"

Check: Package existence for "libstdc++(x86_64)"

Node Name Available Required Status

———— ———————— ———————— ———-

node2 libstdc++(x86_64)-4.1.2-48.el5 libstdc++(x86_64)-4.1.2 passed

node1 libstdc++(x86_64)-4.1.2-48.el5 libstdc++(x86_64)-4.1.2 passed

Result: Package existence check passed for "libstdc++(x86_64)"

Check: Package existence for "libstdc++-devel(x86_64)"

Node Name Available Required Status

———— ———————— ———————— ———-

node2 libstdc++-devel(x86_64)-4.1.2-48.el5 libstdc++-devel(x86_64)-4.1.2 passed

node1 libstdc++-devel(x86_64)-4.1.2-48.el5 libstdc++-devel(x86_64)-4.1.2 passed

Result: Package existence check passed for "libstdc++-devel(x86_64)"

Check: Package existence for "sysstat"

Node Name Available Required Status

———— ———————— ———————— ———-

node2 sysstat-7.0.2-3.el5 sysstat-7.0.2 passed

node1 sysstat-7.0.2-3.el5 sysstat-7.0.2 passed

Result: Package existence check passed for "sysstat"

Check: Package existence for "ksh"

Node Name Available Required Status

———— ———————— ———————— ———-

node2 ksh-20100202-1.el5 ksh-20060214 passed

node1 ksh-20100202-1.el5 ksh-20060214 passed

Result: Package existence check passed for "ksh"

Checking for multiple users with UID value 0

Result: Check for multiple users with UID value 0 passed

Check: Current group ID

Result: Current group ID check passed

Starting check for consistency of primary group of root user

Node Name Status

———————————— ————————

node2 passed

node1 passed

Check for consistency of root user's primary group passed

Starting Clock synchronization checks using Network Time Protocol(NTP)…

NTP Configuration file check started…

Network Time Protocol(NTP) configuration file not found on any of the nodes. Oracle Cluster Time

Synchronization Service(CTSS) can be used instead of NTP for time synchronization on the cluster nodes

No NTP Daemons or Services were found to be running

Result: Clock synchronization check using Network Time Protocol(NTP) passed

Checking Core file name pattern consistency…

Core file name pattern consistency check passed.

Checking to make sure user "grid" is not in "root" group

Node Name Status Comment

———— ———————— ————————

node2 passed does not exist

node1 passed does not exist

Result: User "grid" is not part of "root" group. Check passed

Check default user file creation mask

Node Name Available Required Comment

———— ———————— ———————— ———-

node2 0022 0022 passed

node1 0022 0022 passed

Result: Default user file creation mask check passed

Checking consistency of file "/etc/resolv.conf" across nodes

Checking the file "/etc/resolv.conf" to make sure only one of domain and search entries is defined

File "/etc/resolv.conf" does not have both domain and search entries defined

Checking if domain entry in file "/etc/resolv.conf" is consistent across the nodes…

domain entry in file "/etc/resolv.conf" is consistent across nodes

Checking if search entry in file "/etc/resolv.conf" is consistent across the nodes…

search entry in file "/etc/resolv.conf" is consistent across nodes

Checking file "/etc/resolv.conf" to make sure that only one search entry is defined

All nodes have one search entry defined in file "/etc/resolv.conf"

Checking all nodes to make sure that search entry is "localdomain" as found on node "node2"

All nodes of the cluster have same value for 'search'

Checking DNS response time for an unreachable node

Node Name Status

———————————— ————————

node2 passed

node1 passed

The DNS response time for an unreachable node is within acceptable limit on all nodes

File "/etc/resolv.conf" is consistent across nodes

Check: Time zone consistency

Result: Time zone consistency check passed

Fixup information has been generated for following node(s):

node2,node1

Please run the following script on each node as "root" user to execute the fixups:

'/tmp/CVU_11.2.0.3.0_grid/runfixup.sh'

Pre-check for cluster services setup was unsuccessful on all the nodes.

node1->

从上面的预检查结果中，可以看到不成功，其实错误的原因是grid用户不属于dba组！不过，Oracle自动给我们提供的修复的脚本，根据上述提示，分别以root用户在两个节点上执行/tmp/CVU_11.2.0.3.0_grid/runfixup.sh脚本来修复。

node1：

[root@node1 ~]# id grid

uid=1100(grid) gid=1000(oinstall) groups=1000(oinstall),1200(asmadmin),1201(asmdba),1202(asmoper)

[root@node1 ~]#

看到，grid的确不属于dba组。执行脚本，进行修复：

[root@node1 ~]# sh /tmp/CVU_11.2.0.3.0_grid/runfixup.sh

Response file being used is :/tmp/CVU_11.2.0.3.0_grid/fixup.response

Enable file being used is :/tmp/CVU_11.2.0.3.0_grid/fixup.enable

Log file location: /tmp/CVU_11.2.0.3.0_grid/orarun.log

uid=1100(grid) gid=1000(oinstall) groups=1000(oinstall),1200(asmadmin),1201(asmdba),1202(asmoper)

[root@node1 ~]# id grid

uid=1100(grid) gid=1000(oinstall) groups=1000(oinstall),1200(asmadmin),1201(asmdba),1202(asmoper),1300(dba)

[root@node1 ~]#

同样，在node2上执行上述脚本：

[root@node2 ~]# sh /tmp/CVU_11.2.0.3.0_grid/runfixup.sh

Response file being used is :/tmp/CVU_11.2.0.3.0_grid/fixup.response

Enable file being used is :/tmp/CVU_11.2.0.3.0_grid/fixup.enable

Log file location: /tmp/CVU_11.2.0.3.0_grid/orarun.log

uid=1100(grid) gid=1000(oinstall) groups=1000(oinstall),1200(asmadmin),1201(asmdba),1202(asmoper)

[root@node2 ~]# id grid

uid=1100(grid) gid=1000(oinstall) groups=1000(oinstall),1200(asmadmin),1201(asmdba),1202(asmoper),1300(dba)

[root@node2 ~]#

执行完修复脚本后，重新执行预检查：

node1-> ./runcluvfy.sh stage -pre crsinst -n node1,node2 -fixup -verbose

…