Delete a Node from Oracle RAC 11gR2(11.2.0.3) on Oracle Linux 6
This guide shows how to add a node to an existing 11gR2(11.2.0.3) Oracle RAC cluster. It is assumed that the node in question is available and is not part of a GNS/Grid Plug and Play cluster. In other words, the database
is considered to be "Administrator-Managed". Also, the database software is non-shared and uses role separation where "grid" is the clusterware owner and "oracle" owns the database software. This guide uses a 2-node cluster running Oracle Linux 6.3 (x64).
There are two pre-existing nodes "node1" and "node2". We will be adding "node3" to the cluster. This guide does not cover node preparation steps/prerequisites. The assumption is that since there is a pre-existing cluster the user knows how to prepare a node
– from a prerequisite perspective – for cluster addition.
From an existing node, run "cluvfy" to ensure that "node3" – the cluster node to be added – is ready from a hardware and operating system perspective.
cluvfy stage -post hwos -n node3
The cluster verify utility – "cluvfy" – is used to determine the integrity of the cluster and whether it is ready for a new node. From an existing node, run "cluvfy" to verify the integrity of the cluster.
cluvfy stage -pre nodeadd -n node3 -fixup -verbose
[grid@node1 ~]$ cluvfy stage -pre crsinst -n node1,node2,node3 -fixup -verbose Performing pre-checks for cluster services setup Checking node reachability... Check: Node reachability from node "node1" Destination Node Reachable? ------------------------------------ ------------------------ node3 yes node2 yes node1 yes Result: Node reachability check passed from node "node1" Checking user equivalence... Check: User equivalence for user "grid" Node Name Status ------------------------------------ ------------------------ node3 passed node2 passed node1 passed Result: User equivalence check passed for user "grid" Checking node connectivity... Checking hosts config file... Node Name Status ------------------------------------ ------------------------ node3 passed node2 passed node1 passed Verification of the hosts config file successful Interface information for node "node3" Name IP Address Subnet Gateway Def. Gateway HW Address MTU ------ --------------- --------------- --------------- --------------- ----------------- ------ eth0 192.168.1.53 192.168.0.0 0.0.0.0 192.168.1.254 08:00:27:B5:7E:7A 1500 eth1 10.10.10.53 10.10.0.0 0.0.0.0 192.168.1.254 08:00:27:73:27:02 1500 Interface information for node "node2" Name IP Address Subnet Gateway Def. Gateway HW Address MTU ------ --------------- --------------- --------------- --------------- ----------------- ------ eth0 192.168.1.52 192.168.0.0 0.0.0.0 192.168.1.254 08:00:27:B1:3A:93 1500 eth0 192.168.1.152 192.168.0.0 0.0.0.0 192.168.1.254 08:00:27:B1:3A:93 1500 eth1 10.10.10.52 10.10.0.0 0.0.0.0 192.168.1.254 08:00:27:4C:3F:43 1500 eth1 169.254.95.34 169.254.0.0 0.0.0.0 192.168.1.254 08:00:27:4C:3F:43 1500 Interface information for node "node1" Name IP Address Subnet Gateway Def. Gateway HW Address MTU ------ --------------- --------------- --------------- --------------- ----------------- ------ eth0 192.168.1.51 192.168.0.0 0.0.0.0 192.168.1.254 08:00:27:7E:5C:93 1500 eth0 192.168.1.151 192.168.0.0 0.0.0.0 192.168.1.254 08:00:27:7E:5C:93 1500 eth0 192.168.1.59 192.168.0.0 0.0.0.0 192.168.1.254 08:00:27:7E:5C:93 1500 eth1 10.10.10.51 10.10.0.0 0.0.0.0 192.168.1.254 08:00:27:3C:19:3F 1500 eth1 169.254.72.131 169.254.0.0 0.0.0.0 192.168.1.254 08:00:27:3C:19:3F 1500 Check: Node connectivity of subnet "192.168.0.0" Source Destination Connected? ------------------------------ ------------------------------ ---------------- node3[192.168.1.53] node2[192.168.1.52] yes node3[192.168.1.53] node2[192.168.1.152] yes node3[192.168.1.53] node1[192.168.1.51] yes node3[192.168.1.53] node1[192.168.1.151] yes node3[192.168.1.53] node1[192.168.1.59] yes node2[192.168.1.52] node2[192.168.1.152] yes node2[192.168.1.52] node1[192.168.1.51] yes node2[192.168.1.52] node1[192.168.1.151] yes node2[192.168.1.52] node1[192.168.1.59] yes node2[192.168.1.152] node1[192.168.1.51] yes node2[192.168.1.152] node1[192.168.1.151] yes node2[192.168.1.152] node1[192.168.1.59] yes node1[192.168.1.51] node1[192.168.1.151] yes node1[192.168.1.51] node1[192.168.1.59] yes node1[192.168.1.151] node1[192.168.1.59] yes Result: Node connectivity passed for subnet "192.168.0.0" with node(s) node3,node2,node1 Check: TCP connectivity of subnet "192.168.0.0" Source Destination Connected? ------------------------------ ------------------------------ ---------------- node1:192.168.1.51 node3:192.168.1.53 passed node1:192.168.1.51 node2:192.168.1.52 passed node1:192.168.1.51 node2:192.168.1.152 passed node1:192.168.1.51 node1:192.168.1.151 passed node1:192.168.1.51 node1:192.168.1.59 passed Result: TCP connectivity check passed for subnet "192.168.0.0" Check: Node connectivity of subnet "10.10.0.0" Source Destination Connected? ------------------------------ ------------------------------ ---------------- node3[10.10.10.53] node2[10.10.10.52] yes node3[10.10.10.53] node1[10.10.10.51] yes node2[10.10.10.52] node1[10.10.10.51] yes Result: Node connectivity passed for subnet "10.10.0.0" with node(s) node3,node2,node1 Check: TCP connectivity of subnet "10.10.0.0" Source Destination Connected? ------------------------------ ------------------------------ ---------------- node1:10.10.10.51 node3:10.10.10.53 passed node1:10.10.10.51 node2:10.10.10.52 passed Result: TCP connectivity check passed for subnet "10.10.0.0" Check: Node connectivity of subnet "169.254.0.0" Source Destination Connected? ------------------------------ ------------------------------ ---------------- node2[169.254.95.34] node1[169.254.72.131] yes Result: Node connectivity passed for subnet "169.254.0.0" with node(s) node2,node1 Check: TCP connectivity of subnet "169.254.0.0" Source Destination Connected? ------------------------------ ------------------------------ ---------------- node1:169.254.72.131 node2:169.254.95.34 passed Result: TCP connectivity check passed for subnet "169.254.0.0" Interfaces found on subnet "192.168.0.0" that are likely candidates for VIP are: node3 eth0:192.168.1.53 node2 eth0:192.168.1.52 eth0:192.168.1.152 node1 eth0:192.168.1.51 eth0:192.168.1.151 eth0:192.168.1.59 Interfaces found on subnet "10.10.0.0" that are likely candidates for a private interconnect are: node3 eth1:10.10.10.53 node2 eth1:10.10.10.52 node1 eth1:10.10.10.51 Checking subnet mask consistency... Subnet mask consistency check passed for subnet "192.168.0.0". Subnet mask consistency check passed for subnet "10.10.0.0". Subnet mask consistency check passed for subnet "169.254.0.0". Subnet mask consistency check passed. Result: Node connectivity check passed Checking multicast communication... Checking subnet "192.168.0.0" for multicast communication with multicast group "230.0.1.0"... Check of subnet "192.168.0.0" for multicast communication with multicast group "230.0.1.0" passed. Checking subnet "10.10.0.0" for multicast communication with multicast group "230.0.1.0"... Check of subnet "10.10.0.0" for multicast communication with multicast group "230.0.1.0" passed. Checking subnet "169.254.0.0" for multicast communication with multicast group "230.0.1.0"... Check of subnet "169.254.0.0" for multicast communication with multicast group "230.0.1.0" passed. Check of multicast communication passed. Checking ASMLib configuration. Node Name Status ------------------------------------ ------------------------ node3 passed node2 passed node1 passed Result: Check for ASMLib configuration passed. Check: Total memory Node Name Available Required Status ------------ ------------------------ ------------------------ ---------- node3 1.9599GB (2055112.0KB) 1.5GB (1572864.0KB) passed node2 1.9599GB (2055112.0KB) 1.5GB (1572864.0KB) passed node1 1.9599GB (2055112.0KB) 1.5GB (1572864.0KB) passed Result: Total memory check passed Check: Available memory Node Name Available Required Status ------------ ------------------------ ------------------------ ---------- node3 1.7953GB (1882508.0KB) 50MB (51200.0KB) passed node2 975.3867MB (998796.0KB) 50MB (51200.0KB) passed node1 940.7617MB (963340.0KB) 50MB (51200.0KB) passed Result: Available memory check passed Check: Swap space Node Name Available Required Status ------------ ------------------------ ------------------------ ---------- node3 4GB (4194300.0KB) 2.9399GB (3082668.0KB) passed node2 4GB (4194300.0KB) 2.9399GB (3082668.0KB) passed node1 4GB (4194300.0KB) 2.9399GB (3082668.0KB) passed Result: Swap space check passed Check: Free disk space for "node3:/u01/app/11.2.0/grid,node3:/tmp" Path Node Name Mount point Available Required Status ---------------- ------------ ------------ ------------ ------------ ------------ /u01/app/11.2.0/grid node3 / 71.0596GB 8.5GB passed /tmp node3 / 71.0596GB 8.5GB passed Result: Free disk space check passed for "node3:/u01/app/11.2.0/grid,node3:/tmp" Check: Free disk space for "node2:/u01/app/11.2.0/grid,node2:/tmp" Path Node Name Mount point Available Required Status ---------------- ------------ ------------ ------------ ------------ ------------ /u01/app/11.2.0/grid node2 / 62.9854GB 8.5GB passed /tmp node2 / 62.9854GB 8.5GB passed Result: Free disk space check passed for "node2:/u01/app/11.2.0/grid,node2:/tmp" Check: Free disk space for "node1:/u01/app/11.2.0/grid,node1:/tmp" Path Node Name Mount point Available Required Status ---------------- ------------ ------------ ------------ ------------ ------------ /u01/app/11.2.0/grid node1 / 56.2256GB 8.5GB passed /tmp node1 / 56.2256GB 8.5GB passed Result: Free disk space check passed for "node1:/u01/app/11.2.0/grid,node1:/tmp" Check: User existence for "grid" Node Name Status Comment ------------ ------------------------ ------------------------ node3 passed exists(1100) node2 passed exists(1100) node1 passed exists(1100) Checking for multiple users with UID value 1100 Result: Check for multiple users with UID value 1100 passed Result: User existence check passed for "grid" Check: Group existence for "oinstall" Node Name Status Comment ------------ ------------------------ ------------------------ node3 passed exists node2 passed exists node1 passed exists Result: Group existence check passed for "oinstall" Check: Group existence for "dba" Node Name Status Comment ------------ ------------------------ ------------------------ node3 passed exists node2 passed exists node1 passed exists Result: Group existence check passed for "dba" Check: Membership of user "grid" in group "oinstall" [as Primary] Node Name User Exists Group Exists User in Group Primary Status ---------------- ------------ ------------ ------------ ------------ ------------ node3 yes yes yes yes passed node2 yes yes yes yes passed node1 yes yes yes yes passed Result: Membership check for user "grid" in group "oinstall" [as Primary] passed Check: Membership of user "grid" in group "dba" Node Name User Exists Group Exists User in Group Status ---------------- ------------ ------------ ------------ ---------------- node3 yes yes yes passed node2 yes yes yes passed node1 yes yes yes passed Result: Membership check for user "grid" in group "dba" passed Check: Run level Node Name run level Required Status ------------ ------------------------ ------------------------ ---------- node3 3 3,5 passed node2 3 3,5 passed node1 5 3,5 passed Result: Run level check passed Check: Hard limits for "maximum open file descriptors" Node Name Type Available Required Status ---------------- ------------ ------------ ------------ ---------------- node3 hard 65536 65536 passed node2 hard 65536 65536 passed node1 hard 65536 65536 passed Result: Hard limits check passed for "maximum open file descriptors" Check: Soft limits for "maximum open file descriptors" Node Name Type Available Required Status ---------------- ------------ ------------ ------------ ---------------- node3 soft 1024 1024 passed node2 soft 1024 1024 passed node1 soft 1024 1024 passed Result: Soft limits check passed for "maximum open file descriptors" Check: Hard limits for "maximum user processes" Node Name Type Available Required Status ---------------- ------------ ------------ ------------ ---------------- node3 hard 16384 16384 passed node2 hard 16384 16384 passed node1 hard 16384 16384 passed Result: Hard limits check passed for "maximum user processes" Check: Soft limits for "maximum user processes" Node Name Type Available Required Status ---------------- ------------ ------------ ------------ ---------------- node3 soft 2047 2047 passed node2 soft 2047 2047 passed node1 soft 2047 2047 passed Result: Soft limits check passed for "maximum user processes" Check: System architecture Node Name Available Required Status ------------ ------------------------ ------------------------ ---------- node3 x86_64 x86_64 passed node2 x86_64 x86_64 passed node1 x86_64 x86_64 passed Result: System architecture check passed Check: Kernel version Node Name Available Required Status ------------ ------------------------ ------------------------ ---------- node3 2.6.39-200.24.1.el6uek.x86_64 2.6.32 passed node2 2.6.39-200.24.1.el6uek.x86_64 2.6.32 passed node1 2.6.39-200.24.1.el6uek.x86_64 2.6.32 passed Result: Kernel version check passed Check: Kernel parameter for "semmsl" Node Name Current Configured Required Status Comment ---------------- ------------ ------------ ------------ ------------ ------------ node3 250 250 250 passed node2 250 250 250 passed node1 250 250 250 passed Result: Kernel parameter check passed for "semmsl" Check: Kernel parameter for "semmns" Node Name Current Configured Required Status Comment ---------------- ------------ ------------ ------------ ------------ ------------ node3 32000 32000 32000 passed node2 32000 32000 32000 passed node1 32000 32000 32000 passed Result: Kernel parameter check passed for "semmns" Check: Kernel parameter for "semopm" Node Name Current Configured Required Status Comment ---------------- ------------ ------------ ------------ ------------ ------------ node3 100 100 100 passed node2 100 100 100 passed node1 100 100 100 passed Result: Kernel parameter check passed for "semopm" Check: Kernel parameter for "semmni" Node Name Current Configured Required Status Comment ---------------- ------------ ------------ ------------ ------------ ------------ node3 128 128 128 passed node2 128 128 128 passed node1 128 128 128 passed Result: Kernel parameter check passed for "semmni" Check: Kernel parameter for "shmmax" Node Name Current Configured Required Status Comment ---------------- ------------ ------------ ------------ ------------ ------------ node3 4398046511104 4398046511104 1052217344 passed node2 4398046511104 4398046511104 1052217344 passed node1 4398046511104 4398046511104 1052217344 passed Result: Kernel parameter check passed for "shmmax" Check: Kernel parameter for "shmmni" Node Name Current Configured Required Status Comment ---------------- ------------ ------------ ------------ ------------ ------------ node3 4096 4096 4096 passed node2 4096 4096 4096 passed node1 4096 4096 4096 passed Result: Kernel parameter check passed for "shmmni" Check: Kernel parameter for "shmall" Node Name Current Configured Required Status Comment ---------------- ------------ ------------ ------------ ------------ ------------ node3 1073741824 1073741824 2097152 passed node2 1073741824 1073741824 2097152 passed node1 1073741824 1073741824 2097152 passed Result: Kernel parameter check passed for "shmall" Check: Kernel parameter for "file-max" Node Name Current Configured Required Status Comment ---------------- ------------ ------------ ------------ ------------ ------------ node3 6815744 6815744 6815744 passed node2 6815744 6815744 6815744 passed node1 6815744 6815744 6815744 passed Result: Kernel parameter check passed for "file-max" Check: Kernel parameter for "ip_local_port_range" Node Name Current Configured Required Status Comment ---------------- ------------ ------------ ------------ ------------ ------------ node3 between 9000.0 & 65500.0 between 9000.0 & 65500.0 between 9000.0 & 65500.0 passed node2 between 9000.0 & 65500.0 between 9000.0 & 65500.0 between 9000.0 & 65500.0 passed node1 between 9000.0 & 65500.0 between 9000.0 & 65500.0 between 9000.0 & 65500.0 passed Result: Kernel parameter check passed for "ip_local_port_range" Check: Kernel parameter for "rmem_default" Node Name Current Configured Required Status Comment ---------------- ------------ ------------ ------------ ------------ ------------ node3 262144 262144 262144 passed node2 262144 262144 262144 passed node1 262144 262144 262144 passed Result: Kernel parameter check passed for "rmem_default" Check: Kernel parameter for "rmem_max" Node Name Current Configured Required Status Comment ---------------- ------------ ------------ ------------ ------------ ------------ node3 4194304 4194304 4194304 passed node2 4194304 4194304 4194304 passed node1 4194304 4194304 4194304 passed Result: Kernel parameter check passed for "rmem_max" Check: Kernel parameter for "wmem_default" Node Name Current Configured Required Status Comment ---------------- ------------ ------------ ------------ ------------ ------------ node3 262144 262144 262144 passed node2 262144 262144 262144 passed node1 262144 262144 262144 passed Result: Kernel parameter check passed for "wmem_default" Check: Kernel parameter for "wmem_max" Node Name Current Configured Required Status Comment ---------------- ------------ ------------ ------------ ------------ ------------ node3 1048576 1048576 1048576 passed node2 1048576 1048576 1048576 passed node1 1048576 1048576 1048576 passed Result: Kernel parameter check passed for "wmem_max" Check: Kernel parameter for "aio-max-nr" Node Name Current Configured Required Status Comment ---------------- ------------ ------------ ------------ ------------ ------------ node3 1048576 1048576 1048576 passed node2 1048576 1048576 1048576 passed node1 1048576 1048576 1048576 passed Result: Kernel parameter check passed for "aio-max-nr" Check: Package existence for "binutils" Node Name Available Required Status ------------ ------------------------ ------------------------ ---------- node3 binutils-2.20.51.0.2-5.34.el6 binutils-2.20.51.0.2 passed node2 binutils-2.20.51.0.2-5.34.el6 binutils-2.20.51.0.2 passed node1 binutils-2.20.51.0.2-5.34.el6 binutils-2.20.51.0.2 passed Result: Package existence check passed for "binutils" Check: Package existence for "compat-libcap1" Node Name Available Required Status ------------ ------------------------ ------------------------ ---------- node3 compat-libcap1-1.10-1 compat-libcap1-1.10 passed node2 compat-libcap1-1.10-1 compat-libcap1-1.10 passed node1 compat-libcap1-1.10-1 compat-libcap1-1.10 passed Result: Package existence check passed for "compat-libcap1" Check: Package existence for "compat-libstdc++-33(x86_64)" Node Name Available Required Status ------------ ------------------------ ------------------------ ---------- node3 compat-libstdc++-33(x86_64)-3.2.3-69.el6 compat-libstdc++-33(x86_64)-3.2.3 passed node2 compat-libstdc++-33(x86_64)-3.2.3-69.el6 compat-libstdc++-33(x86_64)-3.2.3 passed node1 compat-libstdc++-33(x86_64)-3.2.3-69.el6 compat-libstdc++-33(x86_64)-3.2.3 passed Result: Package existence check passed for "compat-libstdc++-33(x86_64)" Check: Package existence for "libgcc(x86_64)" Node Name Available Required Status ------------ ------------------------ ------------------------ ---------- node3 libgcc(x86_64)-4.4.6-4.el6 libgcc(x86_64)-4.4.4 passed node2 libgcc(x86_64)-4.4.6-4.el6 libgcc(x86_64)-4.4.4 passed node1 libgcc(x86_64)-4.4.6-4.el6 libgcc(x86_64)-4.4.4 passed Result: Package existence check passed for "libgcc(x86_64)" Check: Package existence for "libstdc++(x86_64)" Node Name Available Required Status ------------ ------------------------ ------------------------ ---------- node3 libstdc++(x86_64)-4.4.6-4.el6 libstdc++(x86_64)-4.4.4 passed node2 libstdc++(x86_64)-4.4.6-4.el6 libstdc++(x86_64)-4.4.4 passed node1 libstdc++(x86_64)-4.4.6-4.el6 libstdc++(x86_64)-4.4.4 passed Result: Package existence check passed for "libstdc++(x86_64)" Check: Package existence for "libstdc++-devel(x86_64)" Node Name Available Required Status ------------ ------------------------ ------------------------ ---------- node3 libstdc++-devel(x86_64)-4.4.6-4.el6 libstdc++-devel(x86_64)-4.4.4 passed node2 libstdc++-devel(x86_64)-4.4.6-4.el6 libstdc++-devel(x86_64)-4.4.4 passed node1 libstdc++-devel(x86_64)-4.4.6-4.el6 libstdc++-devel(x86_64)-4.4.4 passed Result: Package existence check passed for "libstdc++-devel(x86_64)" Check: Package existence for "sysstat" Node Name Available Required Status ------------ ------------------------ ------------------------ ---------- node3 sysstat-9.0.4-20.el6 sysstat-9.0.4 passed node2 sysstat-9.0.4-20.el6 sysstat-9.0.4 passed node1 sysstat-9.0.4-20.el6 sysstat-9.0.4 passed Result: Package existence check passed for "sysstat" Check: Package existence for "gcc" Node Name Available Required Status ------------ ------------------------ ------------------------ ---------- node3 gcc-4.4.6-4.el6 gcc-4.4.4 passed node2 gcc-4.4.6-4.el6 gcc-4.4.4 passed node1 gcc-4.4.6-4.el6 gcc-4.4.4 passed Result: Package existence check passed for "gcc" Check: Package existence for "gcc-c++" Node Name Available Required Status ------------ ------------------------ ------------------------ ---------- node3 gcc-c++-4.4.6-4.el6 gcc-c++-4.4.4 passed node2 gcc-c++-4.4.6-4.el6 gcc-c++-4.4.4 passed node1 gcc-c++-4.4.6-4.el6 gcc-c++-4.4.4 passed Result: Package existence check passed for "gcc-c++" Check: Package existence for "ksh" Node Name Available Required Status ------------ ------------------------ ------------------------ ---------- node3 ksh-20100621-16.el6 ksh-20100621 passed node2 ksh-20100621-16.el6 ksh-20100621 passed node1 ksh-20100621-16.el6 ksh-20100621 passed Result: Package existence check passed for "ksh" Check: Package existence for "make" Node Name Available Required Status ------------ ------------------------ ------------------------ ---------- node3 make-3.81-20.el6 make-3.81 passed node2 make-3.81-20.el6 make-3.81 passed node1 make-3.81-20.el6 make-3.81 passed Result: Package existence check passed for "make" Check: Package existence for "glibc(x86_64)" Node Name Available Required Status ------------ ------------------------ ------------------------ ---------- node3 glibc(x86_64)-2.12-1.80.el6_3.6 glibc(x86_64)-2.12 passed node2 glibc(x86_64)-2.12-1.80.el6_3.6 glibc(x86_64)-2.12 passed node1 glibc(x86_64)-2.12-1.80.el6_3.6 glibc(x86_64)-2.12 passed Result: Package existence check passed for "glibc(x86_64)" Check: Package existence for "glibc-devel(x86_64)" Node Name Available Required Status ------------ ------------------------ ------------------------ ---------- node3 glibc-devel(x86_64)-2.12-1.80.el6_3.6 glibc-devel(x86_64)-2.12 passed node2 glibc-devel(x86_64)-2.12-1.80.el6_3.6 glibc-devel(x86_64)-2.12 passed node1 glibc-devel(x86_64)-2.12-1.80.el6_3.6 glibc-devel(x86_64)-2.12 passed Result: Package existence check passed for "glibc-devel(x86_64)" Check: Package existence for "libaio(x86_64)" Node Name Available Required Status ------------ ------------------------ ------------------------ ---------- node3 libaio(x86_64)-0.3.107-10.el6 libaio(x86_64)-0.3.107 passed node2 libaio(x86_64)-0.3.107-10.el6 libaio(x86_64)-0.3.107 passed node1 libaio(x86_64)-0.3.107-10.el6 libaio(x86_64)-0.3.107 passed Result: Package existence check passed for "libaio(x86_64)" Check: Package existence for "libaio-devel(x86_64)" Node Name Available Required Status ------------ ------------------------ ------------------------ ---------- node3 libaio-devel(x86_64)-0.3.107-10.el6 libaio-devel(x86_64)-0.3.107 passed node2 libaio-devel(x86_64)-0.3.107-10.el6 libaio-devel(x86_64)-0.3.107 passed node1 libaio-devel(x86_64)-0.3.107-10.el6 libaio-devel(x86_64)-0.3.107 passed Result: Package existence check passed for "libaio-devel(x86_64)" Checking for multiple users with UID value 0 Result: Check for multiple users with UID value 0 passed Check: Current group ID Result: Current group ID check passed Starting check for consistency of primary group of root user Node Name Status ------------------------------------ ------------------------ node3 passed node2 passed node1 passed Check for consistency of root user's primary group passed Starting Clock synchronization checks using Network Time Protocol(NTP)... NTP Configuration file check started... Network Time Protocol(NTP) configuration file not found on any of the nodes. Oracle Cluster Time Synchronization Service(CTSS) can be used instead of NTP for time synchronization on the cluster nodes No NTP Daemons or Services were found to be running Result: Clock synchronization check using Network Time Protocol(NTP) passed Checking Core file name pattern consistency... Core file name pattern consistency check passed. Checking to make sure user "grid" is not in "root" group Node Name Status Comment ------------ ------------------------ ------------------------ node3 passed does not exist node2 passed does not exist node1 passed does not exist Result: User "grid" is not part of "root" group. Check passed Check default user file creation mask Node Name Available Required Comment ------------ ------------------------ ------------------------ ---------- node3 0022 0022 passed node2 0022 0022 passed node1 0022 0022 passed Result: Default user file creation mask check passed Checking consistency of file "/etc/resolv.conf" across nodes Checking the file "/etc/resolv.conf" to make sure only one of domain and search entries is defined File "/etc/resolv.conf" does not have both domain and search entries defined Checking if domain entry in file "/etc/resolv.conf" is consistent across the nodes... domain entry in file "/etc/resolv.conf" is consistent across nodes Checking if search entry in file "/etc/resolv.conf" is consistent across the nodes... search entry in file "/etc/resolv.conf" is consistent across nodes Checking DNS response time for an unreachable node Node Name Status ------------------------------------ ------------------------ node3 passed node2 passed node1 passed The DNS response time for an unreachable node is within acceptable limit on all nodes File "/etc/resolv.conf" is consistent across nodes Check: Time zone consistency Result: Time zone consistency check passed Pre-check for cluster services setup was successful.
Extend Clusterware
The clusterware software will be extended to the new node.
Run "addNode.sh"
From an existing node, run "addNode.sh" to extend the clusterware to the new node "node3":
[grid@node1 ~]$ export IGNORE_PREADDNODE_CHECKS=Y
[grid@node1 ~]$ cd $ORACLE_HOME/oui/bin
[grid@node1 bin]$ ./addNode.sh -silent "CLUSTER_NEW_NODES={node3}" "CLUSTER_NEW_VIRTUAL_HOSTNAMES={node3-vip}" "CLUSTER_NEW_PRIVATE_NODE_NAMES={node3-priv}"
Starting Oracle Universal Installer...
Checking swap space: must be greater than 500 MB. Actual 5667 MB Passed
Oracle Universal Installer, Version 11.2.0.3.0 Production
Copyright (C) 1999, 2011, Oracle. All rights reserved.
Performing tests to see whether nodes node2,node3 are available
............................................................... 100% Done.
.
-----------------------------------------------------------------------------
Cluster Node Addition Summary
Global Settings
Source: /u01/app/11.2.0/grid
New Nodes
Space Requirements
New Nodes
node3
/: Required 4.31GB : Available 82.72GB
Installed Products
Product Names
Oracle Grid Infrastructure 11.2.0.3.0
Sun JDK 1.5.0.30.03
Installer SDK Component 11.2.0.3.0
Oracle One-Off Patch Installer 11.2.0.1.7
Oracle Universal Installer 11.2.0.3.0
Oracle USM Deconfiguration 11.2.0.3.0
Oracle Configuration Manager Deconfiguration 10.3.1.0.0
Enterprise Manager Common Core Files 10.2.0.4.4
Oracle DBCA Deconfiguration 11.2.0.3.0
Oracle RAC Deconfiguration 11.2.0.3.0
Oracle Quality of Service Management (Server) 11.2.0.3.0
Installation Plugin Files 11.2.0.3.0
Universal Storage Manager Files 11.2.0.3.0
Oracle Text Required Support Files 11.2.0.3.0
Automatic Storage Management Assistant 11.2.0.3.0
Oracle Database 11g Multimedia Files 11.2.0.3.0
Oracle Multimedia Java Advanced Imaging 11.2.0.3.0
Oracle Globalization Support 11.2.0.3.0
Oracle Multimedia Locator RDBMS Files 11.2.0.3.0
Oracle Core Required Support Files 11.2.0.3.0
Bali Share 1.1.18.0.0
Oracle Database Deconfiguration 11.2.0.3.0
Oracle Quality of Service Management (Client) 11.2.0.3.0
Expat libraries 2.0.1.0.1
Oracle Containers for Java 11.2.0.3.0
Perl Modules 5.10.0.0.1
Secure Socket Layer 11.2.0.3.0
Oracle JDBC/OCI Instant Client 11.2.0.3.0
Oracle Multimedia Client Option 11.2.0.3.0
LDAP Required Support Files 11.2.0.3.0
Character Set Migration Utility 11.2.0.3.0
Perl Interpreter 5.10.0.0.2
PL/SQL Embedded Gateway 11.2.0.3.0
OLAP SQL Scripts 11.2.0.3.0
Database SQL Scripts 11.2.0.3.0
Oracle Extended Windowing Toolkit 3.4.47.0.0
SSL Required Support Files for InstantClient 11.2.0.3.0
SQL*Plus Files for Instant Client 11.2.0.3.0
Oracle Net Required Support Files 11.2.0.3.0
Oracle Database User Interface 2.2.13.0.0
RDBMS Required Support Files for Instant Client 11.2.0.3.0
RDBMS Required Support Files Runtime 11.2.0.3.0
XML Parser for Java 11.2.0.3.0
Oracle Security Developer Tools 11.2.0.3.0
Oracle Wallet Manager 11.2.0.3.0
Enterprise Manager plugin Common Files 11.2.0.3.0
Platform Required Support Files 11.2.0.3.0
Oracle JFC Extended Windowing Toolkit 4.2.36.0.0
RDBMS Required Support Files 11.2.0.3.0
Oracle Ice Browser 5.2.3.6.0
Oracle Help For Java 4.2.9.0.0
Enterprise Manager Common Files 10.2.0.4.3
Deinstallation Tool 11.2.0.3.0
Oracle Java Client 11.2.0.3.0
Cluster Verification Utility Files 11.2.0.3.0
Oracle Notification Service (eONS) 11.2.0.3.0
Oracle LDAP administration 11.2.0.3.0
Cluster Verification Utility Common Files 11.2.0.3.0
Oracle Clusterware RDBMS Files 11.2.0.3.0
Oracle Locale Builder 11.2.0.3.0
Oracle Globalization Support 11.2.0.3.0
Buildtools Common Files 11.2.0.3.0
Oracle RAC Required Support Files-HAS 11.2.0.3.0
SQL*Plus Required Support Files 11.2.0.3.0
XDK Required Support Files 11.2.0.3.0
Agent Required Support Files 10.2.0.4.3
Parser Generator Required Support Files 11.2.0.3.0
Precompiler Required Support Files 11.2.0.3.0
Installation Common Files 11.2.0.3.0
Required Support Files 11.2.0.3.0
Oracle JDBC/THIN Interfaces 11.2.0.3.0
Oracle Multimedia Locator 11.2.0.3.0
Oracle Multimedia 11.2.0.3.0
HAS Common Files 11.2.0.3.0
Assistant Common Files 11.2.0.3.0
PL/SQL 11.2.0.3.0
HAS Files for DB 11.2.0.3.0
Oracle Recovery Manager 11.2.0.3.0
Oracle Database Utilities 11.2.0.3.0
Oracle Notification Service 11.2.0.3.0
SQL*Plus 11.2.0.3.0
Oracle Netca Client 11.2.0.3.0
Oracle Net 11.2.0.3.0
Oracle JVM 11.2.0.3.0
Oracle Internet Directory Client 11.2.0.3.0
Oracle Net Listener 11.2.0.3.0
Cluster Ready Services Files 11.2.0.3.0
Oracle Database 11g 11.2.0.3.0
-----------------------------------------------------------------------------
Instantiating scripts for add node (Friday, January 4, 2013 11:19:46 AM CST)
. 1% Done.
Instantiation of add node scripts complete
Copying to remote nodes (Friday, January 4, 2013 11:19:48 AM CST)
............................................................................................... 96% Done.
Home copied to new nodes
Saving inventory on nodes (Friday, January 4, 2013 11:25:19 AM CST)
. 100% Done.
Save inventory complete
WARNING:A new inventory has been created on one or more nodes in this session. However, it has not yet been registered as the central inventory of this system.
To register the new inventory please run the script at '/u01/app/oraInventory/orainstRoot.sh' with root privileges on nodes 'node3'.
If you do not register the inventory, you may not be able to update or patch the products you installed.
The following configuration scripts need to be executed as the "root" user in each new cluster node. Each script in the list below is followed by a list of nodes.
/u01/app/oraInventory/orainstRoot.sh #On nodes node3
/u01/app/11.2.0/grid/root.sh #On nodes node3
To execute the configuration scripts:
1. Open a terminal window
2. Log in as "root"
3. Run the scripts in each cluster node
The Cluster Node Addition of /u01/app/11.2.0/grid was successful.
Please check '/tmp/silentInstall.log' for more details.
[grid@node1 bin]$
Run the "root.sh" commands on the new node as directed
[root@node3 ~]# /u01/app/oraInventory/orainstRoot.sh Creating the Oracle inventory pointer file (/etc/oraInst.loc) Changing permissions of /u01/app/oraInventory. Adding read,write permissions for group. Removing read,write,execute permissions for world. Changing groupname of /u01/app/oraInventory to oinstall. The execution of the script is complete. [root@node3 ~]#
[root@node3 ~]# /u01/app/11.2.0/grid/root.sh
Performing root user operation for Oracle 11g
The following environment variables are set as:
ORACLE_OWNER= grid
ORACLE_HOME= /u01/app/11.2.0/grid
Enter the full pathname of the local bin directory: [/usr/local/bin]:
Copying dbhome to /usr/local/bin ...
Copying oraenv to /usr/local/bin ...
Copying coraenv to /usr/local/bin ...
Creating /etc/oratab file...
Entries will be added to the /etc/oratab file as needed by
Database Configuration Assistant when a database is created
Finished running generic part of root script.
Now product-specific root actions will be performed.
Using configuration parameter file: /u01/app/11.2.0/grid/crs/install/crsconfig_params
Creating trace directory
User ignored Prerequisites during installation
OLR initialization - successful
Adding Clusterware entries to upstart
CRS-4402: The CSS daemon was started in exclusive mode but found an active CSS daemon on node node1, number 1, and is terminating
An active cluster was found during exclusive startup, restarting to join the cluster
clscfg: EXISTING configuration version 5 detected.
clscfg: version 5 is 11g Release 2.
Successfully accumulated necessary OCR keys.
Creating OCR keys for user 'root', privgrp 'root'..
Operation successful.
Configure Oracle Grid Infrastructure for a Cluster ... succeeded
If successful, clusterware daemons, the listener, the ASM instance, etc. should be started by the "root.sh" script
[grid@node3 ~]$ ps -ef|grep grid|grep -v grep root 1872 1785 0 Jan04 pts/0 00:00:00 su - grid grid 1873 1872 0 Jan04 pts/0 00:00:00 -bash root 11348 1 1 09:16 ? 00:00:11 /u01/app/11.2.0/grid/bin/ohasd.bin reboot grid 11961 1 0 09:18 ? 00:00:02 /u01/app/11.2.0/grid/bin/oraagent.bin grid 11973 1 0 09:18 ? 00:00:00 /u01/app/11.2.0/grid/bin/mdnsd.bin grid 11986 1 0 09:18 ? 00:00:00 /u01/app/11.2.0/grid/bin/gpnpd.bin root 12003 1 0 09:18 ? 00:00:00 /u01/app/11.2.0/grid/bin/cssdmonitor grid 12005 1 0 09:18 ? 00:00:02 /u01/app/11.2.0/grid/bin/gipcd.bin root 12026 1 0 09:18 ? 00:00:00 /u01/app/11.2.0/grid/bin/cssdagent grid 12051 1 1 09:18 ? 00:00:08 /u01/app/11.2.0/grid/bin/ocssd.bin root 12104 1 0 09:19 ? 00:00:05 /u01/app/11.2.0/grid/bin/orarootagent.bin root 12116 1 0 09:19 ? 00:00:02 /u01/app/11.2.0/grid/bin/octssd.bin root 12138 1 1 09:19 ? 00:00:09 /u01/app/11.2.0/grid/bin/osysmond.bin grid 12232 1 0 09:20 ? 00:00:00 asm_pmon_+ASM3 grid 12234 1 0 09:20 ? 00:00:00 asm_psp0_+ASM3 grid 12236 1 8 09:20 ? 00:00:47 asm_vktm_+ASM3 grid 12240 1 0 09:20 ? 00:00:00 asm_gen0_+ASM3 grid 12242 1 0 09:20 ? 00:00:00 asm_diag_+ASM3 grid 12244 1 0 09:20 ? 00:00:00 asm_ping_+ASM3 grid 12246 1 0 09:20 ? 00:00:02 asm_dia0_+ASM3 grid 12248 1 0 09:20 ? 00:00:02 asm_lmon_+ASM3 grid 12250 1 0 09:20 ? 00:00:01 asm_lmd0_+ASM3 grid 12252 1 0 09:20 ? 00:00:03 asm_lms0_+ASM3 grid 12256 1 0 09:20 ? 00:00:00 asm_lmhb_+ASM3 grid 12258 1 0 09:20 ? 00:00:00 asm_mman_+ASM3 grid 12260 1 0 09:20 ? 00:00:00 asm_dbw0_+ASM3 grid 12262 1 0 09:20 ? 00:00:00 asm_lgwr_+ASM3 grid 12264 1 0 09:20 ? 00:00:00 asm_ckpt_+ASM3 grid 12266 1 0 09:20 ? 00:00:00 asm_smon_+ASM3 grid 12268 1 0 09:20 ? 00:00:00 asm_rbal_+ASM3 grid 12270 1 0 09:20 ? 00:00:00 asm_gmon_+ASM3 grid 12272 1 0 09:20 ? 00:00:00 asm_mmon_+ASM3 grid 12274 1 0 09:20 ? 00:00:00 asm_mmnl_+ASM3 grid 12282 1 0 09:20 ? 00:00:00 asm_lck0_+ASM3 grid 12284 1 0 09:20 ? 00:00:00 oracle+ASM3 (DESCRIPTION=(LOCAL=YES)(ADDRESS=(PROTOCOL=beq))) root 12292 1 1 09:20 ? 00:00:07 /u01/app/11.2.0/grid/bin/crsd.bin reboot grid 12311 1 0 09:20 ? 00:00:02 /u01/app/11.2.0/grid/bin/evmd.bin grid 12314 1 0 09:20 ? 00:00:00 oracle+ASM3_ocr (DESCRIPTION=(LOCAL=YES)(ADDRESS=(PROTOCOL=beq))) grid 12320 1 0 09:20 ? 00:00:00 asm_asmb_+ASM3 grid 12322 1 0 09:20 ? 00:00:00 oracle+ASM3_asmb_+asm3 (DESCRIPTION=(LOCAL=YES)(ADDRESS=(PROTOCOL=beq))) grid 12413 12311 0 09:21 ? 00:00:00 /u01/app/11.2.0/grid/bin/evmlogger.bin -o /u01/app/11.2.0/grid/evm/log/evmlogger.info -l /u01/app/11.2.0/grid/evm/log/evmlogger.log grid 12456 1 0 09:21 ? 00:00:01 /u01/app/11.2.0/grid/bin/oraagent.bin root 12460 1 0 09:21 ? 00:00:02 /u01/app/11.2.0/grid/bin/orarootagent.bin grid 12493 1 0 09:21 ? 00:00:00 oracle+ASM3 (DESCRIPTION=(LOCAL=YES)(ADDRESS=(PROTOCOL=beq))) grid 12502 1 0 09:21 ? 00:00:00 oracle+ASM3 (DESCRIPTION=(LOCAL=YES)(ADDRESS=(PROTOCOL=beq))) grid 12505 1 0 09:21 ? 00:00:00 oracle+ASM3 (DESCRIPTION=(LOCAL=YES)(ADDRESS=(PROTOCOL=beq))) grid 12551 1 0 09:21 ? 00:00:00 /u01/app/11.2.0/grid/opmn/bin/ons -d grid 12552 12551 0 09:21 ? 00:00:00 /u01/app/11.2.0/grid/opmn/bin/ons -d grid 13032 1 0 09:21 ? 00:00:00 /u01/app/11.2.0/grid/bin/tnslsnr LISTENER -inherit grid 13288 1873 0 09:29 pts/0 00:00:00 ps -ef
[grid@node3 ~]$ crs_stat -t -v -c node3 Name Type R/RA F/FT Target State Host ---------------------------------------------------------------------- ora....SM3.asm application 0/5 0/0 ONLINE ONLINE node3 ora....E3.lsnr application 0/5 0/0 ONLINE ONLINE node3 ora.node3.ons application 0/3 0/0 ONLINE ONLINE node3 ora.node3.vip ora....t1.type 0/0 0/0 ONLINE ONLINE node3
[grid@node3 ~]$ crs_stat -t -v Name Type R/RA F/FT Target State Host ---------------------------------------------------------------------- ora.CRS.dg ora....up.type 0/5 0/ ONLINE ONLINE node1 ora.DATADG.dg ora....up.type 0/5 0/ ONLINE ONLINE node1 ora.FRADG.dg ora....up.type 0/5 0/ ONLINE ONLINE node1 ora....ER.lsnr ora....er.type 0/5 0/ ONLINE ONLINE node1 ora....N1.lsnr ora....er.type 0/5 0/0 ONLINE ONLINE node1 ora.asm ora.asm.type 0/5 0/ ONLINE ONLINE node1 ora.cvu ora.cvu.type 0/5 0/0 ONLINE ONLINE node1 ora.gsd ora.gsd.type 0/5 0/ OFFLINE OFFLINE ora....network ora....rk.type 0/5 0/ ONLINE ONLINE node1 ora....SM1.asm application 0/5 0/0 ONLINE ONLINE node1 ora....E1.lsnr application 0/5 0/0 ONLINE ONLINE node1 ora.node1.gsd application 0/5 0/0 OFFLINE OFFLINE ora.node1.ons application 0/3 0/0 ONLINE ONLINE node1 ora.node1.vip ora....t1.type 0/0 0/0 ONLINE ONLINE node1 ora....SM2.asm application 0/5 0/0 ONLINE ONLINE node2 ora....E2.lsnr application 0/5 0/0 ONLINE ONLINE node2 ora.node2.gsd application 0/5 0/0 OFFLINE OFFLINE ora.node2.ons application 0/3 0/0 ONLINE ONLINE node2 ora.node2.vip ora....t1.type 0/0 0/0 ONLINE ONLINE node2 ora....SM3.asm application 0/5 0/0 ONLINE ONLINE node3 ora....E3.lsnr application 0/5 0/0 ONLINE ONLINE node3 ora.node3.gsd application 0/5 0/0 OFFLINE OFFLINE ora.node3.ons application 0/3 0/0 ONLINE ONLINE node3 ora.node3.vip ora....t1.type 0/0 0/0 ONLINE ONLINE node3 ora.oc4j ora.oc4j.type 0/1 0/2 ONLINE ONLINE node1 ora.ons ora.ons.type 0/3 0/ ONLINE ONLINE node1 ora.scan1.vip ora....ip.type 0/0 0/0 ONLINE ONLINE node1 ora.zhongwc.db ora....se.type 0/2 0/1 ONLINE ONLINE node1
[grid@node3 ~]$ crsctl stat res -t
--------------------------------------------------------------------------------
NAME TARGET STATE SERVER STATE_DETAILS
--------------------------------------------------------------------------------
Local Resources
--------------------------------------------------------------------------------
ora.CRS.dg
ONLINE ONLINE node1
ONLINE ONLINE node2
ONLINE ONLINE node3
ora.DATADG.dg
ONLINE ONLINE node1
ONLINE ONLINE node2
ONLINE ONLINE node3
ora.FRADG.dg
ONLINE ONLINE node1
ONLINE ONLINE node2
ONLINE ONLINE node3
ora.LISTENER.lsnr
ONLINE ONLINE node1
ONLINE ONLINE node2
ONLINE ONLINE node3
ora.asm
ONLINE ONLINE node1 Started
ONLINE ONLINE node2 Started
ONLINE ONLINE node3 Started
ora.gsd
OFFLINE OFFLINE node1
OFFLINE OFFLINE node2
OFFLINE OFFLINE node3
ora.net1.network
ONLINE ONLINE node1
ONLINE ONLINE node2
ONLINE ONLINE node3
ora.ons
ONLINE ONLINE node1
ONLINE ONLINE node2
ONLINE ONLINE node3
--------------------------------------------------------------------------------
Cluster Resources
--------------------------------------------------------------------------------
ora.LISTENER_SCAN1.lsnr
1 ONLINE ONLINE node1
ora.cvu
1 ONLINE ONLINE node1
ora.node1.vip
1 ONLINE ONLINE node1
ora.node2.vip
1 ONLINE ONLINE node2
ora.node3.vip
1 ONLINE ONLINE node3
ora.oc4j
1 ONLINE ONLINE node1
ora.scan1.vip
1 ONLINE ONLINE node1
ora.zhongwc.db
1 ONLINE ONLINE node1 Open
2 ONLINE ONLINE node2 Open
Extend Oracle Database Software
From an existing node – as the database software owner – run the following command to extend the Oracle database software to the new node "node3"
[oracle@node1 ~]$ cd $ORACLE_HOME/oui/bin
[oracle@node1 bin]$ ./addNode.sh -silent "CLUSTER_NEW_NODES={node3}"Performing pre-checks for node addition
Checking node reachability...
Node reachability check passed from node "node1"Checking user equivalence...
User equivalence check passed for user "oracle"WARNING:
Node "node3" already appears to be part of clusterPre-check for node addition was successful.
Starting Oracle Universal Installer...Checking swap space: must be greater than 500 MB. Actual 3919 MB Passed
Oracle Universal Installer, Version 11.2.0.3.0 Production
Copyright (C) 1999, 2011, Oracle. All rights reserved.Performing tests to see whether nodes node2,node3 are available
............................................................... 100% Done....
-----------------------------------------------------------------------------
Cluster Node Addition Summary
Global Settings
Source: /u01/app/oracle/product/11.2.0/dbhome_1
New Nodes
Space Requirements
New Nodes
node3
/: Required 4.13GB : Available 63.03GB
Installed Products
Product Names
Oracle Database 11g 11.2.0.3.0
Sun JDK 1.5.0.30.03
Installer SDK Component 11.2.0.3.0
Oracle One-Off Patch Installer 11.2.0.1.7
Oracle Universal Installer 11.2.0.3.0
Oracle USM Deconfiguration 11.2.0.3.0
Oracle Configuration Manager Deconfiguration 10.3.1.0.0
Oracle DBCA Deconfiguration 11.2.0.3.0
Oracle RAC Deconfiguration 11.2.0.3.0
Oracle Database Deconfiguration 11.2.0.3.0
Oracle Configuration Manager Client 10.3.2.1.0
Oracle Configuration Manager 10.3.5.0.1
Oracle ODBC Driverfor Instant Client 11.2.0.3.0
LDAP Required Support Files 11.2.0.3.0
SSL Required Support Files for InstantClient 11.2.0.3.0
Bali Share 1.1.18.0.0
Oracle Extended Windowing Toolkit 3.4.47.0.0
Oracle JFC Extended Windowing Toolkit 4.2.36.0.0
Oracle Real Application Testing 11.2.0.3.0
Oracle Database Vault J2EE Application 11.2.0.3.0
Oracle Label Security 11.2.0.3.0
Oracle Data Mining RDBMS Files 11.2.0.3.0
Oracle OLAP RDBMS Files 11.2.0.3.0
Oracle OLAP API 11.2.0.3.0
Platform Required Support Files 11.2.0.3.0
Oracle Database Vault option 11.2.0.3.0
Oracle RAC Required Support Files-HAS 11.2.0.3.0
SQL*Plus Required Support Files 11.2.0.3.0
Oracle Display Fonts 9.0.2.0.0
Oracle Ice Browser 5.2.3.6.0
Oracle JDBC Server Support Package 11.2.0.3.0
Oracle SQL Developer 11.2.0.3.0
Oracle Application Express 11.2.0.3.0
XDK Required Support Files 11.2.0.3.0
RDBMS Required Support Files for Instant Client 11.2.0.3.0
SQLJ Runtime 11.2.0.3.0
Database Workspace Manager 11.2.0.3.0
RDBMS Required Support Files Runtime 11.2.0.3.0
Oracle Globalization Support 11.2.0.3.0
Exadata Storage Server 11.2.0.1.0
Provisioning Advisor Framework 10.2.0.4.3
Enterprise Manager Database Plugin -- Repository Support 11.2.0.3.0
Enterprise Manager Repository Core Files 10.2.0.4.4
Enterprise Manager Database Plugin -- Agent Support 11.2.0.3.0
Enterprise Manager Grid Control Core Files 10.2.0.4.4
Enterprise Manager Common Core Files 10.2.0.4.4
Enterprise Manager Agent Core Files 10.2.0.4.4
RDBMS Required Support Files 11.2.0.3.0
regexp 2.1.9.0.0
Agent Required Support Files 10.2.0.4.3
Oracle 11g Warehouse Builder Required Files 11.2.0.3.0
Oracle Notification Service (eONS) 11.2.0.3.0
Oracle Text Required Support Files 11.2.0.3.0
Parser Generator Required Support Files 11.2.0.3.0
Oracle Database 11g Multimedia Files 11.2.0.3.0
Oracle Multimedia Java Advanced Imaging 11.2.0.3.0
Oracle Multimedia Annotator 11.2.0.3.0
Oracle JDBC/OCI Instant Client 11.2.0.3.0
Oracle Multimedia Locator RDBMS Files 11.2.0.3.0
Precompiler Required Support Files 11.2.0.3.0
Oracle Core Required Support Files 11.2.0.3.0
Sample Schema Data 11.2.0.3.0
Oracle Starter Database 11.2.0.3.0
Oracle Message Gateway Common Files 11.2.0.3.0
Oracle XML Query 11.2.0.3.0
XML Parser for Oracle JVM 11.2.0.3.0
Oracle Help For Java 4.2.9.0.0
Installation Plugin Files 11.2.0.3.0
Enterprise Manager Common Files 10.2.0.4.3
Expat libraries 2.0.1.0.1
Deinstallation Tool 11.2.0.3.0
Oracle Quality of Service Management (Client) 11.2.0.3.0
Perl Modules 5.10.0.0.1
JAccelerator (COMPANION) 11.2.0.3.0
Oracle Containers for Java 11.2.0.3.0
Perl Interpreter 5.10.0.0.2
Oracle Net Required Support Files 11.2.0.3.0
Secure Socket Layer 11.2.0.3.0
Oracle Universal Connection Pool 11.2.0.3.0
Oracle JDBC/THIN Interfaces 11.2.0.3.0
Oracle Multimedia Client Option 11.2.0.3.0
Oracle Java Client 11.2.0.3.0
Character Set Migration Utility 11.2.0.3.0
Oracle Code Editor 1.2.1.0.0I
PL/SQL Embedded Gateway 11.2.0.3.0
OLAP SQL Scripts 11.2.0.3.0
Database SQL Scripts 11.2.0.3.0
Oracle Locale Builder 11.2.0.3.0
Oracle Globalization Support 11.2.0.3.0
SQL*Plus Files for Instant Client 11.2.0.3.0
Required Support Files 11.2.0.3.0
Oracle Database User Interface 2.2.13.0.0
Oracle ODBC Driver 11.2.0.3.0
Oracle Notification Service 11.2.0.3.0
XML Parser for Java 11.2.0.3.0
Oracle Security Developer Tools 11.2.0.3.0
Oracle Wallet Manager 11.2.0.3.0
Cluster Verification Utility Common Files 11.2.0.3.0
Oracle Clusterware RDBMS Files 11.2.0.3.0
Oracle UIX 2.2.24.6.0
Enterprise Manager plugin Common Files 11.2.0.3.0
HAS Common Files 11.2.0.3.0
Precompiler Common Files 11.2.0.3.0
Installation Common Files 11.2.0.3.0
Oracle Help for the Web 2.0.14.0.0
Oracle LDAP administration 11.2.0.3.0
Buildtools Common Files 11.2.0.3.0
Assistant Common Files 11.2.0.3.0
Oracle Recovery Manager 11.2.0.3.0
PL/SQL 11.2.0.3.0
Generic Connectivity Common Files 11.2.0.3.0
Oracle Database Gateway for ODBC 11.2.0.3.0
Oracle Programmer 11.2.0.3.0
Oracle Database Utilities 11.2.0.3.0
Enterprise Manager Agent 10.2.0.4.