Environment:
Each node requires at least Two 2 NIC cards one for Public IP and other for Private interconnect.
Node1 : racnode1.ukatru.com
Public IP Address : 192.168.2.52(racnode1.ukatru.com)
Virtual IP Address : 192.168.2.54(racnode1-vip.ukatru.com)
Node2:racnode2.ukatru.com
Public IP Address : 192.168.2.53(racnode2.ukatru.com)
Virtual IP Address : 192.168.2.55(racnode2-vip.ukatru.com)
We need 15GB disk on both nodes to install oracle Cluster software,ASM home and Database home.
Set Kernel Parameters
Add the following lines to the /etc/sysctl.conf file:# Controls the maximum number of shared memory segments, in pages
kernel.shmall = 1073741824
# For 11g, recommended value for file-max is 6815744
fs.file-max = 6815744
# For 10g, uncomment 'fs.file-max 327679', comment other entries for this parameter and re-run sysctl -p
# fs.file-max:327679
kernel.msgmni = 2878
kernel.sem = 250 32000 100 142
kernel.shmmni = 4096
net.core.rmem_default = 262144
# For 11g, recommended value for net.core.rmem_max is 4194304
net.core.rmem_max = 4194304
# For 10g, uncomment 'net.core.rmem_max 2097152', comment other entries for this parameter and re-run sysctl -p
# net.core.rmem_max=2097152
net.core.wmem_default = 262144
# For 11g, recommended value for wmem_max is 1048576
net.core.wmem_max = 1048576
# For 10g, uncomment 'net.core.wmem_max 262144', comment other entries for this parameter and re-run sysctl -p
# net.core.wmem_max:262144
fs.aio-max-nr = 3145728
# For 11g, recommended value for ip_local_port_range is 9000 65500
net.ipv4.ip_local_port_range = 9000 65500
# For 10g, uncomment 'net.ipv4.ip_local_port_range 1024 65000', comment other entries for this parameter and re-run sysctl -p
# net.ipv4.ip_local_port_range:1024 65000
# Added min_free_kbytes 50MB to avoid OOM killer on EL4/EL5
vm.min_free_kbytes = 51200
[root@oral01 ~]# sysctl -p
Add the following lines to the /etc/security/limits.conf file:
oracle soft nofile 131072
oracle hard nofile 131072
oracle soft nproc 131072
oracle hard nproc 131072
oracle soft core unlimited
oracle hard core unlimited
oracle soft memlock 50000000
oracle hard memlock 50000000
grid soft nproc 2047
grid hard nproc 16384
grid soft nofile 1024
grid hard nofile 65536
Configure the iSCSI (Initiator) Service:
rpm -Uvh iscsi-initiator-utils-6.2.0.871-0.10.el5.x86_64.rpm
warning: iscsi-initiator-utils-6.2.0.871-0.10.el5.x86_64.rpm: Header V3 DSA signature: NOKEY, key ID 1e5e0159
Preparing... ########################################### [100%]
1:iscsi-initiator-utils ########################################### [100%]
root@racnode1 Server]# service iscsid start
Turning off network shutdown. Starting iSCSI daemon: [ OK ]
[ OK ]
[root@racnode1 Server]# chkconfig iscsid on
[root@racnode1 Server]# chkconfig iscsi on
[root@racnode2 Server]# service iscsid start
Turning off network shutdown. Starting iSCSI daemon: [ OK ]
[ OK ]
[root@racnode2 Server]# chkconfig iscsid on
[root@racnode2 Server]# chkconfig iscsi on
Configure oracleasm :
[root@racnode1 tmp]# /etc/init.d/oracleasm configure
Configuring the Oracle ASM library driver.
This will configure the on-boot properties of the Oracle ASM library
driver. The following questions will determine whether the driver is
loaded on boot and what permissions it will have. The current values
will be shown in brackets ('[]'). Hitting <ENTER> without typing an
answer will keep that current value. Ctrl-C will abort.
Default user to own the driver interface []: grid
Default group to own the driver interface []: dba
Start Oracle ASM library driver on boot (y/n) [n]: y
Scan for Oracle ASM disks on boot (y/n) [y]:
Writing Oracle ASM library driver configuration: done
Initializing the Oracle ASMLib driver: [ OK ]
Scanning the system for Oracle ASMLib disks: [ OK ]
Create ASM Disks(You need to do this on one node only).
[root@racnode1 ~]# oracleasm createdisk ASM1 /dev/sdc1
Writing disk header: done
Instantiating disk: done
Logonto racnode2 and execute the following command to scan asm disks.
[root@racnode2 ~]# /etc/init.d/oracleasm scandisks
Scanning the system for Oracle ASMLib disks: [ OK ]
[root@racnode2 ~]# /etc/init.d/oracleasm listdisks
ASM1
$ ./runInstaller
Starting Oracle Universal Installer...
Checking Temp space: must be greater than 120 MB. Actual 12428 MB Passed
Checking swap space: must be greater than 150 MB. Actual 3999 MB Passed
Checking monitor: must be configured to display at least 256 colors. Actual 16777216 Passed
Preparing to launch Oracle Universal Installer from /tmp/OraInstall2011-02-13_02-13-26PM. Please wait ...$ You can find the log of this install session at:
/u01/app/oraInventory/logs/installActions2011-02-13_02-13-26PM.log
$ ./runInstaller
Starting Oracle Universal Installer...
Checking Temp space: must be greater than 120 MB. Actual 12357 MB Passed
Checking swap space: must be greater than 150 MB. Actual 3845 MB Passed
Checking monitor: must be configured to display at least 256 colors. Actual 16777216 Passed
Preparing to launch Oracle Universal Installer from /tmp/OraInstall2011-02-13_03-07-26PM. Please wait ...$
[root@racnode1 ~]#
Running Oracle 11g root script...
The following environment variables are set as:
ORACLE_OWNER= grid
ORACLE_HOME= /u01/app/11.2.0/grid
Enter the full pathname of the local bin directory: [/usr/local/bin]:
Copying dbhome to /usr/local/bin ...
Copying oraenv to /usr/local/bin ...
Copying coraenv to /usr/local/bin ...
Creating /etc/oratab file...
Entries will be added to the /etc/oratab file as needed by
Database Configuration Assistant when a database is created
Finished running generic part of root script.
Now product-specific root actions will be performed.
Using configuration parameter file: /u01/app/11.2.0/grid/crs/install/crsconfig_params
Creating trace directory
LOCAL ADD MODE
Creating OCR keys for user 'root', privgrp 'root'..
Operation successful.
OLR initialization - successful
root wallet
root wallet cert
root cert export
peer wallet
profile reader wallet
pa wallet
peer wallet keys
pa wallet keys
peer cert request
pa cert request
peer cert
pa cert
peer root cert TP
profile reader root cert TP
pa root cert TP
peer pa cert TP
pa peer cert TP
profile reader pa cert TP
profile reader peer cert TP
peer user cert
pa user cert
Adding daemon to inittab
ACFS-9200: Supported
ACFS-9300: ADVM/ACFS distribution files found.
ACFS-9307: Installing requested ADVM/ACFS software.
ACFS-9308: Loading installed ADVM/ACFS drivers.
ACFS-9321: Creating udev for ADVM/ACFS.
ACFS-9323: Creating module dependencies - this may take some time.
ACFS-9327: Verifying ADVM/ACFS devices.
ACFS-9309: ADVM/ACFS installation correctness verified.
CRS-2672: Attempting to start 'ora.mdnsd' on 'racnode1'
CRS-2676: Start of 'ora.mdnsd' on 'racnode1' succeeded
CRS-2672: Attempting to start 'ora.gpnpd' on 'racnode1'
CRS-2676: Start of 'ora.gpnpd' on 'racnode1' succeeded
CRS-2672: Attempting to start 'ora.cssdmonitor' on 'racnode1'
CRS-2672: Attempting to start 'ora.gipcd' on 'racnode1'
CRS-2676: Start of 'ora.cssdmonitor' on 'racnode1' succeeded
CRS-2676: Start of 'ora.gipcd' on 'racnode1' succeeded
CRS-2672: Attempting to start 'ora.cssd' on 'racnode1'
CRS-2672: Attempting to start 'ora.diskmon' on 'racnode1'
CRS-2676: Start of 'ora.diskmon' on 'racnode1' succeeded
CRS-2676: Start of 'ora.cssd' on 'racnode1' succeeded
ASM created and started successfully.
Disk Group CRS_VOTING created successfully.
clscfg: -install mode specified
Successfully accumulated necessary OCR keys.
Creating OCR keys for user 'root', privgrp 'root'..
Operation successful.
CRS-4256: Updating the profile
Successful addition of voting disk a8528c91f2eb4f99bf077f568d5b94b1.
Successfully replaced voting disk group with +CRS_VOTING.
CRS-4256: Updating the profile
CRS-4266: Voting file(s) successfully replaced
## STATE File Universal Id File Name Disk group
-- ----- ----------------- --------- ---------
1. ONLINE a8528c91f2eb4f99bf077f568d5b94b1 (ORCL:ASM1) [CRS_VOTING]
Located 1 voting disk(s).
CRS-2672: Attempting to start 'ora.asm' on 'racnode1'
CRS-2676: Start of 'ora.asm' on 'racnode1' succeeded
CRS-2672: Attempting to start 'ora.CRS_VOTING.dg' on 'racnode1'
CRS-2676: Start of 'ora.CRS_VOTING.dg' on 'racnode1' succeeded
ACFS-9200: Supported
ACFS-9200: Supported
CRS-2672: Attempting to start 'ora.registry.acfs' on 'racnode1'
CRS-2676: Start of 'ora.registry.acfs' on 'racnode1' succeeded
Configure Oracle Grid Infrastructure for a Cluster ... succeeded
**********************************************
Running Oracle 11g root script...
The following environment variables are set as:
ORACLE_OWNER= grid
ORACLE_HOME= /u01/app/11.2.0/grid
Enter the full pathname of the local bin directory: [/usr/local/bin]:
Copying dbhome to /usr/local/bin ...
Copying oraenv to /usr/local/bin ...
Copying coraenv to /usr/local/bin ...
Creating /etc/oratab file...
Entries will be added to the /etc/oratab file as needed by
Database Configuration Assistant when a database is created
Finished running generic part of root script.
Now product-specific root actions will be performed.
Using configuration parameter file: /u01/app/11.2.0/grid/crs/install/crsconfig_params
Creating trace directory
LOCAL ADD MODE
Creating OCR keys for user 'root', privgrp 'root'..
Operation successful.
OLR initialization - successful
Adding daemon to inittab
ACFS-9200: Supported
ACFS-9300: ADVM/ACFS distribution files found.
ACFS-9307: Installing requested ADVM/ACFS software.
ACFS-9308: Loading installed ADVM/ACFS drivers.
ACFS-9321: Creating udev for ADVM/ACFS.
ACFS-9323: Creating module dependencies - this may take some time.
ACFS-9327: Verifying ADVM/ACFS devices.
ACFS-9309: ADVM/ACFS installation correctness verified.
CRS-4402: The CSS daemon was started in exclusive mode but found an active CSS daemon on node racnode1, number 1, and is terminating
An active cluster was found during exclusive startup, restarting to join the cluster
Configure Oracle Grid Infrastructure for a Cluster ... succeeded
[root@racnode2 bin]# ./crs_stat -t
Name Type Target State Host
------------------------------------------------------------
ora....TING.dg ora....up.type ONLINE ONLINE racnode1
ora....ER.lsnr ora....er.type ONLINE ONLINE racnode1
ora....N1.lsnr ora....er.type ONLINE ONLINE racnode2
ora....N2.lsnr ora....er.type ONLINE ONLINE racnode1
ora....N3.lsnr ora....er.type ONLINE ONLINE racnode1
ora.asm ora.asm.type ONLINE ONLINE racnode1
ora.cvu ora.cvu.type ONLINE ONLINE racnode1
ora....network ora....rk.type ONLINE ONLINE racnode1
ora.oc4j ora.oc4j.type ONLINE ONLINE racnode1
ora.ons ora.ons.type ONLINE ONLINE racnode1
ora....SM1.asm application ONLINE ONLINE racnode1
ora....E1.lsnr application ONLINE ONLINE racnode1
ora....de1.ons application ONLINE ONLINE racnode1
ora....de1.vip ora....t1.type ONLINE ONLINE racnode1
ora....SM2.asm application ONLINE ONLINE racnode2
ora....E2.lsnr application ONLINE ONLINE racnode2
ora....de2.ons application ONLINE ONLINE racnode2
ora....de2.vip ora....t1.type ONLINE ONLINE racnode2
ora....ry.acfs ora....fs.type ONLINE ONLINE racnode1
ora.scan1.vip ora....ip.type ONLINE ONLINE racnode2
ora.scan2.vip ora....ip.type ONLINE ONLINE racnode1
ora.scan3.vip ora....ip.type ONLINE ONLINE racnode1
Create ACFS File system---ora home shared by both nodes:
$ ./asmca &
[1] 10311
ACFS file system is running on racnode1,racnode2
[root@racnode1 sysconfig]# df -h
/dev/asm/orahomevg-12
9.0G 83M 9.0G 1% /u01/app/oracle
[root@racnode2 disks]# df -h
/dev/asm/orahomevg-12
9.0G 83M 9.0G 1% /u01/app/oracle
$ ./runInstaller
Starting Oracle Universal Installer...
Checking Temp space: must be greater than 120 MB. Actual 12358 MB Passed
Checking swap space: must be greater than 150 MB. Actual 3836 MB Passed
$ export ORACLE_HOME=/u01/app/oracle/product/11.2.0.2/db_1
$ ./dbca &
[1] 25448
racnode1.ukatru.com:/home/oracle>sqlplus / as sysdba
SQL*Plus: Release 11.2.0.2.0 Production on Sun Feb 13 19:23:45 2011
Copyright (c) 1982, 2010, Oracle. All rights reserved.
Connected to:
Oracle Database 11g Enterprise Edition Release 11.2.0.2.0 - 64bit Production
With the Partitioning, Real Application Clusters, Automatic Storage Management, OLAP,
Data Mining and Real Application Testing options
SQL>
Now you have fully functional RAC two node 11gR2 database is available.
