Thursday, 25 July 2013

ASM Instance

Hi,
This is a simple answer:
Automatic Storage Management (ASM)

ASM was a new storage option introduced with Oracle Database 10gR1 that provides the services of a filesystem, logical volume manager, and software RAID in a platform-independent manner. ASM can stripe and mirror your disks, allow disks to be added or removed while the database is under load, and automatically balance I/O to remove "hot spots." It also supports direct and asynchronous I/O and implements the Oracle Data Manager API (simplified I/O system call interface) introduced in Oracle9i.

ASM is not a general-purpose filesystem and can be used only for Oracle data files, redo logs, and control files. Files in ASM can be created and named automatically by the database (by use of the Oracle Managed Files feature) or manually by the DBA. Because the files stored in ASM are not accessible to the operating system, the only way to perform backup and recovery operations on databases that use ASM files is through Recovery Manager (RMAN).

ASM is implemented as a separate Oracle instance that must be up if other databases are to be able to access it. Memory requirements for ASM are light: only 64 MB for most systems.

Installing ASM

On Linux platforms, ASM can use raw devices or devices managed via the ASMLib interface. Oracle recommends ASMLib over raw devices for ease-of-use and performance reasons. ASMLib 2.0 is available for free download from OTN. This section walks through the process of configuring a simple ASM instance by using ASMLib 2.0 and building a database that uses ASM for disk storage.

Determine Which Version of ASMLib You Need

ASMLib 2.0 is delivered as a set of three Linux packages:

* oracleasmlib-2.0 - the ASM libraries
* oracleasm-support-2.0 - utilities needed to administer ASMLib
* oracleasm - a kernel module for the ASM library

Each Linux distribution has its own set of ASMLib 2.0 packages, and within each distribution, each kernel version has a corresponding oracleasm package. The following paragraphs describe how to determine which set of packages you need.

First, determine which kernel you are using by logging in as root and running the following command:

uname -rm

Ex:
# uname -rm
2.6.9-22.ELsmp i686

The example shows that this is a 2.6.9-22 kernel for an SMP (multiprocessor) box using Intel i686 CPUs.

Use this information to find the correct ASMLib packages on OTN:

1. Point your Web browser to http://www.oracle.com/technology/tech/linux/asmlib/index.html
2. Select the link for your version of Linux.
3. Download the oracleasmlib and oracleasm-support packages for your version of Linux
4. Download the oracleasm package corresponding to your kernel. In the example above, the oracleasm-2.6.9-22.ELsmp-2.0.0-1.i686.rpm package was used.

Next, install the packages by executing the following command as root:

rpm -Uvh oracleasm-kernel_version-asmlib_version.cpu_type.rpm \
oracleasmlib-asmlib_version.cpu_type.rpm \
oracleasm-support-asmlib_version.cpu_type.rpm

Ex:
# rpm -Uvh \
> oracleasm-2.6.9-22.ELsmp-2.0.0-1.i686.rpm \
> oracleasmlib-2.0.1-1.i386.rpm \
> oracleasm-support-2.0.1-1.i386.rpm
Preparing... ########################################### [100%]
1:oracleasm-support ########################################### [ 33%]
2:oracleasm-2.6.9-22.ELsm########################################### [ 67%]
3:oracleasmlib ########################################### [100%]

Configuring ASMLib

Before using ASMLib, you must run a configuration script to prepare the driver. Run the following command as root, and answer the prompts as shown in the example below.

# /etc/init.d/oracleasm configure
Configuring the Oracle ASM library driver.

This will configure the on-boot properties of the Oracle ASM library
driver. The following questions will determine whether the driver is
loaded on boot and what permissions it will have. The current values
will be shown in brackets ('[]'). Hitting <ENTER> without typing an
answer will keep that current value. Ctrl-C will abort.

Default user to own the driver interface []: oracle
Default group to own the driver interface []: dba
Start Oracle ASM library driver on boot (y/n) [n]: y
Fix permissions of Oracle ASM disks on boot (y/n) [y]: y
Writing Oracle ASM library driver configuration: [  OK  ]
Creating /dev/oracleasm mount point: [  OK  ]
Loading module "oracleasm": [  OK  ]
Mounting ASMlib driver filesystem: [  OK  ]
Scanning system for ASM disks: [  OK  ]

Next you tell the ASM driver which disks you want it to use. Oracle recommends that each disk contain a single partition for the entire disk. See Partitioning the Disks at the beginning of this section for an example of creating disk partitions.

You mark disks for use by ASMLib by running the following command as root:

/etc/init.d/oracleasm createdisk DISK_NAME device_name

Tip: Enter the DISK_NAME in UPPERCASE letters.

Ex:
# /etc/init.d/oracleasm createdisk VOL1 /dev/sdb1
Marking disk "/dev/sdb1" as an ASM disk: [  OK  ]
# /etc/init.d/oracleasm createdisk VOL1 /dev/sdc1
Marking disk "/dev/sdc1" as an ASM disk: [  OK  ]
# /etc/init.d/oracleasm createdisk VOL1 /dev/sdd1
Marking disk "/dev/sdd1" as an ASM disk: [  OK  ]

Verify that ASMLib has marked the disks:

# /etc/init.d/oracleasm listdisks
VOL1
VOL2
VOL3

Create the ASM Instance

ASM runs as a separate Oracle instance which can be created and configured using the Oracle Universal Installer. Now that ASMLib is installed and the disks are marked for use, you can create an ASM instance.

Log in as oracle and start runInstaller:

$ ./runInstaller

1. Select Installation Method
* Select Advanced Installation
* Click on Next
2. Specify Inventory Directory and Credentials
* Inventory Directory: /u01/app/oracle/oraInventory
* Operating System group name: oinstall
* Click on Next
3. Select Installation Type
* Select Enterprise Edition
* Click on Next
4. Specify Home Details
* Name: OraDB10gASM
* Path: /u01/app/oracle/product/10.2.0/asm
Note:Oracle recommends using a different ORACLE_HOME for ASM than the ORACLE_HOME used for the database for ease of administration.
* Click on Next
5. Product-specific Prerequisite Checks
* If you've been following the steps in this guide, all the checks should pass without difficulty. If one or more checks fail, correct the problem before proceeding.
* Click on Next
6. Select Configuration Option
* Select Configure Automatic Storage Management (ASM)
* Enter the ASM SYS password and confirm
* Click on Next
7. Configure Automatic Storage Management
* Disk Group Name: DATA
* Redundancy
- High mirrors data twice.
- Normal mirrors data once. This is the default.
- External does not mirror data within ASM. This is typically used if an external RAID array is providing redundancy.
* Add Disks
The disks you configured for use with ASMLib are listed as Candidate Disks. Select each disk you wish to include in the disk group.
* Click on Next
8. Summary
* A summary of the products being installed is presented.
* Click on Install.
9. Execute Configuration Scripts
* At the end of the installation, a pop up window will appear indicating scripts that need to be run as root. Login as root and run the indicated scripts.
* Click on OK when finished.
10. Configuration Assistants
* The Oracle Net, Oracle Database, and iSQL*Plus configuration assistants will run automatically
11. End of Installation
* Make note of the URLs presented in the summary, and click on Exit when ready.
12. Congratulations! Your new Oracle ASM Instance is up and ready for use.

Kind regards
Mohamed
·         Report Abuse

·         Like (0)
786141Newbie
786141 Jul 20, 2010 8:57 AM (in response to MohamedELAzab)
manaually?? Not gui。。。。
o    Report Abuse

o    Like (0)
MohamedELAzabExpert
MohamedELAzab Jul 20, 2010 9:05 AM (in response to 786141)
Hi ,
here is the steps:
Creating ASM Instance and diskgroups manually without dbca:

create the password file:

[oracle@shree ~]$ orapwd file=$ORACLE_HOME/dbs/orapw+ASM password=changeIt entries=5

Create required directories:

[oracle@shree ~]$ mkdir -p $ORACLE_BASE/admin/+ASM
[oracle@shree ~]$ cd $ORACLE_BASE/admin/+ASM
[oracle@shree +ASM]$ mkdir bdump
[oracle@shree +ASM]$ mkdir udump
[oracle@shree +ASM]$ mkdir cdump
[oracle@shree +ASM]$ mkdir pfile


Create the init+ASM.ora file:

Using vi editor or any other editor you like, create the init+ASM.ora file under the $ORACLE_HOME/dbs
directory and add the below lines into this file.

background_dump_dest='/u01/app/admin/+ASM/bdump'
core_dump_dest='/u01/app/admin/+ASM/cdump'
instance_type='asm'
large_pool_size=12M
remote_login_passwordfile='SHARED'
user_dump_dest='/u01/app/admin/+ASM/udump'

[oracle@shree ~]$ cat $ORACLE_HOME/dbs/init+ASM.ora
background_dump_dest='/u01/app/admin/+ASM/bdump'
core_dump_dest='/u01/app/admin/+ASM/cdump'
instance_type='asm'
large_pool_size=12M
remote_login_passwordfile='SHARED'
user_dump_dest='/u01/app/admin/+ASM/udump'
[oracle@shree ~]$

Create spfile+ASM.ora and start the instance using that file:

[oracle@shree ~]$ export ORACLE_SID=+ASM
[oracle@shree ~]$ sqlplus / as sysdba

SQL*Plus: Release 10.2.0.1.0 - Production on Thu Dec 1 14:06:35 2005

Copyright (c) 1982, 2005, Oracle. All rights reserved.

Connected to an idle instance.

SQL> create spfile from pfile;

File created.

SQL> startup mount

ASM instance started
Total System Global Area 83886080 bytes
Fixed Size 1217836 bytes
Variable Size 57502420 bytes
ASM Cache 25165824 bytes
ORA-15110: no diskgroups mounted

SQL> alter system set asm_diskstring = '/dev/raw/raw1', '/dev/raw/raw2', '/dev/raw/raw3', '/dev/raw/raw4', '/dev/raw/raw5';

System altered.

SQL> alter system set asm_diskgroups = 'DATA_GRP';

System altered.

SQL> create diskgroup data_grp
2 failgroup data_grp_f1 disk '/dev/raw/raw4'
3 failgroup data_grp_f2 disk '/dev/raw/raw5';

Diskgroup created.
 

SQL> set linesize 100
SQL> col path format a15
SQL> select name, path from v$asm_disk where name is not null;

NAME PATH
--------------- ---------------
DATA_GRP_0001 /dev/raw/raw5
DATA_GRP_0000 /dev/raw/raw4

SQL> select name, type, total_mb, free_mb from v$asm_diskgroup;

NAME TYPE TOTAL_MB FREE_MB
--------------- ------ ---------- ----------
DATA_GRP NORMAL 62776 62701


Open the /etc/oratab file and add the following line at the end:

+ASM:/u01/app/oracle/product/10.2.0/db_1:Y

Use ASM storage option to an Existing Database which is currently using Filesystem option:

SQL> set linesize 100
SQL> col path format a15
SQL> col name format a50
SQL> select name from v$datafile;

NAME
--------------------------------------------------
/u01/app/oradata/db102/system01.dbf
/u01/app/oradata/db102/undotbs01.dbf
/u01/app/oradata/db102/sysaux01.dbf
/u01/app/oradata/db102/users01.dbf

SQL> select name, path from v$asm_disk where name is not null;

no rows selected

SQL> create tablespace indx01 datafile '+DATA_GRP';

Tablespace created.

SQL> drop tablespace indx01;

Tablespace dropped.

SQL> create tablespace indx01 datafile '+DATA_GRP' SIZE 100m extent management local uniform size 1m;

Tablespace created.

SQL> drop tablespace indx01;

Tablespace dropped.

SQL> create tablespace indx01
2 datafile '+DATA_GRP' SIZE 100m
3 extent management local
4 segment space management auto
5 uniform size 1m;

Tablespace created.

SQL> select name from v$datafile;

NAME
--------------------------------------------------
/u01/app/oradata/db102/system01.dbf
/u01/app/oradata/db102/undotbs01.dbf
/u01/app/oradata/db102/sysaux01.dbf
/u01/app/oradata/db102/users01.dbf
+DATA_GRP/db102/datafile/indx01.258.576105687
Please refer to this link:
http://www.oracledba.org/10g/asm/asm_install.html
Kind regards
Mohamed
Oracle DBA
sans- M� f ��  : 13px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;">Set the hangcheck kernel module parameters by adding the following line to the "/etc/modprobe.conf" file.

options hangcheck-timer hangcheck_tick=30 hangcheck_margin=180
To load the module immediately, execute "modprobe -v hangcheck-timer".
Create the new groups and users.
groupadd oinstall
groupadd dba
groupadd oper

useradd -g oinstall -G dba oracle
passwd oracle
Create the directories in which the Oracle software will be installed.
mkdir -p /u01/crs/oracle/product/10.2.0/crs
mkdir -p /u01/app/oracle/product/10.2.0/db_1
mkdir -p /u01/oradata
chown -R oracle.oinstall /u01
During the installation, both RSH and RSH-Server were installed. Enable remote shell and rlogin by doing the following.
chkconfig rsh on
chkconfig rlogin on
service xinetd reload
Create the "/etc/hosts.equiv" file as the root user.
touch /etc/hosts.equiv
chmod 600 /etc/hosts.equiv
chown root:root /etc/hosts.equiv
Edit the "/etc/hosts.equiv" file to include all the RAC nodes:
+rac1 oracle
+rac2 oracle
+rac1-priv oracle
+rac2-priv oracle
Login as the oracle user and add the following lines at the end of the ".bash_profile" file.
# Oracle Settings
TMP=/tmp; export TMP
TMPDIR=$TMP; export TMPDIR

ORACLE_BASE=/u01/app/oracle; export ORACLE_BASE
ORACLE_HOME=$ORACLE_BASE/product/10.2.0/db_1; export ORACLE_HOME
ORACLE_SID=RAC1; export ORACLE_SID
ORACLE_TERM=xterm; export ORACLE_TERM
PATH=/usr/sbin:$PATH; export PATH
PATH=$ORACLE_HOME/bin:$PATH; export PATH

LD_LIBRARY_PATH=$ORACLE_HOME/lib:/lib:/usr/lib; export LD_LIBRARY_PATH
CLASSPATH=$ORACLE_HOME/JRE:$ORACLE_HOME/jlib:$ORACLE_HOME/rdbms/jlib; export CLASSPATH

if [ $USER = "oracle" ]; then
  if [ $SHELL = "/bin/ksh" ]; then
    ulimit -p 16384
    ulimit -n 65536
  else
    ulimit -u 16384 -n 65536
  fi
fi

Install VMware Client Tools

Login as the root user on the RAC1 virtual machine, then select the "VM > Install VMware Tools..." option from the main VMware Server Console menu.
Install VMware Tools
This should mount a virtual CD containing the VMware Tools software. Double-click on the CD icon labelled "VMware Tools" to open the CD. Right-click on the ".rpm" package and select the "Open with 'Install Packages'" menu option.
Install VMware Tools Start
Click the "Continue" button on the "Completed System Preparation" screen and wait for the installation to complete.
Install VMware Tools Completed Prep
Once the package is loaded, the CD should unmount automatically. You must then run the "vmware-config-tools.pl" script as the root user. The following listing is an example of the output you should expect.
# vmware-config-tools.pl

Stopping VMware Tools services in the virtual machine:
   Guest operating system daemon:                          [  OK  ]
Trying to find a suitable vmhgfs module for your running kernel.

The module bld-2.6.9-11.EL-i686up-RHEL4 loads perfectly in the running kernel.

pcnet32                30409  0
Unloading pcnet32 module

Trying to find a suitable vmxnet module for your running kernel.

The module bld-2.6.9-11.EL-i686up-RHEL4 loads perfectly in the running kernel.



Detected X.org version 6.8.


Please choose one of the following display sizes (1 - 13):

[1]  "640x480"
[2]  "800x600"
[3]  "1024x768"
[4]  "1152x864"
[5]  "1280x800"
[6]  "1152x900"
[7]  "1280x1024"
[8]  "1376x1032"
[9]  "1400x1050"
[10]  "1680x1050"
[11]  "1600x1200"
[12]< "1920x1200"
[13]  "2364x1773"
Please enter a number between 1 and 13:

[12] 3


X Window System Version 6.8.2
Release Date: 9 February 2005
X Protocol Version 11, Revision 0, Release 6.8.2
Build Operating System: Linux 2.6.9-11.EL i686 [ELF]
Current Operating System: Linux rac1.localdomain 2.6.9-22.EL #1 Sat Oct 8 17:48:27 CDT 2005 i686
Build Date: 07 October 2005
Build Host: x8664-build.home.local

        Before reporting problems, check http://wiki.X.Org
        to make sure that you have the latest version.
Module Loader present
OS Kernel: Linux version 2.6.9-22.EL (buildcentos@louisa.home.local)
  (gcc version 3.4.4 20050721 (Red Hat 3.4.4-2)) #1 Sat Oct 8 17:48:27 CDT 2005 P
Markers: (--) probed, (**) from config file, (==) default setting,
        (++) from command line, (!!) notice, (II) informational,
        (WW) warning, (EE) error, (NI) not implemented, (??) unknown.
(++) Log file: "/tmp/vmware-config0/XF86ConfigLog.3674", Time: Thu Apr 13 21:17:37 2006
(++) Using config file: "/tmp/vmware-config0/XF86Config.3674"

X is running fine with the new config file.

(WW) VMWARE(0): Failed to set up write-combining range (0xf0000000,0x1000000)
Starting VMware Tools services in the virtual machine:
   Switching to guest configuration:                       [  OK  ]
   Guest filesystem driver:                                [  OK  ]
   Guest vmxnet fast network device:                       [  OK  ]
   DMA setup:                                              [  OK  ]
   Guest operating system daemon:                          [  OK  ]

The configuration of VMware Tools e.x.p build-22874 for Linux for this running
kernel completed successfully.

You must restart your X session before any mouse or graphics changes take
effect.

You can now run VMware Tools by invoking the following command:
"/usr/bin/vmware-toolbox" during an XFree86 session.

To use the vmxnet driver, restart networking using the following commands:
/etc/init.d/network stop
rmmod pcnet32
rmmod vmxnet
depmod -a
modprobe vmxnet
/etc/init.d/network start

Enjoy,

--the VMware team

#
The VMware client tools are now installed.

Create Shared Disks

Shut down the RAC1 virtual machine using the following command.
# shutdown -h now
Create a directory on the host system to hold the shared virtual disks.
# mkdir -p /u01/VM/shared
On the VMware Server Console, click the "Edit virtual machine settings" button. On the "Virtual Machine Settings" screen, click the "+ Add" button.
Virtual Machine Settings Hard Disk
Select the hardware type of "Hard Disk" and click the "Next" button.
Virtual Machine Settings Hard Disk
Accept the "Create a new virtual disk" option by clicking the "Next" button.
Virtual Machine Settings New Disk
Accept the "SCSI" option by clicking the "Next" button.
Virtual Machine Settings SCSI
Set the disk size to "10.0" GB and uncheck the "Allocate all disk space now" option, then click the "Next" button.
Virtual Machine Settings Size
Set the disk name to "/u01/VM/shared/ocr.vmdk" and click the "Advanced" button.
Virtual Machine Settings Disk Name
Set the virtual device node to "SCSI 1:1" and the mode to "Independent" and "Persistent", then click the "Finish" button.
Virtual Machine Settings Disk Advanced
Repeat the previous hard disk creation steps 4 more times, using the following values.
  • File Name: /u01/VM/shared/votingdisk.vmdk
    Virtual Device Node: SCSI 1:2
    Mode: Independent and Persistent
  • File Name: /u01/VM/shared/asm1.vmdk
    Virtual Device Node: SCSI 1:3
    Mode: Independent and Persistent
  • File Name: /u01/VM/shared/asm2.vmdk
    Virtual Device Node: SCSI 1:4
    Mode: Independent and Persistent
  • File Name: /u01/VM/shared/asm3.vmdk
    Virtual Device Node: SCSI 1:5
    Mode: Independent and Persistent
At the end of this process, the virtual machine should look something like the picture below.
Virtual Machine Settings Final
Edit the contents of the "/u01/VM/RAC1/RAC1.vmx" file using a text editor, making sure the following entries are present. Some of the tries will already be present, some will not.
disk.locking = "FALSE"
diskLib.dataCacheMaxSize = "0" 
diskLib.dataCacheMaxReadAheadSize = "0" 
diskLib.dataCacheMinReadAheadSize = "0" 
diskLib.dataCachePageSize = "4096" 
diskLib.maxUnsyncedWrites = "0"

scsi1.present = "TRUE"
scsi1.virtualDev = "lsilogic" 
scsi1.sharedBus = "VIRTUAL"

scsi1:1.present = "TRUE"
scsi1:1.mode = "independent-persistent"
scsi1:1.fileName = "/u01/VM/shared/ocr.vmdk"
scsi1:1.deviceType = "plainDisk"
scsi1:1.redo = ""

scsi1:2.present = "TRUE"
scsi1:2.mode = "independent-persistent"
scsi1:2.fileName = "/u01/VM/shared/votingdisk.vmdk"
scsi1:2.deviceType = "plainDisk"
scsi1:2.redo = ""

scsi1:3.present = "TRUE"
scsi1:3.mode = "independent-persistent"
scsi1:3.fileName = "/u01/VM/shared/asm1.vmdk"
scsi1:3.deviceType = "plainDisk"
scsi1:3.redo = ""

scsi1:4.present = "TRUE"
scsi1:4.mode = "independent-persistent"
scsi1:4.fileName = "/u01/VM/shared/asm2.vmdk"
scsi1:4.deviceType = "plainDisk"
scsi1:4.redo = ""

scsi1:5.present = "TRUE"
scsi1:5.mode = "independent-persistent"
scsi1:5.fileName = "/u01/VM/shared/asm3.vmdk"
scsi1:5.deviceType = "plainDisk"
scsi1:5.redo = ""
Start the RAC1 virtual machine by clicking the "Power on this virtual machine" button on the VMware Server Console. When the server has started, log in as the root user so you can partition the disks. The current disks can be seen by issueing the following commands.
# cd /dev
# ls sd*
sda  sda1  sda2  sdb  sdc  sdd  sde  sdf
#
Use the "fdisk" command to partition the disks sdb to sdf. The following output shows the expected fdisk output for the sdb disk.
# fdisk /dev/sdb
Device contains neither a valid DOS partition table, nor Sun, SGI or OSF disklabel
Building a new DOS disklabel. Changes will remain in memory only,
until you decide to write them. After that, of course, the previous
content won't be recoverable.


The number of cylinders for this disk is set to 1305.
There is nothing wrong with that, but this is larger than 1024,
and could in certain setups cause problems with:
1) software that runs at boot time (e.g., old versions of LILO)
2) booting and partitioning software from other OSs
   (e.g., DOS FDISK, OS/2 FDISK)
Warning: invalid flag 0x0000 of partition table 4 will be corrected by w(rite)

Command (m for help): n
Command action
   e   extended
   p   primary partition (1-4)
p
Partition number (1-4): 1
First cylinder (1-1305, default 1):
Using default value 1
Last cylinder or +size or +sizeM or +sizeK (1-1305, default 1305):
Using default value 1305

Command (m for help): p

Disk /dev/sdb: 10.7 GB, 10737418240 bytes
255 heads, 63 sectors/track, 1305 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

   Device Boot      Start         End      Blocks   Id  System
/dev/sdb1               1        1305    10482381   83  Linux

Command (m for help): w
The partition table has been altered!

Calling ioctl() to re-read partition table.
Syncing disks.
#
In each case, the sequence of answers is "n", "p", "1", "Return", "Return", "p" and "w".
Once all the disks are partitioned, the results can be seen by repeating the previous "ls" command.
# cd /dev
# ls sd*
sda  sda1  sda2  sdb  sdb1  sdc  sdc1  sdd  sdd1  sde  sde1  sdf  sdf1
#
Edit the "/etc/sysconfig/rawdevices" file, adding the following lines.
/dev/raw/raw1 /dev/sdb1
/dev/raw/raw2 /dev/sdc1
/dev/raw/raw3 /dev/sdd1
/dev/raw/raw4 /dev/sde1
/dev/raw/raw5 /dev/sdf1
Restart the rawdevices service using the following command.
service rawdevices restart
Create some symbolic links to the raw devices. This is not really necessary, but it acts as a reminder of the true locatons.
ln -s /dev/raw/raw1  /u01/oradata/ocr
ln -s /dev/raw/raw2  /u01/oradata/votingdisk
ln -s /dev/raw/raw3  /u01/oradata/asm1
ln -s /dev/raw/raw4  /u01/oradata/asm2
ln -s /dev/raw/raw5  /u01/oradata/asm3
Run the following commands and add them the "/etc/rc.local" file.
chown oracle:oinstall /dev/raw/raw1
chown oracle:oinstall /dev/raw/raw2
chown oracle:oinstall /dev/raw/raw3
chown oracle:oinstall /dev/raw/raw4
chown oracle:oinstall /dev/raw/raw5
chmod 600 /dev/raw/raw1
chmod 600 /dev/raw/raw2
chmod 600 /dev/raw/raw3
chmod 600 /dev/raw/raw4
chmod 600 /dev/raw/raw5
The shared disks are now configured.

Clone the Virtual Machine

The current version of VMware Server does not include an option to clone a virtual machine, but the following steps illustrate how this can be achieved manually.
Shut down the RAC1 virtual machine using the following command.
# shutdown -h now
Copy the RAC1 virtual machine using the following command.
# cp -R /u01/VM/RAC1 /u01/VM/RAC2
Edit the contents of the "/u01/VM/RAC2/RAC1.vmx" file, making the following change.
displayName = "RAC2"
Ignore discrepancies with the file names in the "/u01/VM/RAC2" directory. This does not affect the action of the virtual machine.
In the VMware Server Console, select the File > Open menu options and browse for the "/u01/VM/RAC2/RAC1.vmx" file. Once opened, the RAC2 virtual machine is visible on the console. Start the RAC2 virtual machine by clicking the "Power on this virtual machine" button and click the "Create" button on the subsequent "Question" screen.
Question
Ignore any errors during the server startup. We are expecting the networking components to fail at this point.
Log in to the RAC2 virtual machine as the root user and start the "Network Configuration" tool (Applications > System Settings > Network).
Network Configuration
Highlight the "eth0" interface and click the "Edit" button on the toolbar and alter the IP address to "192.168.2.102" in the resulting screen.
eth0 General
Click on the "Hardware Device" tab and click the "Probe" button. Then accept the changes by clicking the "OK" button.
eth0 Hardware Devices
Repeat the process for the "eth1" interface, this time setting the IP Address to "192.168.0.102".
Click on the "DNS" tab and change the host name to "rac2.localdomain", then click on the "Devices" tab.
Network Configuration DNS
Once you are finished, save the changes (File > Save) and activate the network interfaces by highlighting them and clicking the "Activate" button. Once activated, the screen should look like the following image.
Network Configuration Final
Edit the "/home/oracle/.bash_profile" file on the RAC2 node to correct the ORACLE_SID value.
ORACLE_SID=RAC2; export ORACLE_SID
Start the RAC1 virtual machine and restart the RAC2 virtual machine. Whilst starting up, the "Kudzu" detection screen may be displayed.
Kudzu
Press a key and accept the configuration change on the following screen.
Kudzu Configure
When both nodes have started, check they can both ping all the public and private IP addresses using the following commands.
ping -c 3 rac1
ping -c 3 rac1-priv
ping -c 3 rac2
ping -c 3 rac2-priv
At this point the virtual IP addresses defined in the /etc/hosts file will not work, so don't bother testing them.
Note. In the original installation I used RSH for inter-node communication. If you wish to use SSH instead, perform the following configurations. If you prefer to use RSH, jump straight to the runcluvfy.sh note.
Configure SSH on each node in the cluster. Log in as the "oracle" user and perform the following tasks on each node.
su - oracle
mkdir ~/.ssh
chmod 700 ~/.ssh
/usr/bin/ssh-keygen -t rsa # Accept the default settings.
The RSA public key is written to the "~/.ssh/id_rsa.pub" file and the private key to the "~/.ssh/id_rsa" file.
Log in as the "oracle" user on RAC1, generate an "authorized_keys" file on RAC1 and copy it to RAC2 using the following commands.
su - oracle
cd ~/.ssh
cat id_rsa.pub >> authorized_keys
scp authorized_keys rac2:/home/oracle/.ssh/
Next, log in as the "oracle" user on RAC2 and perform the following commands.
su - oracle
cd ~/.ssh
cat id_rsa.pub >> authorized_keys
scp authorized_keys rac1:/home/oracle/.ssh/
The "authorized_keys" file on both servers now contains the public keys generated on all RAC nodes.
To enable SSH user equivalency on the cluster member nodes issue the following commands on each node.
ssh rac1 date
ssh rac2 date
ssh rac1.localdomain date
ssh rac2.localdomain date
exec /usr/bin/ssh-agent $SHELL
/usr/bin/ssh-add
You should now be able to SSH and SCP between servers without entering passwords.
Before installing the clusterware, check the prerequisites have been met using the "runcluvfy.sh" utility in the clusterware root directory.
/mountpoint/clusterware/runcluvfy.sh stage -pre crsinst -n rac1,rac2 -verbose
If you get any failures be sure to correct them before proceeding.
It's a good idea to take a snapshot of the virtual machines, so you can repeat the following stages if you run into any problems. To do this, shutdown both virtual machines and issue the following commands.
# cd /u01/VM
# tar -cvf RAC-PreClusterware.tar RAC1 RAC2 shared
# gzip RAC-PreClusterware.tar
The virtual machine setup is now complete.

Install the Clusterware Software

Start the RAC1 and RAC2 virtual machines, login to RAC1 as the oracle user and start the Oracle installer.
./runInstaller
On the "Welcome" screen, click the "Next" button.
Clusterware Welcome
Accept the default inventory location by clicking the "Next" button.
Clusterware Inventory
Enter the appropriate name and path for the Oracle Home and click the "Next" button.
Clusterware Oracle Home
Wait while the prerequisite checks are done. If you have any failures correct them and retry the tests before clicking the "Next" button.
Clusterware Prerequisites
You can choose to ignore the warnings from the prerequisite checks and click the "Next" button. If you do, you will also need to ignore the subsequent warning message by clicking the "Yes" button.
Clusterware Prerequisites Warning
The "Specify Cluster Configuration" screen shows only the RAC1 node in the cluster. Click the "Add" button to continue.
Clusterware Cluster Config
Enter the details for the RAC2 node and click the "OK" button.
Clusterware Cluster Config Add
Click the "Next" button to continue.
Clusterware Cluster Config
The "Specific Network Interface Usage" screen defines how each network interface will be used. Highlight the "eth0" interface and click the "Edit" button.
Clusterware Network Interface Usage
Set the "eht0" interface type to "Public" and click the "OK" button.
Clusterware Network Interface Eth0 Type
Leave the "eth1" interface as private and click the "Next" button.
Clusterware Network Interface Usage
Click the "External Redundancy" option, enter "/dev/raw/raw1" as the OCR Location and click the "Next" button. To have greater redundancy we would need to define another shared disk for an alternate location.
Clusterware OCR Location
Click the "External Redundancy" option, enter "/dev/raw/raw2" as the Voting Disk Location and click the "Next" button. To have greater redundancy we would need to define another shared disk for an alternate location.
Clusterware Voting Disk Location
On the "Summary" screen, click the "Install" button to continue.
Clusterware Summary
Wait while the installation takes place.
Clusterware Install
Once the install is complete, run the orainstRoot.sh and root.sh scripts on both nodes as directed on the following screen.
Clusterware Post Install Scripts
The output from the orainstRoot.sh file should look something like that listed below.
# cd /u01/app/oracle/oraInventory
# ./orainstRoot.sh
Changing permissions of /u01/app/oracle/oraInventory to 770.
Changing groupname of /u01/app/oracle/oraInventory to oinstall.
The execution of the script is complete
#
The output of the root.sh will vary a little depending on the node it is run on. The following text is the output from the RAC1 node.
# cd /u01/crs/oracle/product/10.2.0/crs
# ./root.sh
WARNING: directory '/u01/crs/oracle/product/10.2.0' is not owned by root
WARNING: directory '/u01/crs/oracle/product' is not owned by root
WARNING: directory '/u01/crs/oracle' is not owned by root
WARNING: directory '/u01/crs' is not owned by root
WARNING: directory '/u01' is not owned by root
Checking to see if Oracle CRS stack is already configured
/etc/oracle does not exist. Creating it now.

Setting the permissions on OCR backup directory
Setting up NS directories
Oracle Cluster Registry configuration upgraded successfully
WARNING: directory '/u01/crs/oracle/product/10.2.0' is not owned by root
WARNING: directory '/u01/crs/oracle/product' is not owned by root
WARNING: directory '/u01/crs/oracle' is not owned by root
WARNING: directory '/u01/crs' is not owned by root
WARNING: directory '/u01' is not owned by root
assigning default hostname rac1 for node 1.
assigning default hostname rac2 for node 2.
Successfully accumulated necessary OCR keys.
Using ports: CSS=49895 CRS=49896 EVMC=49898 and EVMR=49897.
node <nodenumber>: <nodename> <private interconnect name> <hostname>
node 1: rac1 rac1-priv rac1
node 2: rac2 rac2-priv rac2
Creating OCR keys for user 'root', privgrp 'root'..
Operation successful.
Now formatting voting device: /dev/raw/raw2
Format of 1 voting devices complete.
Startup will be queued to init within 90 seconds.
Adding daemons to inittab
Expecting the CRS daemons to be up within 600 seconds.
CSS is active on these nodes.
        rac1
CSS is inactive on these nodes.
        rac2
Local node checking complete.
Run root.sh on remaining nodes to start CRS daemons.
#
Ignore the directory ownership warnings. We should really use a separate directory structure for the clusterware so it can be owned by the root user, but it has little effect on the finished results.
The output from the RAC2 node is listed below.
# cd /u01/crs/oracle/product/10.2.0/crs
# ./root.sh
WARNING: directory '/u01/crs/oracle/product/10.2.0' is not owned by root
WARNING: directory '/u01/crs/oracle/product' is not owned by root
WARNING: directory '/u01/crs/oracle' is not owned by root
WARNING: directory '/u01/crs' is not owned by root
WARNING: directory '/u01' is not owned by root
Checking to see if Oracle CRS stack is already configured
/etc/oracle does not exist. Creating it now.

Setting the permissions on OCR backup directory
Setting up NS directories
Oracle Cluster Registry configuration upgraded successfully
WARNING: directory '/u01/crs/oracle/product/10.2.0' is not owned by root
WARNING: directory '/u01/crs/oracle/product' is not owned by root
WARNING: directory '/u01/crs/oracle' is not owned by root
WARNING: directory '/u01/crs' is not owned by root
WARNING: directory '/u01' is not owned by root
clscfg: EXISTING configuration version 3 detected.
clscfg: version 3 is 10G Release 2.
assigning default hostname rac1 for node 1.
assigning default hostname rac2 for node 2.
Successfully accumulated necessary OCR keys.
Using ports: CSS=49895 CRS=49896 EVMC=49898 and EVMR=49897.
node <nodenumber>: <nodename> <private interconnect name> <hostname>
node 1: rac1 rac1-priv rac1
node 2: rac2 rac2-priv rac2
clscfg: Arguments check out successfully.

NO KEYS WERE WRITTEN. Supply -force parameter to override.
-force is destructive and will destroy any previous cluster
configuration.
Oracle Cluster Registry for cluster has already been initialized
Startup will be queued to init within 90 seconds.
Adding daemons to inittab
Expecting the CRS daemons to be up within 600 seconds.
CSS is active on these nodes.
        rac1
        rac2
CSS is active on all nodes.
Waiting for the Oracle CRSD and EVMD to start
Waiting for the Oracle CRSD and EVMD to start
Waiting for the Oracle CRSD and EVMD to start
Waiting for the Oracle CRSD and EVMD to start
Waiting for the Oracle CRSD and EVMD to start
Waiting for the Oracle CRSD and EVMD to start
Waiting for the Oracle CRSD and EVMD to start
Oracle CRS stack installed and running under init(1M)
Running vipca(silent) for configuring nodeapps
The given interface(s), "eth0" is not public. Public interfaces should be used to configure virtual IPs.
#
Here you can see that some of the configuration steps are omitted as they were done by the first node. In addition, the final part of the script ran the Virtual IP Configuration Assistant (VIPCA) in silent mode, but it failed. This is because my public IP addresses are actually within the "192.168.255.255" range which is a private IP range. If you were using "legal" IP addresses you would not see this and you could ignore the following VIPCA steps.
Run the VIPCA manually as the root user on the RAC2 node using the following command.
# cd /u01/crs/oracle/product/10.2.0/crs/bin
# ./vipca
Click the "Next" button on the VIPCA welcome screen.
Clusterware VIPCA Welcome
Highlight the "eth0" interface and click the "Next" button.
Clusterware VIPCA Interfaces
Enter the vitual IP alias and address for each node. Once you enter the first alias, the remaining values should default automatically. Click the "Next" button to continue.
Clusterware VIPCA Virtual IPs
Accept the summary information by clicking the "Finish" button.
Clusterware VIPCA Summary
Wait until the configuration is complete, then click the "OK" button.
Clusterware VIPCA Complete
Accept the VIPCA results by clicking the "Exit" button.
Clusterware VIPCA Results
You should now return to the "Execute Configuration Scripts" screen on RAC1 and click the "OK" button.
Clusterware Post Install Scripts
Wait for the configuration assistants to complete.
Clusterware Config Assistants
When the installation is complete, click the "Exit" button to leave the installer.
Clusterware Complete
It's a good idea to take a snapshot of the virtual machines, so you can repeat the following stages if you run into any problems. To do this, shutdown both virtual machines and issue the following commands.
# cd /u01/VM
# tar -cvf RAC-PostClusterware.tar RAC1 RAC2 shared
# gzip RAC-PostClusterware.tar
The clusterware installation is now complete.

Install the Database Software and Create an ASM Instance

Start the RAC1 and RAC2 virtual machines, login to RAC1 as the oracle user and start the Oracle installer.
./runInstaller
On the "Welcome" screen, click the "Next" button.
Database Welcome
Select the "Enterprise Edition" option and click the "Next" button.
Database Installation Type
Enter the name and path for the Oracle Home and click the "Next" button.
Database Specify Home
Select the "Cluster Install" option and make sure both RAC nodes are selected, the click the "Next" button.
Database Cluster Installation
Wait while the prerequisite checks are done. If you have any failures correct them and retry the tests before clicking the "Next" button.
Database Prerequisites
You can choose to ignore the warnings from the prerequisite checks and click the "Next" button. If you do, you will also need to ignore the subsequent warning message by clicking the "Yes" button.
Database Prerequisites Warning
Select the "Configure Automatic Storage Management (ASM)" option, enter the SYS password for the ASM instance, then click the "Next" button.
Database Configuration Option
Select the "External" redundancy option (no mirroring), select all three raw disks (raw3, raw4 and raw5), then click the "Next" button.
Database Configure ASM
On the "Summary" screen, click the "Install" button to continue.
Database Summary
Wait while the database software installs.
Database Install
Once the installation is complete, wait while the configuration assistants run.
Database Configuration Asistants
Execute the "root.sh" scripts on both nodes, as instructed on the "Execute Configuration scripts" screen, then click the "OK" button.
Database Post Install Scripts
When the installation is complete, click the "Exit" button to leave the installer.
Database Complete
It's a good idea to take a snapshot of the virtual machines, so you can repeat the following stages if you run into any problems. To do this, shutdown both virtual machines and issue the following commands.
# cd /u01/VM
# tar -cvf RAC-PostASM.tar RAC1 RAC2 shared
# gzip RAC-PostASM.tar
The database software installation and ASM creation step is now complete.

Create a Database using the DBCA

Start the RAC1 and RAC2 virtual machines, login to RAC1 as the oracle user and start the Database Configuration Assistant.
dbca
On the "Welcome" screen, select the "Oracle Real Application Clusters database" option and click the "Next" button.
DBCA Welcome
Select the "Create a Database" option and click the "Next" button.
DBCA Create Database
Highlight both RAC nodes and click the "Next" button.
DBCA Node Selection
Select the "Custom Database" option and click the "Next" button.
DBCA CustomDatabase
Enter the values "RAC.WORLD" and "RAC" for the Global Database Name and SID Prefix respectively, then click the "Next" button.
DBCA Database ID
Accept the management options by clicking the "Next" button. If you are attempting the installation on a server with limited memory, you may prefer not to configure Enterprise Manager at this time.
DBCA Management Options
Enter database passwords then click the "Next" button.
DBCA Database Credentials
Select the "Automatic Storage Management (ASM)" option, then click the "Next" button.
DBCA ASM
Select the "DATA" disk group, then click the "Next" button.
DBCA Disk Groups
Accept the "Use Oracle-Managed Files" database location by the "Next" button.
DBCA Database Location
Check both the "Specify Flash Recovery Area" and "Enable Archiving" options. Enter "+DATA" as the Flash Recovery Area, then click the "Next" button.
DBCA Recovery Config
Uncheck all but the "Enterprise Manager Repository" option, then click the "Standard Database Components..." button.
DBCA Database Content
Uncheck all but the "Oracle JVM" option, then click the "OK" button, followed by the "Next" button on the previous screen. If you are attempting the installation on a server with limited memory, you may prefer not to install the JVM at this time.
DBCA Standard DB Components
Accept the current database services configuration by clicking the "Next" button.
DBCA Services
Select the "Custom" memory management option and accept the default settings by clicking the "Next" button.
DBCA Parameters
Accept the database storage settings by clicking the "Next" button.
DBCA Storage
Accept the database creation options by clicking the "Finish" button.
DBCA Creation Options
Accept the summary information by clicking the "OK" button.
DBCA Summary
Wait while the database is created.
DBCA Creation
Once the database creation is complete you are presented with the following screen. Make a note of the information on the screen and click the "Exit" button.
DBCA Complete
The RAC database creation is now complete.

TNS Configuration

Once the installation is complete, the "$ORACLE_HOME/network/admin/listener.ora" file on each RAC node will contain entries similar to the following.
LISTENER_RAC1 =
  (DESCRIPTION_LIST =
    (DESCRIPTION =
      (ADDRESS = (PROTOCOL = IPC)(KEY = EXTPROC1))
      (ADDRESS = (PROTOCOL = TCP)(HOST = rac1-vip.localdomain)(PORT = 1521)(IP = FIRST))
      (ADDRESS = (PROTOCOL = TCP)(HOST = 192.168.2.101)(PORT = 1521)(IP = FIRST))
    )
  )

SID_LIST_LISTENER_RAC1 =
  (SID_LIST =
    (SID_DESC =
      (SID_NAME = PLSExtProc)
      (ORACLE_HOME = /u01/app/oracle/product/10.2.0/db_1)
      (PROGRAM = extproc)
    )
  )
The "$ORACLE_HOME/network/admin/tnsnames.ora" file on each RAC node will contain entries similar to the following.
RAC =
  (DESCRIPTION =
    (ADDRESS = (PROTOCOL = TCP)(HOST = rac1-vip.localdomain)(PORT = 1521))
    (ADDRESS = (PROTOCOL = TCP)(HOST = rac2-vip.localdomain)(PORT = 1521))
    (LOAD_BALANCE = yes)
    (CONNECT_DATA =
      (SERVER = DEDICATED)
      (SERVICE_NAME = RAC.WORLD)
    )
  )

LISTENERS_RAC =
  (ADDRESS_LIST =
    (ADDRESS = (PROTOCOL = TCP)(HOST = rac1-vip.localdomain)(PORT = 1521))
    (ADDRESS = (PROTOCOL = TCP)(HOST = rac2-vip.localdomain)(PORT = 1521))
  )

RAC2 =
  (DESCRIPTION =
    (ADDRESS = (PROTOCOL = TCP)(HOST = rac2-vip.localdomain)(PORT = 1521))
    (CONNECT_DATA =
      (SERVER = DEDICATED)
      (SERVICE_NAME = RAC.WORLD)
      (INSTANCE_NAME = RAC2)
    )
  )

RAC1 =
  (DESCRIPTION =
    (ADDRESS = (PROTOCOL = TCP)(HOST = rac1-vip.localdomain)(PORT = 1521))
    (CONNECT_DATA =
      (SERVER = DEDICATED)
      (SERVICE_NAME = RAC.WORLD)
      (INSTANCE_NAME = RAC1)
    )
  )
This configuration allows direct connections to specific instance, or using a load balanced connection to the main service.
$ sqlplus / as sysdba

SQL*Plus: Release 10.2.0.1.0 - Production on Tue Apr 18 12:27:11 2006

Copyright (c) 1982, 2005, Oracle.  All rights reserved.


Connected to:
Oracle Database 10g Enterprise Edition Release 10.2.0.1.0 - Production
With the Partitioning, Real Application Clusters, OLAP and Data Mining options

SQL> CONN sys/password@rac1 AS SYSDBA
Connected.
SQL> SELECT instance_name, host_name FROM v$instance;

INSTANCE_NAME    HOST_NAME
---------------- ----------------------------------------------------------------
RAC1             rac1.localdomain

SQL> CONN sys/password@rac2 AS SYSDBA
Connected.
SQL> SELECT instance_name, host_name FROM v$instance;

INSTANCE_NAME    HOST_NAME
---------------- ----------------------------------------------------------------
RAC2             rac2.localdomain

SQL> CONN sys/password@rac AS SYSDBA
Connected.
SQL> SELECT instance_name, host_name FROM v$instance;

INSTANCE_NAME    HOST_NAME
---------------- ----------------------------------------------------------------
RAC1             rac1.localdomain

SQL>

Check the Status of the RAC

There are several ways to check the status of the RAC. The srvctl utility shows the current configuration and status of the RAC database.
$ srvctl config database -d RAC
rac1 RAC1 /u01/app/oracle/product/10.2.0/db_1
rac2 RAC2 /u01/app/oracle/product/10.2.0/db_1
$

$ srvctl status database -d RAC
Instance RAC1 is running on node rac1
Instance RAC2 is running on node rac2
$
The V$ACTIVE_INSTANCES view can also display the current status of the instances.
$ sqlplus / as sysdba

SQL*Plus: Release 10.2.0.1.0 - Production on Tue Apr 18 12:15:15 2006

Copyright (c) 1982, 2005, Oracle.  All rights reserved.


Connected to:
Oracle Database 10g Enterprise Edition Release 10.2.0.1.0 - Production
With the Partitioning, Real Application Clusters, OLAP and Data Mining options

SQL> SELECT * FROM v$active_instances;

INST_NUMBER INST_NAME
----------- ------------------------------------------------------------
          1 rac1.localdomain:RAC1
          2 rac2.localdomain:RAC2

SQL>
Finally, the GV$ allow you to display global information for the whole RAC.
SQL> SELECT inst_id, username, sid, serial# FROM gv$session WHERE username IS NOT NULL;

   INST_ID USERNAME                              SID    SERIAL#
---------- ------------------------------ ---------- ----------
         1 SYS                                   127          2
         1 SYS                                   128         28
         1 SYS                                   130         10
         1 SYS                                   131          4
         1 SYS                                   133          9
         1 DBSNMP                                134         27
         1 DBSNMP                                135          1
         1 SYS                                   153        122
         2 SYSMAN                                120        243
         2 DBSNMP                                122         37
         2 DBSNMP                                124         93

   INST_ID USERNAME                              SID    SERIAL#
---------- ------------------------------ ---------- ----------
         2 SYSMAN                                125          2
         2 SYSMAN                                127          6
         2 SYS                                   128         26
         2 SYS                                   129         30
         2 SYS                                   130          3
         2 SYS                                   133        149
         2 SYSMAN                                134         58
         2 SYS                                   136         32

19 rows selected.

SQL>
If you have configured Enterprise Manager, it can be used to view the configuration and current status of the database.
Oracle Enterprise Manager

Direct and Asynchronous I/O

Remember to use direct I/O and asynchronous I/O to improve performance. Direct I/O has been supported over NFS for some time, but support for asynchronous I/O over NFS was only introduced in RHEL 4 Update 3 (and its clones), so you need to use an up to date version of your Linux distribution to take advantage of this feature.
You can get details about this Direct and Asynchronous I/O by following the link.
For more information see:

No comments: