Startup sequence of Grid Infrastructure daemons and its resources in 11gR2 RAC.
OHASD Phase
OHASD Agent Phase
CRSD Agent Phase
CRSD Agent Phase crsd.bin starts two more agents (crsd orarootagent(Oracle root agent) and crsd oraagent(Oracle Agent))
1. When a node of an Oracle Clusterware cluster start/restarts, OHASD is started by platform-specific means. OHASD is the root for bringing up Oracle Clusterware. OHASD has access to the OLR (Oracle Local Registry) stored on the local file system. OLR provides needed data to complete OHASD initialization.
2. OHASD brings up GPNPD and CSSD. CSSD has access to the GPNP Profile stored on the local file system. This profile contains the following vital bootstrap data;
a. ASM Diskgroup Discovery String
b. ASM SPFILE location (Diskgroup name)
c. Name of the ASM Diskgroup containing the Voting Files
3. The Voting Files locations on ASM Disks are accessed by CSSD with well-known pointers in the ASM Disk headers and CSSD is able to complete initialization and start or join an existing cluster.
4. OHASD starts an ASM instance and ASM can now operate with CSSD initialized and operating. The ASM instance uses special code to locate the contents of the ASM SPFILE, assuming it is stored in a Diskgroup.
5. With an ASM instance operating and its Diskgroups mounted, access to Clusterware’s OCR is available to CRSD.
6. OHASD starts CRSD with access to the OCR in an ASM Diskgroup.
7. Clusterware completes initialization and brings up other services under its control.
When Clusterware starts three files are involved.
1. OLR – Is the first file to be read and opened. This file is local and this file contains information regarding where the voting disk is stored
and information to startup the ASM. (e.g ASM DiscoveryString)
2. VOTING DISK – This is the second file to be opened and read, this is dependent on only OLR being accessible.
ASM starts after CSSD or ASM does not start if CSSD is offline (i.e voting file missing)
How are Voting Disks stored in ASM?
Voting disks are placed directly on ASMDISK. Oracle Clusterware will store the votedisk on the disk within a disk group that holds the Voting Files.
Oracle Clusterware does not rely on ASM to access the Voting Files, which means Oracle Clusterware does not need of Diskgroup to read and write on ASMDISK. It is possible to check for existence of voting files on a ASMDISK using the V$ASM_DISK column VOTING_FILE.
So, voting files not depend of Diskgroup to be accessed, does not mean that the diskgroup is not needed, diskgroup and voting file are linked by their settings.
3. OCR – Finally the ASM Instance starts and mount all Diskgroups, then Clusterware Deamon (CRSD) opens and reads the OCR which is stored on Diskgroup.
So, if ASM already started, ASM does not depend on OCR or OLR to be online. ASM depends on CSSD (Votedisk) to be online.
Voting & OCR Disk resides in ASM Diskgroups, but as per startup sequence OCSSD starts first before than ASM, how is it possible?
How does OCSSD starts if voting disk & OCR resides in ASM Diskgroups?
You might wonder how CSSD, which is required to start the clustered ASM instance, can be started if voting disks are stored in ASM? This sounds like a chicken-and-egg problem: without access to the voting disks there is no CSS, hence the node cannot join the cluster. But without being part of the cluster, CSSD cannot start the ASM instance. To solve this problem the ASM disk headers have new metadata in 11.2: you can use kfed to read the header of an ASM disk containing a voting disk. The kfdhdb.vfstart and kfdhdb.vfend fields tell CSS where to find the voting file. This does not require the ASM instance to be up. Once the voting disks are located, CSS can access them and joins the cluster.
There is a exclusive mode to start ASM without CSSD (but it’s to restore OCR or VOTE purposes)
As per Oracle Documentation full description,you can find it in MOS Document 1053147.1
1. When a node of an Oracle Clusterware cluster start/restarts, OHASD is started by platform-specific means. OHASD is the root for bringing up Oracle Clusterware. OHASD has access to the OLR (Oracle Local Registry) stored on the local file system. OLR provides needed data to complete OHASD initialization.
2. OHASD brings up GPNPD and CSSD. CSSD has access to the GPNP Profile stored on the local file system. This profile contains the following vital bootstrap data;
a. ASM Diskgroup Discovery String
b. ASM SPFILE location (Diskgroup name)
c. Name of the ASM Diskgroup containing the Voting Files
3. The Voting Files locations on ASM Disks are accessed by CSSD with well-known pointers in the ASM Disk headers and CSSD is able to complete initialization and start or join an existing cluster.
4. OHASD starts an ASM instance and ASM can now operate with CSSD initialized and operating. The ASM instance uses special code to locate the contents of the ASM SPFILE, assuming it is stored in a Diskgroup.
5. With an ASM instance operating and its Diskgroups mounted, access to Clusterware’s OCR is available to CRSD.
6. OHASD starts CRSD with access to the OCR in an ASM Diskgroup.
7. Clusterware completes initialization and brings up other services under its control.
When Clusterware starts three files are involved.
1. OLR – Is the first file to be read and opened. This file is local and this file contains information regarding where the voting disk is stored
and information to startup the ASM. (e.g ASM DiscoveryString)
2. VOTING DISK – This is the second file to be opened and read, this is dependent on only OLR being accessible.
ASM starts after CSSD or ASM does not start if CSSD is offline (i.e voting file missing)
How are Voting Disks stored in ASM?
Voting disks are placed directly on ASMDISK. Oracle Clusterware will store the votedisk on the disk within a disk group that holds the Voting Files.
Oracle Clusterware does not rely on ASM to access the Voting Files, which means Oracle Clusterware does not need of Diskgroup to read and write on ASMDISK. It is possible to check for existence of voting files on a ASMDISK using the V$ASM_DISK column VOTING_FILE.
So, voting files not depend of Diskgroup to be accessed, does not mean that the diskgroup is not needed, diskgroup and voting file are linked by their settings.
3. OCR – Finally the ASM Instance starts and mount all Diskgroups, then Clusterware Deamon (CRSD) opens and reads the OCR which is stored on Diskgroup.
So, if ASM already started, ASM does not depend on OCR or OLR to be online. ASM depends on CSSD (Votedisk) to be online.
Voting & OCR Disk resides in ASM Diskgroups, but as per startup sequence OCSSD starts first before than ASM, how is it possible?
How does OCSSD starts if voting disk & OCR resides in ASM Diskgroups?
You might wonder how CSSD, which is required to start the clustered ASM instance, can be started if voting disks are stored in ASM? This sounds like a chicken-and-egg problem: without access to the voting disks there is no CSS, hence the node cannot join the cluster. But without being part of the cluster, CSSD cannot start the ASM instance. To solve this problem the ASM disk headers have new metadata in 11.2: you can use kfed to read the header of an ASM disk containing a voting disk. The kfdhdb.vfstart and kfdhdb.vfend fields tell CSS where to find the voting file. This does not require the ASM instance to be up. Once the voting disks are located, CSS can access them and joins the cluster.
There is a exclusive mode to start ASM without CSSD (but it’s to restore OCR or VOTE purposes)
As per Oracle Documentation full description,you can find it in MOS Document 1053147.1
OHASD Phase:-
- OHASD (Oracle High Availability Server Daemon) starts Firsts and it will start .
OHASD Agent Phase:-
- OHASD Agent starts and in turn this will start
| gipcd | Grid inter-process communication daemon, used for monitoring cluster interconnect |
| mdnsd | Multicast DNS service It resolves DNS requests on behalf of GNS |
| gns | The Grid Naming Service (GNS), a gateway between DNS and mdnsd, resolves DNS requests |
| gpnpd | Grid Plug and Play Daemon, Basically a profile similar like OCR contents stored in XML format in $GI_HOME/gpnp/profiles/<peer> etc., this is where used by OCSSD also to read the ASM disk locations to start up with out having ASM to be up, moreover this also provides the plug and play profile where this can be distributed across nodes to cluster |
| evmd/
evmlogger
| Evm service will be provided by evmd daemon, which is a information about events happening in cluster, stop node,start node, start instance etc. |
- cssdagent (cluster synchronization service agent), in turn starts
| ocssd | Cluster synchronization service daemon which manages node membership in the cluster |
If cssd found that ocssd is down, it will reboot the node to protect the data integrity.
- cssdmonitor (cluster synchronization service monitor), replaces oprocd and provides I/O fencing
- OHASD orarootagent starts and in turn starts
| crsd.bin | Cluster ready services, which manages high availability of cluster resources , like stopping , starting, failing over etc. |
| diskmon.bin | disk monitor (diskdaemon monitor) provides I/O fencing for exadata storage |
| octssd.bin | Cluster synchronization time services , provides Network time protocol services but manages its own rather depending on OS |
CRSD Agent Phase:- crsd.bin starts two more agents
crsd orarootagent(Oracle root agent) starts and in turn this will start
| gns | Grid interprocess communication daemon, used for monitoring cluster interconnect |
| gns vip | Multicast DNS service It resolves DNS requests on behalf of GNS |
| Network | Monitor the additional networks to provide HAIP to cluster interconnects |
| Scan vip | Monitor the scan vip, if found fail or unreachable failed to other node |
| Node vip | Monitor the node vip, if found fail or unreachable failed to other node |
crsd oraagent(Oracle Agent) starts and in turn it will start (the same functionality in 11gr1 and 10g managed by racgmain and racgimon background process) which is now managed by crs Oracle agent itself.
·
| ASM & disk groups | Start & monitor local asm instance |
| ONS | FAN feature, provides notification to interested client |
| eONS | FAN feature, provides notification to interested client |
| SCAN Listener | Start & Monitor scan listener |
| Node Listener | Start & monitor the node listener (rdbms?) |
Comments
Post a Comment