Skip to main content

EBS Standby Role Tranistion using standby database and standby application Tier

 Role Transitions

A database can operate in either a primary or standby role - these roles are mutually exclusive. Oracle Data Guard enables you to change these roles dynamically by issuing SQL commands, and supports the following transitions:
    • Switchover Allows the primary database to switch roles with one of its standby databases. There is no data loss during a switchover. After a switchover, each database continues to participate in the Oracle Data Guard configuration with its new role.

    • Failover
      Changes a standby database to the primary role in response to a primary database failure.
The following role transitions are discussed:
6.1 Performing a Switchover6.2 Performing a Failover6.3 Performing a Switchback to the Primary Following A Switchover/Failover
Each of these three transitions requires some application configuration to be performed. Most of the application configuration steps are common to all of the transitions and so are grouped in the final part of this section:
6.4 Application Tier Configuration After a Role Transition (Switchover/Failover/Switchback)

6.1 Performing a Switchover

A switchover is typically used to reduce primary database downtime during planned outages, such as operating system or hardware upgrades. A switchover takes place in two phases. In the first phase, the existing primary database undergoes a transition to the standby role. In the second phase, a standby database undergoes a transition to become the primary. In this case, the primary site is accessible and is involved in the switchover.
There will be minimal parameters to change on switchover if you have configured your parameter files as previously defined in sections 3,4 and 5.

6.1.1 Preparing for switchover to standby server

    Note: Before proceeding, it is essential that you complete the ADOP patching cycle: prepare, apply, cutover, cleanup. When using an online patch or hotpatch run fs_clone to make sure that both run and patch file systems are at the same patchlevel.

     
    1. Verify the primary database instance is open and that the standby database instance is mounted.
       
    2. Shut down all the application tier services on all nodes on the primary site.
      $ adstpall.sh <appsuser>/<appspass>
    3. Verify there are no active users connected to the database and shut down all of the sessions in the primary database.
       
    4. Issue the following SQL command on the primary and standby databases to ensure that the last redo data file transmitted from the primary database has been applied to the standby database. If necessary, perform a log file switch prior to running the query.
      SQL>select sequence#,applied from v$archived_log;
    5. Query the switchover_status column of v$database to determine whether the database is ready to switch modes.
      SQL>select switchover_status from v$database;
      If this query returns "TO STANDBY", then the environment is ready to switch. If it returns "ACTIVE SESSIONS", then the switch command should be used with the 'session shutdown' option.

6.1.2. Initiate the Switchover on the Primary Database

    Note For RAC Configurations: Only one primary instance should be open during a switchover; all others must be shut down. After switchover LOG_ARCHIVE_DEST_STATE_2 should be set for all instances (on the current primary after switchover).
Connect as sysdba and issue the following SQL command:
    SQL>alter database commit to switchover to physical standby with session shutdown;
After this statement completes, the primary database will have been converted to a standby database. As part of the statement's execution, the current control file is backed up to the current SQL session's trace file, which makes it possible to reconstruct the control file if necessary.
Change the value of LOG_ARCHIVE_DEST_STATE_2 to defer on the primary.

6.1.3 Shut down and Mount the Primary Database

To complete the transition, the database must be shut down and restarted in a mounted state. In addition, recovery can be started in preparation for the transition:
    SQL>shutdown immediate
    SQL>startup nomount pfile=$ORACLE_HOME/dbs/init<SID>.ora | spfile
    SQL>alter database mount standby database;
    SQL>alter database recover managed standby database disconnect from session;
    Note: At this point in the switchover, both databases are standby databases.

6.1.4 Verify the Switchover Status on the Standby Server

As the ORACLE user on the standby-to-be-primary database server, verify that it is ready to switch to become the primary:
    SQL>select switchover_status from v$database;
This should return a value 'TO PRIMARY'. Any other value, such as SESSIONS ACTIVE, NOT ALLOWED, and so on, should be investigated and corrected as in step 6.1.2 above.

6.1.5 Switch the Selected Standby Database to the Primary Role

    Note For RAC Configurations: As in section 6.1.2, only one primary instance should be open during a switchover; all others must be shut down.
For the switchover process to be coordinated, a standby database must be either mounted and in redo apply mode or open read-only. Once it is mounted in an appropriate mode, issue the following command to transition it to the primary role:
    SQL>alter database commit to switchover to primary with session shutdown;
Update the network parameters on both database servers
As the ORACLE user on both the primary and standby database servers, change the host name in the standby service definitions in the <CONTEXT_NAME>_ifile.ora and < STNDBY_CONTEXT>_ifile.ora files in the <TNS_ADMIN> directories, to point to the new standby server.
Change the LOG_ARCHIVE_DEST_STATE_2 to enable. on the new primary.
To complete the transition, the database must be shut down and re-started:
    SQL>shutdown immediate
    SQL>startup pfile=$ORACLE_HOME/dbs/init<SID>.ora | spfile

6.1.6 Complete the database configuration

    1. Connect to the new primary database using SQL*Plus as user APPS and execute the following commands:
      SQL>exec fnd_net_services.remove_system ('<SID>');
      SQL>commit;
      SQL>exec fnd_conc_clone.setup_clean;
      SQL>exec ad_zd_fixer.clear_valid_nodes_info;
    2. As the ORACLE user on the new primary database server, run AutoConfig to complete the configuration, providing the APPSpwd when prompted:
      $ cd <RDBMS_ORACLE_HOME>/appsutil/scripts/<context>
      $ ./adautocfg.sh
    3. Once autoconfig has completed, stop and start the listener on the new primary database server:
      $ lsnrctl stop <SID>
      $ lsnrctl start <SID>
    4. On the new standby server, stop and start the listener for standby services:
      $ lsnrctl stop <standby service>
      $ lsnrctl start <standby service>
      Note For RAC Configurations: Repeat steps 2-4 on each instance (each node will have its own listener). Rerun AutoConfig on all nodes after completing steps 2-4 for each instance to update all configuration files with all nodes in the cluster.


    5. For applications-specific configurations, follow the steps in 6.4 Configuring Application Tiers After Role Transition (Switchover/Failover/Switchback).

6.2 Performing a Failover

This section describes the steps necessary to perform a failover to your standby site due to a complete failure of the primary site.
There will be minimal parameters to change on switchover if you have configured your parameter files as previously defined in sections 3,4 and 5.
Performing a failover means that you will need to create a new standby database environment from the environment to which you failed over as soon as possible to ensure maximum disaster recovery protection.
Note: If there is an incomplete ADOP patching cycle, follow the steps in Appendix H: Handling Online Patching Following A Standby Failover Event in order to abort the patch cycle after you have performed the Database failover steps.

6.2.1 Flush any Unsent Redo from the Primary Database to the Target Standby Database

If the primary database can be mounted, it may be possible to flush any archived redo from the primary database to the standby database. Ensure that redo apply is active on standby server. Mount the database, but do not open it.
    SQL>alter system flush redo to 'target_db_name';
This statement flushes any unsent redo from the primary database to the standby database, and waits for that redo to be applied to the standby database.

6.2.2 Verify that the Standby Database has the most Recently Archived Redo Log File for each Primary Database Redo Thread

Query the V$ARCHIVED_LOG view on the target standby database to obtain the highest log sequence number for each redo thread:
    SQL>SELECT UNIQUE THREAD# AS THREAD, MAX(SEQUENCE#) OVER (PARTITION BY thread#) AS LAST from V$ARCHIVED_LOG;

6.2.3 Identify and Resolve any Archived Redo Log Gaps

On the standby database server, connect as sysdba to the standby database. Query v$archive_gap to determine whether there are missing archive logs:
    SQL>select * from v$archive_gap;
If this query returns a row, it indicates that at least one archived redo log is missing from the standby. If you still have access to your primary database, you can determine the full name of the redo logs by querying v$archived_log, using the low_sequence# and high_sequence# returned above, as shown in the commands below:
    SQL>select name from v$archived_log
    where thread# = <thread# from above query>
    and sequence# between < low_sequence# above> and <high_sequence# above>;
Locate the missing logs and copy them to the standby server's standby redo log destination, then register them:
    SQL>alter database register physical logfile '<filespec/name on standby>';
Note only one gap at a time is reported in v$archive_gap. If you find a gap and resolve it, repeat this process until no more gaps are reported.

6.2.4 Adjust the Standby Archive Destination Status

On the standby database server, change the initialization parameter for the destination state of the archive logs to be shipped from primary to standby from 'defer' to 'enable'. Make the changes in <RDBMS_ORACLE_HOME>/dbs/<STNDBY_CONTEXT>_ifile.ora, commenting out the 'defer' line.
    # log_archive_dest_state_2 = enable
    log_archive_dest_state_2 = defer

6.2.5 Stop Redo Apply and Finish Applying Redo Data

    Note For RAC Configurations: Shut down all other instances before you perform the following steps.
When all available logs are present and registered on the standby, stop the redo apply:
    SQL>alter database recover managed standby database cancel;
Finish the recovery session:
    SQL>alter database recover managed standby database finish;
When that completes, convert the physical standby to perform a primary database role:
    SQL>alter database commit to switchover to primary;
    SQL>shutdown immediate;
    SQL>startup pfile=?/dbs/init<SID>.ora
    Note: You should back up this database without delay, as you cannot recover any changes made after the failover without a fresh backup.

6.2.6 Update the TNS parameters for the New Standby Database Location

As the ORACLE user on the new-primary database server:
    1. Open the <TNS_ADMIN>/<STNDBY_CONTEXT>_ifile.ora file for editing.
    2. Change the value for the host name in the standby service definition to point to a new primary host.
    3. Save the changes and close the file.

6.2.7 Complete the Database Configuration

    1. Connect to the new primary database using SQL*Plus as the APPS user, and execute the following commands:
      SQL>exec fnd_net_services.remove_system('<SID>');
      SQL>commit;
      SQL>exec fnd_conc_clone.setup_clean;
      SQL>exec ad_zd_fixer.clear_valid_nodes_info
      ;
    2. As the ORACLE user on the new primary database server, use AutoConfig to complete the configuration for the primary operations, providing the APPS password when prompted:
      $ cd <ORACLE_HOME>/appsutil/scripts/<CONTEXT_NAME>
      $ ./adautocfg.sh
    3. When this completes, stop and start the listener on the new primary database server:
      SQL>lsnrctl stop <SID>
      SQL>lsnrctl start <SID>
To complete the application-specific configuration, follow the steps in 6.4 Configuring Application Tiers After a Role Transition (Switchover/Failover/Switchback).

6.3 Performing a Switchback to the Primary Following A Switchover/Failover

6.3.1 Switch Back to the Primary Site after a Switchover

    Note: Before proceeding, it is essential that you complete the ADOP patching cycle: prepare, apply, cutover, cleanup. When using an online patch or hotpatch run fs_clone to make sure that both run and patch file systems are at the same patchlevel.
After switchover to the standby and maintenance is complete, you need to switch back to the primary site. In this case, the pre-switchback configuration is as follows:
    Standby SitePrimary Site
    Primary databaseStandby database

Steps to perform the switchback to the primary site:
    1. Verify the primary database at the standby site is open and the standby database at the primary site is mounted.
    2. Verify all the redo logs are transferred to the standby and applied.
    3. On the primary site, check whether switchover_status from v$database is showing: TO STANDBY.
    4. Shutdown all the services on the primary application tier.
    5. On the primary database, issue the command:
      Note For RAC Configurations: Shut down all the other instances before you perform the following steps.

      SQL>alter database commit to switchover to physical standby;
    6. Adjust the LOG_ARCHIVE_DEST_STATE_2 defer at the standby site (primary database) and enable it at the primary site (standby database).
    7. Update the network configurations as mentioned in 6.1.5 Switch the selected standby database to the primary role.
    8. Shut down and mount the database as a standby at the standby site.
    9. Start the recovery by issuing the following commands at the primary site:
      SQL>alter database commit to switchover to physical primary;
    10. Shut down and start up the database at the primary site. Complete the database configuration by following 6.1.6 Complete the database configurations.
    11. Verify that the redo log files are shipping as expected. Refer to 6.1 Performing a Switchover for the commands.
    12. For application-specific configurations, follow the steps in 6.4 Application Tier Configuration After a Role Transition (switchover/failover/switchback).

6.3.2 Recreating the Original Primary Database Following a Failover

If you performed a failover to a standby database and then resolved the problem at the original primary site, you can now recreate the primary database on the original primary site:
    1. Make a consistent backup of the activated standby database at the standby site.
    2. Restore the backup created at the standby site to the primary database.
    3. Run AutoConfig on database tiers.
    4. Shut down and start up the database.
    5. On the original primary site, create or modify the initialization parameter file with the appropriate values.
    6. Create a new standby database at the original standby site. Follow the instructions in Sections 3, 4 and 5.
    7. For application-specific configurations, follow the steps in 6.4 Application Tier Configuration After a Role Transition (Switchover/Failover/Switchback).

6.4 Configuring Application Tiers After Role Transition (Switchover/Failover/Switchback)

There are two methods explained in this document to configure application tiers Configuration 1 and Configuration 2 in Section 5.
6.4.1 Configuring the Application Tiers after a role transition when the application tiers are configured using Configuration 1
6.4.2 Configuring the Application Tiers after a role transition when the application tiers are configured using Configuration 2

6.4.1 Configuring the Application Tiers after a role transition when the application tiers are configured using Configuration 1

6.4.1.1 Finish the Oracle E-Business Suite configuration on the Application Tiers
    Note: Ensure that you have added all application node names to sqlnet.ora file under $TNS_ADMIN on all database nodes.
After the primary database configuration is complete and its listeners have started, log in to each primary application tier server as the APPLMGR user, and run the final configuration steps on the run file systems:
    $ cd $INST_TOP/admin/scripts
    $ ./adautocfg.sh
Provide the APPSpwd when prompted.
On the patch file system go to $AD_TOP/bin/ and run the following command:
    $ perl adconfig.pl contextfile= <INST_TOP>/appl/admin/PATCH CONTEXT.xml -syncctx
6.4.1.2 Update the Host Name in FND_CONCURRENT_REQUESTS and FND_CONC_REQ_OUTPUTS
If you synchronize your concurrent manager log and out directories, you must update the host name in the fnd_concurrent_requests table to the standby server name:
    SQL>update apps.fnd_concurrent_requests
    set logfile_node_name = <new application tier node>,
    outfile_node_name = < new application tier node>
    where logfile_node_name = <old application tier node>
    and outfile_node_name = <old application tier node>;

    SQL>update apps.fnd_conc_req_outputs set file_node_name=<new applications tier node>
    where file_node_name=<old applications tier node>;
If you do not synchronize your concurrent manager log and out directories, blank out the host name in the fnd_concurrent_requests table to avoid network timeout errors:
    SQL>update apps.fnd_concurrent_requests
    set logfile_node_name = null,
    outfile_node_name = null;

    SQL>update apps.fnd_conc_req_outputs set file_node_name=' ';
If you run the later update, you must execute it before starting the concurrent managers on the system. If you do not execute it before starting the managers, you must add a where clause to limit the rows updated to those pointing to the old host names. This does not need to complete before you run the next step. However, if you let users on to the system before it is committed, they will get errors if they try to access a report's log or out file that was generated on the old primary system.
6.4.1.3 Perform the Cloning Finishing Tasks
Perform the Finishing Tasks outlined in My Oracle Support Knowledge < Document 1383621.1>, Cloning Oracle E-Business Suite Release 12.2 with Rapid Clone.
    • Instance specific profile options at other than site level (Rapid Clone updates the site level instance specific profile options)
    • Printer settings as necessary
    • Workflow configuration settings
    • APPLCSF variable if necessary
6.4.1.4 Direct Users to the New System
The standby system should be available to your users as your new primary system. Direct your users to the new URL.
6.4.1.5 Establish a new standby system
Perform this step if you have performed a failover. Failing over to the standby database (versus switching over) separates it from the old primary. You must create a new standby environment from this new system to again provide disaster recovery protection.
6.4.1.6 Re-point your CM log and out and native PL/SQL object directory rsync scripts (optional)
If you are keeping your concurrent manager log and out directories synchronized across the environments, set up your rsync scripts to move the files from the new primary server to the new standby server.

6.4.2 Configuring the Application Tiers after a role transition when the application tiers are configured using Configuration 2

The following configuration steps need to be performed only on the slave nodes. The admin server is replicated to another server on standby In this configuration.
6.4.2.1 Configure the Standby Admin Server Application Tier.
In this configuration, you need to replicate the primary admin server application tier. If necessary (depending on the method of disk replication that you are using) for the standby WLS Admin server, you may need to attach the storage to the new server.
    1. On the Standby admin server, change the hostname and IP address in /etc/hosts, /etc/sysconfig/network, and the /etc/sysconfig/network-scripts/ifcfg-eth0 files.
      In /etc/hosts add an entry with primary admin node name and IP address in the following format:
      IPAddress prim-node1.domain prim-node1
      In /etc/sysconfig/network:
      NETWORKING=yes
      NETWORKING_IPV6=yes
      HOSTNAME=prim-node1.us.oracle.com
      In /etc/sysconfig/network-scripts/ifcfg-eth0
      DEVICE=eth0 ONBOOT=yes
      BOOTPROTO=static
      IPADDR=<New IP Address>
      NETMASK=<netmask>
      GATEWAY=<gateway>
      TYPE=Ethernet
    2. Reboot the server.
    3. Modify the context variables in the context file and tnsnames.ora to connect to the new primary database. Follow all the steps in Section 6.4.2.1.
    4. Configure password less SSH to each of the slave nodes from this node.
6.4.2.2 Modify the context variables in run and patch context files to change the Database connectivity to new primary on slave application tiers.
    Note for Switchover : Run steps [1-6] on all slave nodes of both the primary and the standby so that all the application tiers will be connected to current primary database.
    Note for Failover : Run steps [1-6] on all slave nodes only as the primary application tiers are not available.

     
    1. In the context files, change s_dbhost and s_dbport to point to the first node of the current primary RAC database. - As we mentioned in the heading , not there in steps
      from:
      <dbhost oa_var="s_dbhost">prdbnode1</dbhost>

      to:
      <dbhost oa_var="s_dbhost">stdbnode1</dbhost>
    2. Mark the services as being enabled on standby application tier:
      <oa_service_group_status oa_var="s_root_status">enabled</oa_service_group_status>
      <oa_service_group_status oa_var="s_web_admin_status">disabled</oa_service_group_status>
      <oa_service_group_status oa_var="s_web_entry_status">enabled</oa_service_group_status>
      <oa_service_group_status oa_var="s_web_applications_status">enabled</oa_service_group_status>
      <oa_service_group_status oa_var="s_batch_status">enabled</oa_service_group_status> ******** Enable this if you are using ICM on that node.
      <oa_service_group_status oa_var="s_other_service_group_status">disabled</oa_service_group_status>
    3. Change the JDBC context variables s_apps_jdbc_connect_descriptor, s_apps_jdbc_patch_connect_descriptor to the new Database Nodes/SCAN Host in the context file:
      <jdbc_url oa_var="s_apps_jdbc_connect_descriptor">jdbc:oracle:thin:@(DESCRIPTION=(ADDRESS_LIST=(LOAD_BALANCE=YES)(FAILOVER=YES)
      (ADDRESS=(PROTOCOL=tcp) (HOST=<NEW SCAN NAME)(PORT=<NEW SCAN PORT>)))(CONNECT_DATA=(SERVICE_NAME=<Service Name>)))</jdbc_url>

      <patch_jdbc_url oa_var="s_apps_jdbc_patch_connect_descriptor">jdbc:oracle:thin:@(DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)
      (HOST=<New SCAN Host)(PORT=<New Scan port>))(CONNECT_DATA=(SERVICE_NAME=ebs_patch)(INSTANCE_NAME=r122b1)))</patch_jdbc_url>
      Note: If you have configured VIP instead of SCAN, change all the database host names to the new primary database nodes VIP host names and port.
    4. Update the EBS Database Source. Change directory to $EBS_DOMAIN_HOME/config/jdbc/ and edit EBSDataSource-<n>-jdbc.xml as follows:
      <url>jdbc:oracle:thin:@(DESCRIPTION=(ADDRESS_LIST=(LOAD_BALANCE=YES)(FAILOVER=YES)(ADDRESS=(PROTOCOL=tcp)(HOST=<SCAN HOST>)(PORT=<SCAN PORT>)))(CONNECT_DATA=(SERVICE_NAME=<service name)))</url>
    5. Run AutoConfig on the run file system on all of the nodes.
      $ $INST_TOP/admin/scripts/adautocfg.sh
    6. To sync the patch file system with above changes, check whether an ADOP cycle is open.
      $ adop -status
      • If the ADOP is open then execute AutoConfig on patch file system. (Skip this step if ADOP cycle is not open)
        $ $INST_TOP/admin/scripts/adautoconfig.sh
      • If there is no open ADOP cycle, source the patch file system go to $AD_TOP/bin/ and run the following command :
        $ perl adconfig.pl contextfile= <INST_TOP>/appl/admin/PATCH CONTEXT.xml -syncctx
        Next execute the following command from run file system on the admin/primary node:


        Note: Run this step on the admin/primary node only.
        $ adop phase=prepare,cutover
    7. Start the services on slave nodes

      $ adstrtal.sh <apps_user>/<apps_password>
6.4.2.3 After a Failover Remove the Primary Slave Nodes from the Configuration.
Following a failover to the standby site, the primary slave nodes do not exist in the new configuration and therefore it is essential that they are deleted using the following command from the admin server node:
$ perl <AD_TOP>/patch/115/bin/adProvisionEBS.pl ebs-delete-node -contextfile=<CONTEXT_FILE> -hostname=<node_name_to_be_deleted> -logfile=<LOG_FILE>
For further information on deleting a node, refer My Oracle Support Knowledge Document 1383621.1, Cloning Oracle E-Business Suite Release 12.2 with Rapid Clone.

Section 7: Oracle E-Business Suite Maintenance with Standby Database

This section describes how to apply an Oracle E-Business Suite patch on the primary, and incrementally update the standby.
7.1 Prepare to Apply an Application Patch
7.2 Patch the Primary
7.3 After Applying the Patch
    Note: If your standby database is running during a patch application, the database changes on primary will be automatically pushed or replicated on the standby. To protect both the integrity of the primary and standby from any problems during patch application, backup both the databases and Oracle E-Business Suite file system before patching. You may be able to achieve this more quickly with certain types of disk systems that use mirroring.

7.1 Prepare to Apply an Application Patch

7.1.1 Stop/remove the recovery delay if is set on the standby

If you have recovery delay set for redo log application on the standby, change the delay. On the standby database, run the following command as a privileged user:
    SQL>alter database recover managed standby database cancel;
    SQL>alter database recover managed standby database nodelay;
    SQL>alter database recover managed standby database disconnect from session;

7.1.2 Switch redo logs on the primary database

On the primary database server, log into SQL*Plus as sysdba and issue the following commands to switch logs; next discover the last log sequence number:
    SQL>alter system switch logfile;
    SQL>select sequence# -1 from v$log where status = 'CURRENT';
    Note For RAC Configurations: Repeat this command to switch the logs on each instance.

7.1.3 Ensure all log files are applied on the standby

On the standby database server, connect as sysdba and monitor the system to ensure that all log files have been shipped and applied:
    SQL>select sequence#, applied, to_char(first_time, 'mm/dd/yy hh24:mi:ss') first
    from v$archived_log
    order by first_time;

7.2 Patch the Primary

Open a patch cycle and apply the patch, either as a hotpatch or an online patch.
    Note: If using hotpatch mode, stop the recovery on the standby. With online patching, before starting the cutover phase, ensure that you shipped all the redo log files and applied them on the standby. Stop the redo shipping and recovery on to the standby, which will isolate it and maintain it's integrity.
      On the Standby stop the recovery using:
    SQL>alter database recover managed standby database cancel;
    On the Primary stop the redo shipping:
    SQL>alter system set log_archive_dest_state_<n>=defer;

7.3 After Applying the Patch

Restart the redo log shipping once you have checked that there are no issues.

7.3.1 Complete the patch cycle

It is essential that you complete the ADOP patching cycle: prepare, apply, cutover, cleanup. When using an online patch or hotpatch run fs_clone to make sure that both run and patch file systems are at the same patchlevel.

7.3.2 Syncronize the Primary and Standby Application Tiers

The primary and standby databases are in sync with the patch data, to syncronize the application tiers:
7.3.2.1 Put the Standby database into Snapshot Standby mode.
  1. Synchronize the primary and standby database by forcing a redo log switch on the primary (In the case of RAC, on all instances):
    SQL>alter system switch logfile;
  2. Follow all of the steps [1-10] in Section 5.1.1 to open the standby database in Snapshot Standby mode, so that the database can be updated by AutoConfig.
7.3.2.2 Run AutoConfig
Execute AutoConfig on each of the standby application tiers.
7.3.2.3 Apply the patches to the Standby using hotpatch mode with the nodatabaseportion option
Note: Ensure that you have closed the ADOP cycle on primary site before applying patches to the standby.
Apply all the patches which were applied to the primary using online patching on the standby.
    1. If the primary and standby run file systems are different, flip the file system on the Standby application tiers to be the same as the Primary using the following commands/steps.

      1. Upload the Patch file system context file to the Database. In a new terminal source PATCH environment and execute CtxSynchronizer to load the PATCH context file to the database.
        $ $ADJVAPRG oracle.apps.ad.autoconfig.oam.CtxSynchronizer action=upload contextfile=<full path to patch context file> logfile=/tmp/patchctxupload.log
      2. Now source the RUN environment and execute one of the following commands to flip the file system. If you only have one standby applications tier, run the following command on that node (which is the master node):
        $ perl $AD_TOP/patch/115/bin/txkADOPCutOverPhaseCtrlScript.pl -action=ctxupdate -contextfile=<full path to current run context file on standy> -patchcontextfile=<full path to current patch file system context file on standby> -outdir=<full path to out directory>
        If you have multiple standby applications tiers, run the following command on the master node, which includes the -multinode=Yes parameter:
         
        $ perl $AD_TOP/patch/115/bin/txkADOPCutOverPhaseCtrlScript.pl -action=ctxupdate -multinode=Yes -contextfile=<full path to current run context file on standby> -patchcontextfile=<full path to current patch file system context file on standby> -outdir=<full path to out directory>
        Note: If either of these commands fail, manually create the TXK_CTRL_ctxupdate directory in the outdir directory if it doesn't exist. Once the file system has been changed, run the environment file so that it now points to the new run file system.
    2. After flipping the filesystem, run AutoConfig to populate ADOP_VALID_NODES table before starting ADOP session.
       
    3. Copy the $PATCH_TOP to the application tier in the standby site.
       
    4. Apply the patches using hotpatch in the order that they were applied on the Primary. Use options=nodatabaseportion so that the they are applied without updating the database:
      $ adop phase=apply patches=patch number apply_mode=hotpatch options=nodatabaseportion
      Note: If you apply a patch using hotpatch mode, you could, but do not need to, run:  adop phase=fs_clone to synchronize the patch file system as this will be done during the next adop phase=prepare phase.

    5. If you have previously applied FMW Patches (for example FMW 10.1.2 Oracle Home patches) on the primary, also apply those patches to the current run file system on the standby site. In this case you will need to run the cloning command:  adop phase=fs_clone to synchronize the patch file system on the standby site.
       
    6. Synchronize the code on any other Application Tiers. 
    7. Put the snapshot standby database into Physical Standby mode by executing the steps in section 5.1.3.

7.3.3 Reconfigure the standby database file systems (optional)

If you had to synchronize the apps utilities on the database server in the previous step, you should also reconfigure the database server. As the ORACLE user on the standby database server, first stop the listener if it is running, then use the cloning toolkit to define the database tier topology at the standby site:
  1. Source the Oracle Applications environment and run the following command to create the file appsutil.zip in <INST_TOP>/admin/out
  2. $ perl <AD_TOP>/bin/admkappsutil.pl
  3. Copy appsutil.zip to the database servers and uncompress appsutil.zip under the <RDBMS ORACLE_HOME>
  4. $ cd <ORACLE_HOME>
    $ unzip -o appsutil.zip
  5. As the ORACLE user on the standby database server, first stop the listener if it is running, then use the cloning toolkit to define the database tier topology at the standby site - Answer the questions when prompted.
  6. $ lsnrctl stop <SID>
    $ cd < RDBMS_ORACLE_HOME>/appsutil/clone/bin
    $ perl adcfgclone.pl dbTechStack
  7. The step above starts the database listener for the primary services. You need to stop and restart it for the standby services:
  8. $ lsnrctl stop <SID>
    $ lsnrctl start < standby service name>

Appendix A: Oracle Net Files

This appendix is divided into a common section (Common Listener Configuration for RAC and Non-RAC) and then:
  • A1: TNS Alias Requirements in a non-RAC Configuration
  • A2: TNS Alias Requirements in an Oracle RAC Configuration
The examples in this section use the following convention where SID is same on both the primary and physical standby:

    Primary Physical Standby
    Oracle Net Service Name dg12dg12s
    SIDdg12dg12

Common Listener Configuration for RAC and Non-RAC:

A sample <TNS_ADMIN>/listener_ifile.ora file

##################################################################
#
# Created to define net services to support a Oracle Data Guard physical
# standby environment.
#
##################################################################
Standby LISTENER.ORA file when server is running as standby
dg12s =
(ADDRESS_LIST =
(ADDRESS=(PROTOCOL=TCP)
(HOST= <standby DB host>)
(PORT= <same as production>)
)
)
SID_LIST_dg12s =
(SID_LIST =
(SID_DESC =
(ORACLE_HOME= ) (GLOBAL_NAME=dg12s)
(SID_NAME=dg12)
)
)

STARTUP_WAIT_TIME_dg12s = 0
CONNECT_TIMEOUT_dg12s = 10 TRACE_LEVEL_dg12s = OFF

LOG_DIRECTORY_dg12s = <same as production>
LOG_FILE_dg12s = STDBY
TRACE_DIRECTORY_dg12s = <same as production>
TRACE_FILE_dg12s = STDBY
ADMIN_RESTRICTIONS_dg12s = OFF

A1: TNS Alias Requirements in a non-RAC Configuration
    Sample < TNS_ADMIN>/<CONTEXT_NAME>_ifile.ora file.

    ##################################################################
    #
    # Created to define net services to support a Oracle Data Guard physical
    # standby environment.
    #
    ##################################################################

    #
    # The Oracle Data Guard physical standby of primary runs on <standby host>.
    # Oracle Data Guard uses the tcp protocol only.
    #
    # This entry must point to the current standby server.
    # IT MUST BE CHANGED on switchover:
    dg12s=
    (DESCRIPTION=
    (ADDRESS=(PROTOCOL=tcp)
    (HOST= < standby DB host name>)
    (PORT= <Same PORT as primary)
    )
    (CONNECT_DATA=(SID=dg12)
    )
    ) #
    # Fetch Archive Log (FAL) service definition.
    # This entry can be set up for use when THIS server hosts a
    # standby database (thus will not need to be changed on switchover),
    # and should point to what would be the PRIMARY AT THAT TIME -
    # i.e. this is the fal_server alias used to communicate from the standby to primary.
    dg12=
    (DESCRIPTION=
    (ADDRESS=(PROTOCOL=tcp)
    (HOST= <prod DB host name when this is standby>)
    (PORT= <same port as primary)
    (CONNECT_DATA=(SID=dg12) ) )
     

A2: TNS Alias Requirements in an Oracle RAC Configuration

For Oracle RAC configurations, the entries for the TNS aliases should be as follows, where prod and stdby are the primary and standby service names, respectively. The entries must be the same on all nodes in the cluster as well as in the standby instances.
    Sample <TNS_ADMIN>/<CONTEXT_NAME>_ifile.ora file.

    ##################################################################
    #
    # Created to define net services to support a Oracle Data Guard physical
    # standby environment.
    #
    ##################################################################

    #
    # The Oracle Data Guard physical standby of primary runs on <standby host>.
    # Oracle Data Guard uses the tcp protocol only.
    #
    # This entry must point to the current standby server
    # IT MUST BE CHANGED on switchover:
    prod=
    (DESCRIPTION=
    (LOAD_BALANCE=NO)
    (FAILOVER=YES)
    (ADDRESS_LIST=
    (ADDRESS=
    (PROTOCOL=tcp)
    (HOST=<Primary DB Host1>)
    (PORT=<dbport>))
    (ADDRESS=
    (PROTOCOL=tcp)
    (HOST=<Primary DB Host2)
    (PORT=<dbport>)) )
    (CONNECT_DATA=(SERVICE_name=prod))
    )
    stdby=
    (DESCRIPTION=
    (LOAD_BALANCE=NO)
    (FAILOVER=YES)
    (ADDRESS_LIST=
    (ADDRESS=
    (PROTOCOL=tcp)
    (HOST=<Stndby DB Host1)
    (PORT=<dbport>))
    (ADDRESS=
    (PROTOCOL=tcp)
    (HOST=<Stndby DB Host2>)
    (PORT=<dbport>))
    )
    (CONNECT_DATA=(SERVICE_name=stdby)(UR=A))
    )

Appendix B: Using Oracle Applications Manager to Register Standby Database

    1. From a client, connect to OAM to register the standby database server as a node. Navigate as follows:

      Site Map > Administration > System Configuration > Hosts > Register (button under Other Hosts)

    2. Next, use OAM to add this host to the list of hosts that need access to the database:

    3. Applications Dashboard > Security > Manage Security Options > Enable Restricted Access > Run Wizard

    4. Select the host you just added, and click 'Continue'.

    5. If the displayed list is correct and includes your new host, click 'Submit'.

Appendix C: Example Standby Database Commands

Example for standby redo log files:
    SQL>Alter database add standby logfile group 4 ('/d1/MAABCU/primary/dg12data/log5.dbf') size 50M;
    SQL>Alter database add standby logfile group 5 ('/d1/MAABCU/primary/dg12data/log6.dbf') size 50M;
    SQL>Alter database add standby logfile group 6 ('/d1/MAABCU/primary/dg12data/log7.dbf') size 50M;
    Note For RAC Configurations: Add standby redo log files for each thread using the following example command that uses a 300MB file.
    SQL>alter database add standby logfile thread 1 group 10 ('+DATA/dbfiles/stdbylog1.dbf') size 300M;

Appendix D: Using RMAN to Create a Physical Standby Database

Perform the following steps when using RMAN to create a physical standby database :
    1. Ensure that you have completed steps 4.1 to 4.4 in Section 4.
    2. Ensure that you have completed the network configurations and started the standby database listeners.
    3. Start the standby database in nomount mode
      sqlplus / as sysdba
      SQL>startup nomount;
    4. Verify the connections to primary and auxiliary database.
      $ sqlplus sys/<password>@<TNS alias to primary>
      $ sqlplus sys/<password>@<TNS alias to standby> as sysdba
    5. Create initialization parameter file for standby. Run the following command on the primary database server:
      SQL>create pfile from spfile
    6. Copy init_<SID>.ora from the primary to the standby database server.

    7. Change the db_unique_name to the standby db_unique_name; this should be different from the primary database.

      For example, db_unique_name=dg12s

    8. For log_archive_dest_2, specify the primary db_unique_name.

      For example, log_archive_dest_2='SERVICE=dg12 LGWR ASYNC DB_UNIQUE_NAME=dg12'
      db_unique_name should specify to ship redo logs from the standby site to the primary site after switch over.

    9. Connect as sysdba and issue the following command to start up but not mount the standby database.
      SQL>startup nomount pfile=<pfile created in above step>
    10. Connect to target and auxiliary databases using RMAN:
      $ rman target sys/manager@dg12 auxiliary sys/manager@dg12s
      (In this example, dg12 = primary service name and dg12s = standby service name)
      Recovery Manager: Release 11.2.0.1.0 - Production on Wed Jan 20 03:16:56 2010 Copyright (c) 1982, 2009, Oracle and/or its affiliates. All rights reserved.
      connected to target database: dg12 (DBID=3753412759)
      connected to auxiliary database: dg12 (not mounted)
      RMAN>
    11. Execute the RMAN DUPLICATE command on standby:
      RMAN>DUPLICATE TARGET DATABASE FOR STANDBY FROM ACTIVE DATABASE
      DORECOVER
      SPFILE SET "db_unique_name"="dg12s"
      SET LOG_ARCHIVE_DEST_2="service=dg12 ASYNC REGISTER VALID_FOR=(online_logfile,primary_role)DB_UNIQUE_NAME=dg12"
      SET FAL_SERVER="dg12" COMMENT "Is primary"
      SET DIAGNOSTIC_DEST=$ORACLE_HOME/admin/<STNDBY_CONTEXT_NAME> COMMENT "DIAGNOSTIC Destination on standby server"
      SET UTL_FILE_DIR= <Appropriate value depending on standby context name
      >
      NOFILENAMECHECK;
      In the above example, RMAN automatically copies the server parameter file to the standby host and then starts the auxiliary instance with this file.

Appendix E: RAC RMAN Clone Example

If you are using Rapid Clone to clone an Oracle RAC primary database to a standby, use the following commands for RMAN backup and restore.
  1. Take a hot backup using RMAN as given in the following example:
    configure device type disk parallelism 5 backup type to backupset;
    configure maxsetsize to 4200m;
    backup as backupset tag 'RapidClone_RAC' database format '/oradata/MAABCU/RAC12STDBY/backupsets/%U';
    backup as backupset tag 'RapidClone_RAC' archivelog all format '/oradata/MAABCU/RAC12STDBY/backupsets/%U';
  2. Create standby database using RMAN. On the standby system first node, start up the database in nomount mode (using a pfile) and run the following command:
    $ rman target sys/manager@prod auxiliary /
    Recovery Manager: Release 11.1.0.7.0 - Production on Thu Mar 11 02:17:43 2010
    Copyright (c) 1982, 2007, Oracle. All rights reserved.
    connected to target database: PROD (DBID=3908691352)
    connected to auxiliary database: PROD (not mounted)
    RMAN> DUPLICATE TARGET DATABASE FOR STANDBY;
  3. Enable recovery on the node that is to be used for the recovery process.
  4. $ sqlplus "/as sysdba" SQL> alter database recover managed standby database disconnect from session;

Appendix F: Using Data Guard Broker (DGMGRL) to Manage Standby Databases

The Oracle Data Guard broker is an easy to use interface to manage standby databases. It is easy to perform role transitions with a single command either for switchover or failover. This section covers DGMGRL - the command line interface used to manage standby databases.
F1. Prerequisites
F2. Configure Data Guard Broker
F3. Role Transitions

F1. Prerequisites:

    • Prior to using Data Guard broker, the standby database should be configured.
    • You must be using a server parameter file (spfile).
    • The Data Guard broker starts database instances during switchover or failover using a statically registered service name. Therefore, it is necessary to add a static descriptor to the custom listener.ora file [<TNS_ADMIN>/<CONTEXT_NAME>_ifile.ora].
       
      If you choose the DGMGRL default, then configure as per option 1 below; if you are using a different static descriptor, then set the DGMGRL StaticConnectIdentifier property, as per option 2.
       
      • Option 1:
        The default option is for the broker to assume the service <db_unique_name>_DGMGRL.<db_domain> has been statically registered with the listener of each instance. Add a SID_DESC entry as seen below:
        (SID_DESC =
        (GLOBAL_DBNAME=<DB_UNIQUE_NAME>_DGMGRL.us.oracle.com)
        (ORACLE_HOME= <ORACLE_HOME>)
        (SID_NAME = <SID>)
        )
      • Option 2:
        Set the StaticConnectIdentifier property of both the primary and standby databases to a TNS alias that resolves to a statically registered descriptor.
        DGMGRL>edit database <Primary Database> set property StaticConnectIdentifier='<dg_prim>' where dg_prim is TNS alias to connect the Primary
        DGMGRL>edit database <Standby Database> set property StaticConnectIdentifier='<dg_stndby>' where dg_stndby is TNS alias to connect the Standby
        Add the two TNS aliases to ifile ( <sid>_<node>_ifile.ora) under TNS_ADMIN on both standby and primary.

        For example:
        dg_prim=(DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=<primary host>(PORT=<port>))(CONNECT_DATA=(SID=<sid>)(SERVER=DEDICATED)))
        dg_stndby=(DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=<standby host>(PORT=<port>))(CONNECT_DATA=(SID=<sid>)(SERVER=DEDICATED)))

F2. Configure Data Guard Broker:

    1. Start the Data Guard broker on both primary and standby databases. The Data Guard broker will create two files under the location specified by the initialization parameter DG_BROKER_CONFIG_FILEn. The default location will be $ORACLE_HOME/dbs/directory.
      alter system set dg_broker_start=TRUE.
    2. Configure the Data Guard broker using DGMGRL.
      dgmgrl sys/****@<primary database alias>
      DGMGRL>CREATE CONFIGURATION '<Any Name>' AS PRIMARY DATABASE IS '<db_unique_name>' CONNECT IDENTIFIER IS <Primary database alias>;
    3. Add standby database using following command:
      ADD DATABASE '<standby unique name>' AS CONNECT IDENTIFIER IS <Standby TNS Alias>;
    4. Check the configuration using "Show Configuration".
       
    5. View the configuration using Show Configuration command.
       
    6. Set the following Data Guard broker properties:
       
      1. Set the configuration protection mode to maximum availability. At any time, you can change the protection mode of configuration. Note that this protection mode requires that there be at least one standby database configured to use standby redo log files, with its LogXptMode configurable database property set to SYNC on both primary and standby.
        DGMGRL>EDIT database <database name> set property LogXptMode='SYNC'
        DGMGRL>EDIT CONFIGURATION SET PROTECTION MODE AS MAXAVAILABILITY;
      2. Do not enable FAST_START_FAILOVER as automatic failover is not supported.

F3. Role Transitions

F3.1 Switchover
    1. Verify that the primary and the target standby databases are in the following states - primary TRANSPORT-ON and standby APPLY-ON.

      On PrimaryOn Standby
      DGMGRL> show database dbbrok
      Database - dbbrok
      Role: PRIMARY
      Intended State: TRANSPORT-ON
      Instance(s): dbbrok
      Database Status: SUCCESS
      DGMGRL> show database stndby
      Database - stndby
      Role: PHYSICAL STANDBY
      Intended State: APPLY-ON
      Instance(s): dbbrok

      Database Status: SUCCESS

    2. Issue switchover command.
      DGMGRL>switchover to <standby database>;
    3. Verify the switchover has been successful.
      show configuration
      Databases:
      stndby - Primary database
      dbbrok - Physical standby database
    4. To complete the switchover, follow the steps in Section 6 from 6.1.6 Complete the database configurations.

F3.2 Switchback

Follow the same steps from the above section, but change the database name to switchover.
    For example
    DGMGRL> switchover to dbbrok; --> where dbbrok is current standby after a switchover.

F3.3 Failover

There are two types of failover using Data Guard broker:
    • Manual failover
    • Automatic failover using the fast start failover option (not supported in Oracle E-Business Suite)

Manual Failover

    1. Connect DGMGRL to the standby database.
      dgmgrl sys/manager@<Standby Database alias>
      DGMGRL> failover to <Standby Database>
      Performing failover NOW, please wait...
      Failover succeeded, new primary is "stndby"
    2. To complete the failover, follow the steps in Section 6 from 6.2.7 Complete the database configuration.

Automatic Failover

Automatic failover is not supported in Oracle E-Business Suite, as you need to run AutoConfig before bringing the standby environment online. At this time, this section will be updated when post failover configurations are automated.

Appendix G: Creating Non-RAC Standby for RAC Primary

This appendix describes the steps to create a non-RAC standby.

G1. Configure a Primary RAC to Create a Non-RAC Standby

Follow the instructions given in Section 3: Preparing the Primary Database for Standby Database Creation to configure the primary RAC database.

G2. Create a Physical Standby

  1. Copy the ORACLE_HOME to the Standby database server.

    Copy the Oracle Home file system to the standby database server. If you have implemented native PL/SQL compilation, set up an rsync job from the primary database server to the standby database server for the file system directories that contain the compiled objects. The standard location for this is RDBMS_ORACLE_HOME/plsql/nativelib.
     
  2. Backup the primary RAC to backupsets using RMAN.

    You should take the backup of the primary RAC database and copy the backupsets to standby server. Refer to step 1 of Appendix E: RAC RMAN Clone Example for complete RMAN instructions.
  3. Perform file-based configuration on the standby database server.

    After the database software copies are complete (which can be done before the database itself is finished copying), log into the standby database server as the oracle user and execute the following commands to update the file system configurations for the new environment. As your environment scripts are not yet set up, you will need to manually resolve the reference to RDBMS_ORACLE_HOME.
    $ cd RDBMS_ORACLE_HOME/appsutil/clone/bin
    $ perl adcfgclone.pl dbTechStack
    Answer the questions when prompted. If you receive any errors registering the new ORACLE_HOME, follow the instructions given by the script to correct them.

    For Target instance is a Real Application Cluster (RAC) instance (y/n) [y]: n

    Enter 'n' as the standby is non-RAC.

    Note: Re-link oracle with rac_off option. If adlnkoh.sh fails then run adcfgclone again.

    Your oracle user environment scripts are now ready to use. Source them for the next steps, using the appropriate OS command.

    For example, in sh or ksh on UNIX:

    $ cd RDBMS_ORACLE_HOME
    $ ./STNDBY_CONTEXT.env
    Modify the initialization parameter as per 4.6 Modify the Database init.ora Parameters on the standby server.
     
  4. Stop the standby listener and configure for net redo transmission

    Stop the listener and modify the <StandbySID>_ifile.ora to configure net redo transmission, refer creating physical standby non-RAC Section 4: Creating a Physical Standby Database.
     
  5. Startup instance in nomount.
    sqlplus / as sysdba
    startup nomount
  6. Create the standby database using RMAN.
    rman target sys/<passwd>@<primaryservice> auxiliary sys/<passwd>@<stndbyservice>
    rman>DUPLICATE DATABASE FOR STANDBY;
    After the above the command execution the database will be in mount state running with the initialization parameter file (pfile).

    Put the standby database in 'managed recover' mode.
    SQL>alter database recover managed standby database disconnect from session;
    For the applications tier configuration, follow the steps given in Section 4: Creating a Physical Standby Database starting at step 4.8 Start shipping redo from the primary to the standby database server up to the end of the section. Next perfom all the steps given in Section 5: Configuration on Application Tiers After Standby Database is Enabled.

Appendix H: Handling Online Patching Following A Standby Failover Event

Oracle E-Business Suite 12.2 supports online patching. However, if there is a failover event during a patch cycle, there is no option to resume the patch on the standby. If you want to apply the patches to the standby, you will first need to run a script to modify the status of the online patching, and then abort the current patch cycle. Steps are included for both possible standby application tier configurations.

For Configuration 1: The standby Application Tiers are connected to the standby database.

The steps to perform on a failover during patching are as follows:
    1. Make sure that you have applied all the patches listed in PreRequisite section.
    2. Before running fnd_conc_clone.setup_clean in the next step, run the following query to get the appltop_id for each of the primary nodes:
    3. SQL>SELECT aat.appl_top_id,node_name
      FROM FND_OAM_CONTEXT_FILES focf, AD_APPL_TOPS aat
      WHERE aat.name=EXTRACTVALUE(XMLType(focf.TEXT),'//APPL_TOP_NAME')
      AND focf.node_name=<Primary Node Name>
      AND EXTRACTVALUE(XMLType(TEXT),'//file_edition_type') = 'run'
      ;
    4. Execute the instructions in step 6.2.7 and also all the steps in Section 6.4.
    5. Get the appl_top_id for each of the standby nodes using the query in step 2; substitute the <standby node name> in place of the <primary node name>.
    6. In order to be able to restart a patching cycle, you need to change the node names from the primary to the standby(current primary). As the APPS user execute the following:
    7. SQL>exec AD_ZD_FIXER.fix_adop_repo_tables(<Primary appltop_id (from Step 2)>,<Primary Node Name>, <Standby appltop_id (from step 4)>,<Standby Node Name >);
      Note: In case of a scale down configuration, there will not be same number of application tier nodes on primary site and standby site. Use the following command for scale down configuration:
      SQL>exec AD_ZD_FIXER.fix_adop_repo_tables(<Primary appltop_id (from Step 2)>,<Master Node on Primary Site>, <Standby appltop_id (from step 4)>,<Master Node on Standby Site>);

    8. The EBS Technology Codelevel Check needs to be run on the database node. Download and apply Patch 17537119. Execute checkDBpatch.sh on the database node.

    9.  
      Note: If you do not do this, you will be prompted when you run prepare.

       
    10. On the standby, abort online patching that was initiated on the primary site by executing the following command:
    11. $ ADOP phase=abort
You will then be able to restart the patch cycle.

For Configuration 2: The standby Application Tiers are connected to the primary database.

  1. Perform the failover steps detailed in Section 6.2 to configure the application tiers
  2. On the standby(New Primary), abort online patching that was initiated on the primary site by executing the following command:
  3. $ ADOP phase=abort
You will then be able to restart the patch cycle.

Comments

Post a Comment

Popular posts from this blog

19c ORACLE HOME Cloning -Linux/Solaris

  Cloning an Oracle home involves creating a copy of the Oracle home and then configuring it for a new environment. If you are performing multiple Oracle Database installations, then you may want to use cloning to create each Oracle home, because copying files from an existing Oracle Database installation takes less time than creating a new version of them. This method is also useful if the Oracle home that you are cloning has had patches applied to it. When you clone the Oracle home, the new Oracle home has the patch updates which is already applied on oracle home. Steps to clone an Oracle home step 1 : Stop Services Stop all processes related to the Oracle home. Step 2 : Create a ZIP or TAR file with the Oracle home (/u01/app/oracle/product/19.0.0/dbhome_1)    Use ROOT user for ZIP and UNZIP  # zip -r dbhome_1.zip /u01/app/oracle/product/19.0.0/dbhome_1 TAR option: # tar -cvf dbhome_1.tar /u01/app/oracle/product/19.0.0/dbhome_1 Step 3: s cp zip/tar to target s...

adgrants issue with 19c PDB database

 adgrant - system.fnd_oracle_userid ORA-06550: line 1094, column 17 SP2-0625: Error printing variable "result_set" adgrants issue with 19c PDB database  Connect to a multitenant 19c database and follow below steps to resolve ORA error 1- source the cdb env file  2-set the  ORACLE_PDB_SID=<PDB_NAME>   3- connect as  sqlplus / as sysdba 4- go to location of adgrant and execute cd /u01/app/oracle/product/19.0.0/dbhome_1/appsutil/sql    SQL>  @adgrants.sql APPS Note: ignore warnings during adgrant session a d.plsql.ad_zd_sys.GIVE_PRIVILEGE