Upgrade from 19c to 23ai using AutoUpgrade – Exadata Database Service on Dedicated Infrastructure (ExaCS) – Part 7 – Fallback options – Downgrade

In case you missed the previous posts:

  • Part 1 – Refreshable Clone PDBs
  • Part 2 – Non-CDB 19c to 23ai PDB
  • Part 3 – 19c PDB to 23ai
  • Part 4 – Fallback options
  • Part 5 – Target_pdb_copy_option
  • Part 6 – Create a 23ai Database with Compatible set to 19c

In Part 3, I covered the process of performing a traditional upgrade from a 19c PDB to 23ai.
In Part 5, I covered the process of performing a traditional upgrade from a 19c PDB to 23ai with the fallback to 19c option when you duplicate the pdb datafiles.
Now, I’ll demonstrate how to fallback from 23ai to 19c using catdwgrd.sql / dbdowngrade.
This technique has one caveat: It requires the 23ai database to be created with the Compatible parameter set to 19. You can see how to do this in part 6.

Let’s create an AutoUpgrade config file:

global.autoupg_log_dir=/u02/AutoUpgrade
global.keystore=/u02/keystore
upg1.source_home=/u02/app/oracle/product/19.0.0.0/dbhome_1
upg1.target_home=/u02/app/oracle/product/23.0.0.0/dbhome_1
upg1.sid=DB191
upg1.pdbs=PDBAZ
upg1.target_cdb=DB23C191

As you can see, I have a 23ai database with the Compatible parameter set to 19:

Another thing to consider is checking the timezone version. Sometimes you will need to use the following to be able to fallback:

upg1.timezone_upg=no

Let’s load the TDE encryption password for both databases:

java -jar autoupgrade.jar -config DB191.cfg -load_password

Let’s run the Analyze mode:

java -jar autoupgrade.jar -config DB191.cfg -mode analyze

As demonstrated, no manual steps are required for this test:

Let’s run the Deploy mode to start the migration:

java -jar autoupgrade.jar -config DB191.cfg -mode deploy

As shown below, the migration from 19c to 23ai was successful:

The PDB PDBAZ is now plugged into the 23ai CDB:

Fallback

Now that I’m using 23ai, let’s consider a fallback scenario. After running my production environment on this version for a while, I decide to revert to the original 19c database. Since I can’t simply restore a backup due to the production workload already processed in the new database, a downgrade is one of the viable options.

The overall steps are:

  • Taking a backup of my PDB before starting
  • Cleaning out the UNIFIED_AUDIT_TRAIL on the PDB (If needed)
  • Starting the PDB in downgrade mode
  • Calling PDB downgrade
    • $ORACLE_HOME/bin/dbdowngrade -c ‘PDBAZ‘
  • Shut down and unplug the PDB from the 23ai
  • Plug back the PDB on the 19c cdb fallback database
  • Run catrelod.sql
  • Run utlrp.sql
  • Collect the dictionary stats

Here you can find more details, including how to clean/backup the Unified Audit Trail.

Let’s start up the pluggable database in downgrade mode:

ALTER PLUGGABLE DATABASE PDBAZ CLOSE INSTANCES=ALL;

ALTER PLUGGABLE DATABASE PDBAZ OPEN DOWNGRADE;

Let’s downgrade the database, you have several options:

  • Run the catdwgrd.sql script while connected to the PDB
  • Use the catcon.pl utility
  • Execute the dbdowngrade command

Additionally, you can define the degree of parallelism based on your preferences. Choose the method that works best for you.

$ORACLE_HOME/bin/dbdowngrade -c 'PDBAZ'

As you can see above, the downgrade finished and no errors were returned.

Now, the next step is to unplug this PDB from 23ai CDB and plug it back into the 19c CDB:

ALTER PLUGGABLE DATABASE PDBAZNEW CLOSE instances=ALL;

ALTER PLUGGABLE DATABASE PDBAZNEW UNPLUG INTO '/u02/pdb_files/PDBAZNEW.xml' ENCRYPT USING *******;

DROP PLUGGABLE DATABASE PDBAZNEW KEEP DATAFILES;

You can find more about pdb_unplug_encrypt here.
Using the “ENCRYPT USING” parameter It will help you when getting errors like:

  • ORA-46680: master keys of the container database must be exported
  • ORA-28374: typed master key not found in wallet
  • ORA-46609: export or import secret is invalid or missing

Let’s plug it back to the 19c CDB:

CREATE PLUGGABLE DATABASE PDBAZNEW USING '/u02/pdb_files/PDBAZNEW.xml' NOCOPY KEYSTORE IDENTIFIED BY ******* DECRYPT USING ******;

ALTER PLUGGABLE DATABASE PDBAZ OPEN UPGRADE;

Let’s run the catrelod.sql, to recreate the database objects for the 19c version:

ALTER SESSION SET CONTAINER=PDBAZ;

set termout off
spool /u02/pdb_files/catrelod.log
@?/rdbms/admin/catrelod.sql

Let’s restart, recompile and refresh the stats:

ALTER PLUGGABLE DATABASE PDBAZ CLOSE INSTANCES=ALL;
ALTER PLUGGABLE DATABASE PDBAZ OPEN;
@?/rdbms/admin/utlrp.sql
EXEC DBMS_STATS.GATHER_DICTIONARY_STATS;

Let’s check the state of the Oracle Data Dictionary:

SELECT COMP_ID, COMP_NAME, VERSION, STATUS FROM DBA_REGISTRY ORDER BY MODIFIED;

Stay tuned for the next post.

Links to the Complete Blog Post Series on AutoUpgrade for ExaCS/ExaCC:

  • Part 1 – Refreshable Clone PDBs
  • Part 2 – Non-CDB 19c to 23ai PDB
  • Part 3 – 19c PDB to 23ai
  • Part 4 – Fallback options
  • Part 5 – Target_pdb_copy_option
  • Part 6 – Create a 23ai Database with Compatible set to 19c
  • Part 7 – Downgrade