| Lesson 3 | Recovery After an Interrupted Online Backup |
| Objective | Diagnose an interrupted online backup in Oracle AI Database 26ai and choose between ending backup mode and performing media recovery. |
A database can fail while a backup is running, but the backup job's failure does not identify the recovery procedure. First determine which backup method was used, whether the original datafiles survived, and whether anyone restored files before you arrived. These facts decide whether the problem is an unfinished user-managed backup, a media failure, or simply an unsuccessful backup job.
This lesson focuses on user-managed online backups that place read/write tablespaces in backup mode with BEGIN BACKUP. It uses a single-instance primary container database (CDB) in ARCHIVELOG mode, with the current control files and server parameter file still available. RMAN backup failures are discussed separately because RMAN does not require this user-managed backup mode.
The central rule is straightforward: end backup mode only when the affected files are known to be current. If they were restored from backup, recover them. If their history is uncertain, investigate and follow the recovery procedure rather than assuming that clearing backup mode will make them current.
During a user-managed online backup, the datafile header's checkpoint information is held at the backup starting point while normal changes to database blocks continue. The file is not a static, frozen copy of the application data. Oracle generates additional redo information so a properly captured backup can be recovered despite blocks changing while the operating system copies them.
END BACKUP ends that mode and allows the checkpoint information to advance. It does not reconstruct missing blocks, repair storage corruption, or complete a partially copied backup file. Likewise, it does not apply all changes that an older restored file needs to become current.
Oracle documents two relevant interruptions: the copy may have completed but the script failed to issue END BACKUP, or an instance failure may have interrupted the copy itself. Both can leave files marked as being in backup mode, but they have different implications for the backup copy's usability.
Review the backup script or RMAN log, the alert log, and storage errors. Establish whether the database instance stopped, whether only the copy process failed, and whether the source files remain accessible. An instance crash with intact files is different from a failed disk containing a datafile.
| Observed situation | Appropriate direction |
|---|---|
| RMAN job failed; database files remain intact | Correct the job failure and assess the completed backup material. Do not assume that END BACKUP is required. |
| User-managed backup interrupted; affected source files are confirmed current | Use the applicable END BACKUP procedure and verify that the database can resume operation. |
| Affected files were restored or their provenance is uncertain | Follow the recovery procedure and obtain the required redo rather than clearing backup mode on an assumption. |
| Source datafiles are missing or damaged | Restore or otherwise reconstruct the affected files as appropriate, then perform media recovery. |
| Control files or required redo were also lost | Use the corresponding recovery scenario; the simple interrupted-backup procedure is insufficient. |
Coordinate with the owner of the backup job before ending backup mode. If a copy is still running, stop or resolve it through the backup procedure before changing the state it depends on. Avoid blindly terminating unrelated database or storage processes.
Confirm that archiving was enabled during the period needed for recovery. Changing to ARCHIVELOG after an incident cannot recreate overwritten redo. Also verify access to relevant backups, archived and online redo, and encryption resources where applicable.
A DBA starts a user-managed online tablespace backup. The instance crashes before the script issues END BACKUP. On restart, Oracle mounts the database but reports that a datafile needs media recovery. The DBA considers taking the tablespace offline so the database can open.
That is not the necessary first step. When the problem is a current file left in backup mode, Oracle provides an END BACKUP procedure while the database is mounted. There is no need to create a circular dependency in which the DBA must open the database before clearing the condition that prevents opening.
Errors such as the following identify the affected file, but they do not prove that leftover backup mode is the only problem. This is illustrative output, not a required file number:
ORA-01113: file 12 needs media recovery
ORA-01110: data file 12: '/u01/oradata/ORC1/APPPDB/app_data01.dbf'
Before choosing END BACKUP, confirm from the incident history that the affected file is the original current file. A matching pathname does not prove this: an administrator might have copied an older backup over that path.
Use SQL*Plus with an authorized administrative account. For this mounted-CDB example, connect to CDB$ROOT as an appropriately privileged common user, such as an authorized SYSDBA account. Verify the instance and container rather than relying only on the shell environment.
SHOW CON_NAME
SELECT instance_name, status, database_status
FROM v$instance;
If the instance is down, mount it with STARTUP MOUNT;. If it is already mounted, do not repeat STARTUP. If it is started but not mounted, mount it using the appropriate administrative procedure. Do not shut down a functioning open database merely to reproduce this example.
Once the CDB is mounted, inspect its identity and the affected files:
SELECT name, database_role, open_mode, log_mode
FROM v$database;
SELECT con_id, file#, status, change#, time
FROM v$backup
WHERE status = 'ACTIVE'
ORDER BY con_id, file#;
SELECT con_id, file#, name, status
FROM v$datafile
ORDER BY con_id, file#;
ACTIVE in V$BACKUP means backup mode. It does not prove that a copy process is running, that a backup finished, or that the file is undamaged. Use CON_ID and the datafile inventory to identify the owning container and pathname. The file number in an example must never substitute for that inspection.
The view has further limitations. A restored or recreated control file may not contain the information required to report backup status accurately. For restored datafiles, the status can describe the older copy rather than the latest file state. Treat the view as evidence to combine with file provenance and diagnostic messages, not as an independent certificate that END BACKUP is appropriate.
This path applies when the affected files are confirmed to be current, none of them was restored from backup, and the incident assessment has not identified a separate media-recovery requirement. The backup copy process must no longer depend on those files remaining in backup mode.
SQL*Plus at CDB$ROOT, CDB mounted but not open:
ALTER DATABASE END BACKUP;
In the documented mounted-database recovery procedure, this ends backup mode for the affected files. Check the result before attempting to open:
SELECT con_id, file#, status
FROM v$backup
WHERE status = 'ACTIVE'
ORDER BY con_id, file#;
No ACTIVE rows is the expected backup-mode result for this scenario. It does not prove that all recovery requirements have disappeared. If the command or subsequent checks report another error, investigate that error instead of treating the state change as a repair of all database problems.
After successfully resolving the backup-mode condition and confirming that no other recovery issue remains, open the CDB normally:
ALTER DATABASE OPEN;
Use normal application procedures to open the required PDBs and restore services. Opening the CDB does not by itself demonstrate that every application PDB is open. Do not add RESETLOGS to this sequence: the scenario has not restored a backup control file or deliberately performed incomplete recovery.
If an affected file came from a backup, it can lack changes already present elsewhere in the database. Clearing its backup-mode state does not supply those changes. Oracle also documents recovery as the alternative when you are unsure whether a restore occurred.
With the current control file, usable datafiles, and the required redo available, the mounted-CDB SQL*Plus recovery path is:
RECOVER DATABASE;
Supply the required redo and inspect the recovery output. If a datafile is missing, an incomplete copy was restored, or the file is physically damaged, obtain an appropriate usable file before expecting recovery to succeed. The preceding lost-datafile lesson explains restore and recovery choices.
Recovery may scan redo from the beginning of the online backup. That extra work is justified when older datafile contents need to be advanced. Required redo may include online logs as well as archived logs; a later sequence does not replace a missing earlier interval.
After recovery succeeds, check for remaining ACTIVE entries and unresolved errors. With complete recovery and the current control file in this scenario, open normally using ALTER DATABASE OPEN;. If recovery stops with an error or cannot obtain required redo, do not treat cancellation as successful completion.
The original generic sequence using RECOVER DATABASE USING BACKUP CONTROLFILE followed by OPEN RESETLOGS belongs to a different set of prerequisites. Use backup-control-file recovery only when that is the actual situation. Restoring a control file unnecessarily introduces additional recovery requirements.
A failed copy script can leave the database open while a tablespace remains in backup mode. After confirming that its files are current and that the intended copying has stopped or completed, end backup mode at the appropriate scope. The database-wide mounted procedure above should not be copied into this open-database case.
For example, in SQL*Plus connected to the affected open PDB, after confirming that APP_DATA is the intended tablespace:
ALTER TABLESPACE app_data END BACKUP;
A file-specific END BACKUP form is another option when the file and scope are established. These are alternatives to the relevant cleanup operation, not steps to execute after every RMAN job. Recheck backup-mode status and the alert log afterward.
In a multitenant database, the connection context determines which container you administer. RAC additionally requires coordination with cluster management and other instances. The single-instance example is not a general instruction to clear backup mode whenever one RAC node fails.
Once the database is open, verify its state and the application PDBs from an authorized root session:
SELECT name, open_mode FROM v$database;
SELECT con_id, name, open_mode FROM v$pdbs ORDER BY con_id;
Review the alert log and perform application checks on the affected data. If errors suggest physical damage, use targeted diagnostic or RMAN validation appropriate to the problem. Header status, block validation, and business-level consistency checks answer different questions; none should be presented as a universal guarantee.
Then assess the backup independently. A partly copied datafile must not be treated as a completed datafile backup. However, a multi-file job that failed partway through may contain some fully completed copies. Establish which files completed successfully, whether their backup procedure was valid, and whether the required redo is retained. An interrupted job does not automatically prove that every earlier completed copy is unusable.
Archiving current redo can be part of completing a valid user-managed backup strategy, but a later log switch cannot repair an incomplete datafile copy. Retain the logs required by the usable copies and arrange a new verified backup to restore the intended coverage. Preserve earlier usable backups until the replacement and retention requirements are satisfied.
RMAN online backups do not require BEGIN BACKUP. If an RMAN job fails because of space, channels, credentials, or media access, address that cause and inspect the job's output. Assess completed backup sets or copies and rerun the appropriate backup operation.
If an instance or storage failure happened at the same time, handle that database incident according to the files and redo actually affected. RMAN's lack of user-managed backup mode does not eliminate ordinary instance recovery or media recovery. Conversely, the presence of ACTIVE rows during an RMAN job can indicate a separate user-managed operation; it is not evidence that RMAN required them.
For user-managed scripts, record each copy's completion, ensure the cleanup procedure matches the actual file state, and alert on failures. Automated END BACKUP must not mask restored files or continue opening after an error. A generic startup script cannot establish file provenance merely by querying V$BACKUP.
Oracle documents the two interrupted-backup procedures in sections 35.3.2.3.1 and 35.3.2.3.2 of the Oracle AI Database 26ai Backup and Recovery User's Guide. Section 35.1.2 explains the limitations of V$BACKUP. For the broader recovery procedure, review recovering an initially closed database.
The next lesson demonstrates how to clear corrupted online redo logs and assess the consequences for recovery.
Review your understanding of recovery choices when an interrupted backup or another unusual failure affects the database.