The client called at 7am on Monday morning with a database down scenario. They arrived to messages from their users that the database was inaccessible. When they logged in, they found the database was down and that it would not start normally:
$ sqlplus / as sysdba
SQL*Plus: Release 12.1.0.2.0 Production on Sun Feb 14 15:47:52 2016
Copyright (c) 1982, 2014, Oracle. All rights reserved.
Connected to an idle instance.
SQL> startup
ORACLE instance started.
Total System Global Area 2147483648 bytes
Fixed Size 3712904 bytes
Variable Size 1476397176 bytes
Database Buffers 654311424 bytes
Redo Buffers 13062144 bytes
ORA-00205: error in identifying control file, check alert log for more info
They looked the alert log and saw messages like this:
ALTER DATABASE MOUNT
Sun Feb 14 15:47:58 2016
ORA-00210: cannot open the specified control file
ORA-00202: control file: '/u01/app/oracle/recovery_area/ora12c/control02.ctl'
ORA-27037: unable to obtain file status
Linux-x86_64 Error: 2: No such file or directory
Additional information: 3
ORA-205 signalled during: ALTER DATABASE MOUNT...
Sun Feb 14 15:47:58 2016
Checker run found 1 new persistent data failures
Sun Feb 14 15:48:00 2016
Using default pga_aggregate_limit of 2048 MB
Because they knew there had been storage corruption from the storage admins, they called us in a panic. Thankfully, and luckily, Oracle forces multiplexing of the control files, so this was an easy recovery. Just find a good copy of the control file and copy it over the corrupted one:
$ sqlplus / as sysdba
SQL*Plus: Release 12.1.0.2.0 Production on Sun Feb 14 15:49:45 2016
Copyright (c) 1982, 2014, Oracle. All rights reserved.
Connected to:
Oracle Database 12c Enterprise Edition Release 12.1.0.2.0 - 64bit Production
SQL> show parameter control_files
NAME TYPE VALUE
--------------------------------- ----------- ------------------------------
control_files string /u01/app/oracle/oradata/ora12c
/control01.ctl,/u01/app/oracl
e/recovery_area/ora12c/control
02.ctl
SQL> host
$ cd /u01/app/oracle/recovery_area/ora12c
$ cp /u01/app/oracle/oradata/ora12c/control01.ctl ./control02.ctl
$ exit
exit
SQL> alter database mount;
Database altered.
SQL> alter database open;
Database altered.
It was a very simple fix, but it really highlights knowing a lot of different recovery scenarios, including some of the easy ones.
Link to script that contains the examples in this post.