Hey guys,
Upgrading is always something critical and a delicate operation but when you have no feedback on in the screen even harder.
I was working on an upgrade and using the GUI to upgrade the GRID from 11g to 12c. The 11gr2 11.2.0.4 was working without issue and ASM was as well (note this point, we will come back here later on).
When it was time to run the rootupgrade.sh, it just got stuck. No matter what, the GRID upgrade to 12c just FROZE. Checking the logs the last message was only this:
CLSRSC-467: Shutdown of the current Oracle Grid Infrastructure stack has successfully completed.
Looking the other logs (/u01/app/12.1.0/grid/cfgtoollogs/crsconfig) there were messages related to OCR, pointing it cannot get OCR key with CLUUTIL, try using OCRDUMP. I checked ORC with ocrdump and ocrcheck. No issues there as well. Also, as I said before, the cluster was working without any issues.
As I had no error code or any thing that would give me a more specific cause. I went to a broad search on google and MOS. Saw all kind of things until I found the MOS: Wrong DiscoveryString /dev/*: rootupgrade.sh/root.sh hangs: Check OCR key using ocrdump (Doc ID 1916106.1)
I checked any my ASM disk discovery string was set to /dev/* which did not strike me as an issue as I mentioned it was working… BUT when I changed the script in ASM to /dev/asm-* the upgrade worked like a charm.
Also as note there is this note, with some best practices for upgrading: How to Upgrade to/Downgrade from Grid Infrastructure 12.1 and Known Issues (Doc ID 1579762.1).
Hope this helps and save some time in your troubleshooting.
Élisson Almeida
Nice post, Elisson!
Found that once too… In my case noticed the “ls -lrt /dev/* |wc-l” was taking ages…
In my case, changed to “/dev/sd*”, due my devices name…
I suspect that maybe was wrongly mounted there, as new mounts use to go to /dev on linux…