Automatic Graceful Shutdown and Consistent Startup Method for Oracle DB on Linux

Hi all,

This is quite a common question whenever I arrive on any new company. The things is, there are more then one way to implement this depending on your environment, licensing and version.

So I decided to compile here some sort of summary for this:

1. Prefer to use Oracle Restart
This is the automated and validated method provided by Oracle, however it can be a bit confusing in some items which can lead us to think it’s not working. Here is a summary of the the configuration I recommend:

a) Configure database management to AUTOMATIC on SRVCTL

srvctl modify database -y AUTOMATIC
  • If AUTOMATIC (the default), the database is automatically restored to its previous running condition (started or stopped) upon restart of the database host computer.
  • If MANUAL, the database is never automatically restarted upon restart of the database host computer.

Refhttps://docs.oracle.com/cd/E11882_01/server.112/e25494/restart.htm#BABHHAHI

b) Set AUTO_START=always on CRSCTL

crsctl modify resource ora.grepora.db -attr AUTO_START=always
  • ALWAYS: Restarts the resource when the server restarts regardless of the state of the resource when the server stopped.
  • RESTORE: Restores the resource to the same state that it was in when the server stopped. Oracle Clusterware attempts to restart the resource if the value of TARGET was ONLINE before the server stopped.
  • NEVER: Oracle Clusterware never restarts the resource regardless of the state of the resource when the server stopped.

NOTE: On Oracle 11.2, the database auto start policy in the clusterware is restore, which means that clusterware will remember the last state of the database. As well as database, Oracle 11.2 comes by default with several important resources with attribute AUTO_START=restore in the profile.

NOTE2: 12c on you might need to use the flag “-unsupported” on command above (crsctl modify resource ora.grepora.db -attr AUTO_START=always – unsupported).

Refhttps://docs.oracle.com/cd/E11882_01/rac.112/e41959/resatt.htm#CHDFFEHJ

Observation: This is recommended for all the required components managed by those tools, like databases, asm, listener, diskgroups, etc.
I wrote an article about it with an script that I made by my own and can help you: https://grepora.com/2018/08/22/services-not-starting-automatically-with-crs-after-reboot/

A common problem: “I set the SRVCTL to Automatic, but databases still not starting automatically’.”
Explanation: When database Management policy is configured as AUTOMATIC and the resource of the database parameter AUTO_START is configured as restore, the cluster will restore its last state, because the cluster level is the first in the chain of commands, it overrides the database.

c) Save desired state of Pluggable Databases in case of Multitenant:
With the PDB in desired state, save it with command below:

ALTER PLUGGABLE DATABASE pdb1 SAVE STATE;

When the CDB start, it will bring the pdbs to it saved states.

2. As second Option, Oracle Provided Scripts

Oracle has some scripts to automate it in a standard and supported way. This is documented for 12.1 in Stopping and Starting Oracle Software.

NOTES:

  • Oracle 11gR2 documentation states the use of the dbstart and dbshut scripts are deprecated. It’s supposed to be replaced by Oracle Restart.
  • The Oracle 12c documentation has no mention of the deprecation of dbstart and dbshut and has reinstated the documentation about them (as I linked above). So, feel free to use dbstart and dbshut in a supported manner for all versions of the database.

I also wrote an article about those, with some info and scripts: https://grepora.com/2017/11/22/how-to-setup-automatic-startup-and-shutdown-of-an-oracle-database-on-linux-not-using-oracle-restart/

Observation: Item 1.c is still recommended here.

3. Community proven scripts

As a third option, we would have some community scripts, which are usually proven and doesn’t require us to remember or to code everything. I’d use some additional time reviewing and testing them though, as they are not Oracle provided/supported.

In general, I’d recommend the material produced by Tim Hall (Oracle Base): https://oracle-base.com/articles/linux/automating-database-startup-and-shutdown-on-linux

He has additional articles that may help for other versions:

  • Automating Shutdown and Startup (12.2)
  • Automating Shutdown and Startup (12.1)
  • Automating Shutdown and Startup (11.2)
  • Automating Shutdown and Startup (10.2)
  • Automating Startup and Shutdown (10.1)
  • Automating Database Startup and Shutdown (9.2)
  • Linux Services (systemd, systemctl)

 

Some Additional Twists:

  • The Oracle Restart configuration assume the CRS is left “enabled”. Disabling it means we don’t want it to start automatically. So, if you want the CRS to start with your server, it need to be enabled. After this, to start targets, depend on configurations as per mentioned on my previous post.
  •  Oracle will no execute any rpm change or relink automatically, as this is not part of any “restart” process. It may be required due any configuration change or corruption, and it cannot be automated.
  • Regarding gracefulness, it depends on the configuration you have on your SRVCTL too. It’s configured using stop and start option, as per example below:
srvctl modify database -d [db_unique_name] -s [start_options] -t [stop_options]
  • So for your case, it seems to me a complete command containing what was recommended on my previous post PLUS gracefulness, it would be:
srvctl modify database -d [db_unique_name] -s OPEN -t IMMEDIATE -y automatic

(Note the SRVCTL syntax can very on the versions. This one is valid for 11.2).

I hope this helps you on understanding the process.

See you next time!

ORA-00600: internal error code, arguments: [kghfrempty:ds]

Hi all,
Annoying thing, just because that sounded veeery unsual to me. I would expect a different error.

I have seen this “[kghfrempty:ds]” in past a couple times, and if you did see it too you may have noticed, this is almost as much generic as the ORA-600 itself. It happens because this is usually in the top of KGH errors in general. Some examples:

1.

kghnerror <- kghfrempty <- kghfrempty_ex <- qerhjFreeSpace
<- qerhjFetch <- qersoProcessULS <- qersoFetch <- opifch2 <- kpoal8
<- opiodr <- kpoodrc <- rpiswu2 <- kpoodr <- upirtrc
<- kpurcsc <- kpufch0 <- kpufch <- OCIStmtFetch2 <- qksanExecSql <- qksanAnalyzeSql

2.

kghfrempty <- kghfrempty_partial <- kghgex <- kghfnd <- kghalo <- kghgex <- kghalf <- ktmrProcessCRClone 
<- ktmbRead <- ktmrget_int <- ktmrget <- kdmsTransGet <- kdst_fetch_imc  <- kdsttgr  <- qertbFetch

And as being it can be related either to analyze bugs (as per stack 1), In-Memory Column Store – IMC (as per stack 2), create views, rollback over cluster, trigger creation, use function based indexes, or several other generic situations (there is no pattern, as you can see).

So what do to if you get this message? Well, generically, read all the MOS notes available and try to match you case. IF you match, pray to have a bug for your current version, or open an SR to either have a backport or start a new investigation.

A few examples of things to check:

  • ALERT: Bug 13605839 ORA-600 [ktbsdp1] ORA-600 [kghfrempty:ds] ORA-600 [kdBlkCheckError]. Corruption in Rollback with Clusterwide Global Transactions in RAC (Doc ID 1527740.1)
  • SQL Analyze Fails with ORA-600[kghfrempty:ds] (Doc ID 2247180.1)
    Bug 21556276 – dataguide createviewonpath hit ora 600 [kghfrempty:ds] (Doc ID 21556276.8)
  • Bug 20878625 – ORA-600[kghfrempty:ds] with IMC enabled (Doc ID 20878625.8)
  • Bug 19212166 – ORA-600 [KGHFREMPTY:DS] With Cache Line Alignment Enabled (Doc ID 19212166.8)
  • Bug 6797925 – ORA-600 [kghfrempty:ds] During Creation Of Trigger In XML Schema (Doc ID 6797925.8)
  • 11.1: ORA-600 [17147] AND [kghfrempty:ds] On Select Using Functional Indexes (Doc ID 884882.1)
  • ORA-600 [kghfrempty:ds] And ORA-600 [kghrcdepth:ds] While Gathering Statistics (Doc ID 1480132.1)

BUUT I just noticed some undocumented scenario causing this and it might help you on fixing this as well.
I noticed I was getting this error for different queries but always from same application. So I addressed a complete review on application layer and noticed the JDBC 7 on this application. Happens we had just upgraded from 12.1 to 19c.

SO, after upgrading jdbc, it started to work again properly and no MOS SR was required. So lesson learned: When you receive this kghfrempty:ds], speacially after migration, also check for your application JDBC version.

For compatibility Matrix: Starting With Oracle JDBC Drivers – Installation, Certification, and More! ( Doc ID 401934.1 )

Hope it helps, cheers!

ORA-65532: cannot alter or drop automatically created indexes

Hello all,
Quickly walking through this, as it was new to me (as for everybody, I guess): So what if I’m not happy with Oracle algorithm and I want to drop and AUTO Index on 19c?

Should be easy, like “drop index XXXX;” right? Wrong. See the struggle:

SQL> drop index GREPORA."SYS_AI_9xu652x5fyu5i";
drop index GREPORA."SYS_AI_9xu652x5fyu5i"
                 *
ERROR at line 1:
ORA-65532: cannot alter or drop automatically created indexes

OK, so let’s have a look on my auto created indexes. You can see them with flag AUTO as YES on query below:

SQL> select owner, index_name, auto, tablespace_name from dba_indexes natural where auto='YES';
OWNER              INDEX_NAME    AUTO    TABLESPACE_NAME
________ _______________________ _______ __________________
GREPORA  SYS_AI_9xu652x5fyu5i    YES     GREPORA
GREPORA  SYS_AI_few32swe423dw    YES     GREPORA
GREPORA  SYS_AI_94osd824n202f    YES     GREPORA

Ok, so I cannot drop, let me alter it and set it as unusable, then, as it would archive my intend anyway:

SQL> alter index ADMIN."SYS_AI_9xu652x5fyu5i" unusable;
alter index ADMIN."SYS_AI_9xu652x5fyu5i" unusable
 *
ERROR at line 1:
ORA-65532: cannot alter or drop automatically created indexes

Hmmm, so it means I cannot alter it as well?
Well, kind of, there is something I can do:

SQL> alter index ADMIN."SYS_AI_9xu652x5fyu5i" rebuild tablespace GREPORA2 online;
Index altered.

Noice, thanks for nothing. What is changed?
Well, you can use a workaround to move it to a new tablespace and then drop the tablespace:

SQL> drop tablespace GREPORA2 including contents;
Tablespace dropped.

And index will be gone:

SQL> select owner,index_name,auto,tablespace_name from dba_indexes natural where auto='YES';
OWNER              INDEX_NAME    AUTO    TABLESPACE_NAME
________ _______________________ _______ __________________
GREPORA  SYS_AI_few32swe423dw    YES     GREPORA
GREPORA  SYS_AI_94osd824n202f    YES     GREPORA

OK, so that’s bad, right… What the worse part? If you simply drop it, Oracle will probably recreate it, based on algorithm, right?
Gee, thanks for nothing (again)!

This, though, is easier to change: Simply alter the Audo Indexing to report mode:

exec dbms_auto_index.configure('AUTO_INDEX_MODE','REPORT');

This is interesting right?
Ok, we have more options, we can even exclude specific schemas from AUTO_INDEX, choose a default tablespace, define retentions and a lot of things.

As usual, the best reference I could find: https://oracle-base.com/articles/19c/automatic-indexing-19c

Just one thing you might want to consider and it’s not on Tim Hall’s post. Specially if the index is bad for one specific query, is to avoid using it during that specific query. We can do it with session/system parameter:

“_optimizer_use_auto_indexes”=OFF

Well, hope it helps you. Cheers!

Stop Migrating Databases to 12c!

If you are starting an upgrade plan, do it directly to 19c. Why?

Well, first let’s say that 19c is “equal” to 12.2.0.3 on previous version model, while 18c is 12.2.0.2, after latest 12.2.0.1. So, thinking reasonably, you would already plan for latest PSU on the future version, right? So this is 19c.

But not only this. It’s well known that Oracle’s Terminal Patchset is the Long Term Release for each version. And for 12.2 family this is 19c.

The image below (From Doc ID 742060.1) should help on clarifying this:

versions

In summary, since August 1st 2019:
– 11.2.0.4 is already on Extended Support (20% more expensive unless you have ULA)
– 12.1 family is already on Extended Support (20% more expensive unless you have ULA)

Which only leave us with 12.2 Family. Now I ask you, why jump to any “PSU” other than the final one?
Also, support on 12.2.0.1 goes only up to Nov 2020 and for 18c (12.2.0.2) only to Jun 2021, while 19c goes up to 2026 with ES/ULA!

I’m not even raising the great improvements we have on newer versions (if you are in 11.2, shame on you, you are basically 6 years behind the world technology wise).
Myself , I already migrated several databases on different clients to 19c and it’s running preeeetty fine. So trust on it!

Ok, so to migrate to 19c? (Some tips):
– To migrate directly to 19c you need to be on supported terminal patch set (11.2.0.4, 12.1.0.2) or on 12.2 family (12.2.0.1, 18c).
– Get in touch and use AutoUpgrade mechanism (you’ll love it!) – AutoUpgrade Tool (Doc ID 2485457.1)
– Check Upgrade Guide: https://docs.oracle.com/en/database/oracle/oracle-database/19/upgrd/index.html
– Have a look on Mike Dietrich blog for news: https://mikedietrichde.com/
– Test it before doing in prod.

Additional Reference:
Release Schedule of Current Database Releases (Doc ID 742060.1)
Very nice and complete post (in portuguese) on this topic from Ricardo Portilho.

I like very much the table below as well:

NewScreenshot 2019-08-13 às 15.11.34

See you!

Quick ways to transfer files to Oracle Support – MOS directly from the Database Server

Hi all,

This will be a quick one but helped me a lot last week.

I was working on an Ora-0600 issue on a 2 node RAC cluster. Working with Oracle Support I was asked to transfer a TFA data as well ADRCI data so they could move forward with the SR, very common process right?

After I generated all the files, it was over 3gb of data to be sent over. If I was to get the file from the database server to my laptop I would need to transfer these files multiple times due to the jump servers in the middle.

This time I was luck as the database server was able to ping the site https://transport.oracle.com so I tried sending the files directly to MOS.

I will show you how I did it. This is not a mistery but can really save you time

You have 2 options which are straight forward

Using curl

curl -T FILE_YOU_WANT_TO_SEND -u MOS_USER https://transport.oracle.com/upload/issue/SR_NUMBER/

Using tfacfl

tfactl upload -sr SR_NUMBER -user MOS_USER FILE_YOU_WANT_TO_SEND

On both you need to provide your MOS credentials the SR number and the file you want o upload

After the upload is done, you will see in the attachments in the SR that the files is with the status like “Transferring to the SR” after a few minutes later you will see the update on the SR saying that the file was uploaded to the SR.

Hope this helps!

Elisson Almeida

Invalid system views after upgrading from 12c to 18c – X_$KGLOB and X_$KGLCURSOR

I was working on a client upgrade from 12c to 18c. The upgrade was done manually. After the dbupgrade 2 system views got invalid, X_$KGLCURSOR and X_$KGLOB

OWNER OBJECT_NAME OBJECT_TYPE STATUS
------------ -------------------------------------------------- ---------------------------------------- -------
SYS X_$KGLOB VIEW INVALID
SYS X_$KGLCURSOR VIEW INVALID

Man system objects invalid after an upgrade is never good.

Troubleshooting, I got the view text and tried to execute its sql

SQL> select TEXT from dba_Views where view_name = 'X_$KGLOB';

TEXT
------------------------------------------------------------------------------------------------------------------------------------------------------
SELECT "ADDR","INDX","INST_ID","CON_ID","KGLHDADR","KGLHDPAR","KGLHDCLT","KGLNAOWN","KGLNAOBJ","KGLFNOBJ","KGLNADLK","KGLNAHSH","KGLNAHSV","KGLNATIM",
"KGLNAPTM","KGLHDNSP","KGLHDNSD","KGLHDLMD","KGLHDPMD","KGLHDFLG","KGLHDOBJ","KGLHDLDC","KGLHDIVC","KGLHDEXC","KGLHDLKC","KGLHDKMK","KGLHDDMK","KGLHDA
MK","KGLOBFLG","KGLOBSTA","KGLOBTYP","KGLOBTYD","KGLOBHMK","KGLOBHS0","KGLOBHS1","KGLOBHS2","KGLOBHS3","KGLOBHS4","KGLOBHS5","KGLOBHS6","KGLOBHS7","KG
LOBHD0","KGLOBHD1","KGLOBHD2","KGLOBHD3","KGLOBHD4","KGLOBHD5","KGLOBHD6","KGLOBHD7","KGLOBPC0","KGLOBPC6","KGLOBTP0","KGLOBT00","KGLOBT01","KGLOBT02"
,"KGLOBT03","KGLOBT04","KGLOBT05","KGLOBT35","KGLOBT06","KGLOBT07","KGLOBT08","KGLOBT09","KGLOBT10","KGLOBT11","KGLOBT12","KGLOBT13","KGLOBT14","KGLOB
T15","KGLOBT16","KGLOBT17","KGLOBT18","KGLOBT19","KGLOBT20","KGLOBT21","KGLOBT22","KGLOBT23","KGLOBT24","KGLOBT25","KGLOBT26","KGLOBT28","KGLOBT29","K
GLOBT30","KGLOBT31","KGLOBT27","KGLOBT32","KGLOBT33","KGLOBWAP","KGLOBWCC","KGLOBWCL","KGLOBWUI","KGLOBWDW","KGLOBT42","KGLOBT43","KGLOBT44","KGLOBT45
","KGLOBT46","KGLOBT47","KGLOBT49","KGLOBT50","KGLOBT52","KGLOBT53","KGLOBTL0","KGLOBTL1","KGLOBTS0","KGLOBTS1","KGLOBTN0","KGLOBTN1","KGLOBTN2","KGLO
BTN3","KGLOBTN4","KGLOBTN5","KGLOBTS2","KGLOBTS3","KGLOBTS5","KGLOBTT0","KGLOBCCE","KGLOBCCEH","KGLOBCLA","KGLOBCLC","KGLOBCCC","KGLOBTS4","KGLOBCBCA"
,"KGLOBT48","KGLOBDSO","KGLOBDEX","KGLOBDPX","KGLOBDLD","KGLOBDIV","KGLOBDPS","KGLOBDDR","KGLOBDDW","KGLOBDBF","KGLOBDRO","KGLOBDCP","KGLOBDEL","KGLOB
DFT","KGLOBDEF","KGLOBDUI","KGLOBDCL","KGLOBDAP","KGLOBDCC","KGLOBDPL","KGLOBDJV","KGLOBACS","KGLOBTS6","KGLOBTS7","KGLOBT54","KGLOBT55","KGLOBT56","K
GLOBT57","KGLOBDCO","KGLOBDCI","KGLOBDRR","KGLOBDRB","KGLOBDWR","KGLOBDWB","KGLOBT58","KGLOBDOR","KGLHDMTX","KGLHDMVL","KGLHDMSP","KGLHDMGT","KGLHDDMT
X","KGLHDDVL","KGLHDDSP","KGLHDDGT","KGLHDBID","KGLHDBMTX","KGLHDBVL","KGLHDBSP","KGLHDBGT","KGLOBT59","KGLOBDCU","KGLOBPROP","KGLNACON","KGLNAO1N","K
GLHDMAR","KGLMPFLG","KGLOBAQP","KGLOBT60","KGLOBT61","KGLIMSCANS","KGLIMBYTESUNCOMP","KGLIMBYTESINMEM" FROM X$KGLOB


1 row selected.
SQL> SELECT "ADDR","INDX","INST_ID","CON_ID","KGLHDADR","KGLHDPAR","KGLHDCLT","KGLNAOWN","KGLNAOBJ","KGLFNOBJ","KGLNADLK","KGLNAHSH","KGLNAHSV","KGLNATIM","KGLNAPTM","KGLHDNSP","KGLHDNSD","KGLHDLMD","KGLHDPMD","KGLHDFLG","KGLHDOBJ","KGLHDLDC","KGLHDIVC","KGLHDEXC","KGLHDLKC","KGLHDKMK","KGLHDDMK","KGLHDAMK","KGLOBFLG","KGLOBSTA","KGLOBTYP","KGLOBTYD","KGLOBHMK","KGLOBHS0","KGLOBHS1","KGLOBHS2","KGLOBHS3","KGLOBHS4","KGLOBHS5","KGLOBHS6","KGLOBHS7","KGLOBHD0","KGLOBHD1","KGLOBHD2","KGLOBHD3","KGLOBHD4","KGLOBHD5","KGLOBHD6","KGLOBHD7","KGLOBPC0","KGLOBPC6","KGLOBTP0","KGLOBT00","KGLOBT01","KGLOBT02","KGLOBT03","KGLOBT04","KGLOBT05","KGLOBT35","KGLOBT06","KGLOBT07","KGLOBT08","KGLOBT09","KGLOBT10","KGLOBT11","KGLOBT12","KGLOBT13","KGLOBT14","KGLOBT15","KGLOBT16","KGLOBT17","KGLOBT18","KGLOBT19","KGLOBT20","KGLOBT21","KGLOBT22","KGLOBT23","KGLOBT24","KGLOBT25","KGLOBT26","KGLOBT28","KGLOBT29","KGLOBT30","KGLOBT31","KGLOBT27","KGLOBT32","KGLOBT33","KGLOBWAP","KGLOBWCC","KGLOBWCL","KGLOBWUI","KGLOBWDW","KGLOBT42","KGLOBT43","KGLOBT44","KGLOBT45","KGLOBT46","KGLOBT47","KGLOBT49","KGLOBT50","KGLOBT52","KGLOBT53","KGLOBTL0","KGLOBTL1","KGLOBTS0","KGLOBTS1","KGLOBTN0","KGLOBTN1","KGLOBTN2","KGLOBTN3","KGLOBTN4","KGLOBTN5","KGLOBTS2","KGLOBTS3","KGLOBTS5","KGLOBTT0","KGLOBCCE","KGLOBCCEH","KGLOBCLA","KGLOBCLC","KGLOBCCC","KGLOBTS4","KGLOBCBCA","KGLOBT48","KGLOBDSO","KGLOBDEX","KGLOBDPX","KGLOBDLD","KGLOBDIV","KGLOBDPS","KGLOBDDR","KGLOBDDW","KGLOBDBF","KGLOBDRO","KGLOBDCP","KGLOBDEL","KGLOBDFT","KGLOBDEF","KGLOBDUI","KGLOBDCL","KGLOBDAP","KGLOBDCC","KGLOBDPL","KGLOBDJV","KGLOBACS","KGLOBTS6","KGLOBTS7","KGLOBT54","KGLOBT55","KGLOBT56","KGLOBT57","KGLOBDCO","KGLOBDCI","KGLOBDRR","KGLOBDRB","KGLOBDWR","KGLOBDWB","KGLOBT58","KGLOBDOR","KGLHDMTX","KGLHDMVL","KGLHDMSP","KGLHDMGT","KGLHDDMTX","KGLHDDVL","KGLHDDSP","KGLHDDGT","KGLHDBID","KGLHDBMTX","KGLHDBVL","KGLHDBSP","KGLHDBGT","KGLOBT59","KGLOBDCU","KGLOBPROP","KGLNACON","KGLNAO1N","KGLHDMAR","KGLMPFLG","KGLOBAQP","KGLOBT60","KGLOBT61","KGLIMSCANS","KGLIMBYTESUNCOMP","KGLIMBYTESINMEM" FROM X$KGLOB;
SELECT "ADDR","INDX","INST_ID","CON_ID","KGLHDADR","KGLHDPAR","KGLHDCLT","KGLNAOWN","KGLNAOBJ","KGLFNOBJ","KGLNADLK","KGLNAHSH","KGLNAHSV","KGLNATIM","KGLNAPTM","KGLHDNSP","KGLHDNSD","KGLHDLMD","KGLHDPMD","KGLHDFLG","KGLHDOBJ","KGLHDLDC","KGLHDIVC","KGLHDEXC","KGLHDLKC","KGLHDKMK","KGLHDDMK","KGLHDAMK","KGLOBFLG","KGLOBSTA","KGLOBTYP","KGLOBTYD","KGLOBHMK","KGLOBHS0","KGLOBHS1","KGLOBHS2","KGLOBHS3","KGLOBHS4","KGLOBHS5","KGLOBHS6","KGLOBHS7","KGLOBHD0","KGLOBHD1","KGLOBHD2","KGLOBHD3","KGLOBHD4","KGLOBHD5","KGLOBHD6","KGLOBHD7","KGLOBPC0","KGLOBPC6","KGLOBTP0","KGLOBT00","KGLOBT01","KGLOBT02","KGLOBT03","KGLOBT04","KGLOBT05","KGLOBT35","KGLOBT06","KGLOBT07","KGLOBT08","KGLOBT09","KGLOBT10","KGLOBT11","KGLOBT12","KGLOBT13","KGLOBT14","KGLOBT15","KGLOBT16","KGLOBT17","KGLOBT18","KGLOBT19","KGLOBT20","KGLOBT21","KGLOBT22","KGLOBT23","KGLOBT24","KGLOBT25","KGLOBT26","KGLOBT28","KGLOBT29","KGLOBT30","KGLOBT31","KGLOBT27","KGLOBT32","KGLOBT33","KGLOBWAP","KGLOBWCC","KGLOBWCL","KGLOBWUI","KGLOBWDW","KGLOBT42","KGLOBT43","KGLOBT44","KGLOBT45","KGLOBT46","KGLOBT47","KGLOBT49","KGLOBT50","KGLOBT52","KGLOBT53","KGLOBTL0","KGLOBTL1","KGLOBTS0","KGLOBTS1","KGLOBTN0","KGLOBTN1","KGLOBTN2","KGLOBTN3","KGLOBTN4","KGLOBTN5","KGLOBTS2","KGLOBTS3","KGLOBTS5","KGLOBTT0","KGLOBCCE","KGLOBCCEH","KGLOBCLA","KGLOBCLC","KGLOBCCC","KGLOBTS4","KGLOBCBCA","KGLOBT48","KGLOBDSO","KGLOBDEX","KGLOBDPX","KGLOBDLD","KGLOBDIV","KGLOBDPS","KGLOBDDR","KGLOBDDW","KGLOBDBF","KGLOBDRO","KGLOBDCP","KGLOBDEL","KGLOBDFT","KGLOBDEF","KGLOBDUI","KGLOBDCL","KGLOBDAP","KGLOBDCC","KGLOBDPL","KGLOBDJV","KGLOBACS","KGLOBTS6","KGLOBTS7","KGLOBT54","KGLOBT55","KGLOBT56","KGLOBT57","KGLOBDCO","KGLOBDCI","KGLOBDRR","KGLOBDRB","KGLOBDWR","KGLOBDWB","KGLOBT58","KGLOBDOR","KGLHDMTX","KGLHDMVL","KGLHDMSP","KGLHDMGT","KGLHDDMTX","KGLHDDVL","KGLHDDSP","KGLHDDGT","KGLHDBID","KGLHDBMTX","KGLHDBVL","KGLHDBSP","KGLHDBGT","KGLOBT59","KGLOBDCU","KGLOBPROP","KGLNACON","KGLNAO1N","KGLHDMAR","KGLMPFLG","KGLOBAQP","KGLOBT60","KGLOBT61","KGLIMSCANS","KGLIMBYTESUNCOMP","KGLIMBYTESINMEM" FROM X$KGLOB
*
ERROR at line 1:
ORA-00904: "KGLOBT24": invalid identifier
SQL> select TEXT from dba_Views where view_name = 'X_$KGLCURSOR'

TEXT
------------------------------------------------------------------------------------------------------------------------------------------------------
SELECT "ADDR","INDX","INST_ID","CON_ID","KGLHDADR","KGLHDPAR","KGLHDCLT","KGLNAOWN","KGLNAOBJ","KGLFNOBJ","KGLNADLK","KGLNAHSH","KGLNAHSV","KGLNATIM",
"KGLNAPTM","KGLHDNSP","KGLHDNSD","KGLHDLMD","KGLHDPMD","KGLHDFLG","KGLHDOBJ","KGLHDLDC","KGLHDIVC","KGLHDEXC","KGLHDLKC","KGLHDKMK","KGLHDDMK","KGLHDA
MK","KGLOBFLG","KGLOBSTA","KGLOBTYP","KGLOBTYD","KGLOBHMK","KGLOBHS0","KGLOBHS1","KGLOBHS2","KGLOBHS3","KGLOBHS4","KGLOBHS5","KGLOBHS6","KGLOBHS7","KG
LOBHD0","KGLOBHD1","KGLOBHD2","KGLOBHD3","KGLOBHD4","KGLOBHD5","KGLOBHD6","KGLOBHD7","KGLOBPC0","KGLOBPC6","KGLOBTP0","KGLOBT00","KGLOBT01","KGLOBT02"
,"KGLOBT03","KGLOBT04","KGLOBT05","KGLOBT35","KGLOBT06","KGLOBT07","KGLOBT08","KGLOBT09","KGLOBT10","KGLOBT11","KGLOBT12","KGLOBT13","KGLOBT14","KGLOB
T15","KGLOBT16","KGLOBT17","KGLOBT18","KGLOBT19","KGLOBT20","KGLOBT21","KGLOBT22","KGLOBT23","KGLOBT24","KGLOBT25","KGLOBT26","KGLOBT28","KGLOBT29","K
GLOBT30","KGLOBT31","KGLOBT27","KGLOBT32","KGLOBT33","KGLOBWAP","KGLOBWCC","KGLOBWCL","KGLOBWUI","KGLOBWDW","KGLOBT42","KGLOBT43","KGLOBT44","KGLOBT45
","KGLOBT46","KGLOBT47","KGLOBT49","KGLOBT50","KGLOBT52","KGLOBT53","KGLOBTL0","KGLOBTL1","KGLOBTS0","KGLOBTS1","KGLOBTN0","KGLOBTN1","KGLOBTN2","KGLO
BTN3","KGLOBTN4","KGLOBTN5","KGLOBTS2","KGLOBTS3","KGLOBTS5","KGLOBTT0","KGLOBCCE","KGLOBCCEH","KGLOBCLA","KGLOBCLC","KGLOBCCC","KGLOBTS4","KGLOBCBCA"
,"KGLOBT48","KGLOBDSO","KGLOBDEX","KGLOBDPX","KGLOBDLD","KGLOBDIV","KGLOBDPS","KGLOBDDR","KGLOBDDW","KGLOBDBF","KGLOBDRO","KGLOBDCP","KGLOBDEL","KGLOB
DFT","KGLOBDEF","KGLOBDUI","KGLOBDCL","KGLOBDAP","KGLOBDCC","KGLOBDPL","KGLOBDJV","KGLOBACS","KGLOBTS6","KGLOBTS7","KGLOBT54","KGLOBT55","KGLOBT56","K
GLOBT57","KGLOBDCO","KGLOBDCI","KGLOBDRR","KGLOBDRB","KGLOBDWR","KGLOBDWB","KGLOBT58","KGLOBDOR","KGLHDMTX","KGLHDMVL","KGLHDMSP","KGLHDMGT","KGLHDDMT
X","KGLHDDVL","KGLHDDSP","KGLHDDGT","KGLHDBID","KGLHDBMTX","KGLHDBVL","KGLHDBSP","KGLHDBGT","KGLOBT59","KGLOBDCU","KGLOBPROP","KGLNACON","KGLNAO1N","K
GLHDMAR","KGLMPFLG","KGLOBAQP","KGLOBT60","KGLOBT61","KGLIMSCANS","KGLIMBYTESUNCOMP","KGLIMBYTESINMEM" FROM X$KGLCURSOR


1 row selected.
SQL> SELECT "ADDR","INDX","INST_ID","CON_ID","KGLHDADR","KGLHDPAR","KGLHDCLT","KGLNAOWN","KGLNAOBJ","KGLFNOBJ","KGLNADLK","KGLNAHSH","KGLNAHSV","KGLNATIM",
"KGLNAPTM","KGLHDNSP","KGLHDNSD","KGLHDLMD","KGLHDPMD","KGLHDFLG","KGLHDOBJ","KGLHDLDC","KGLHDIVC","KGLHDEXC","KGLHDLKC","KGLHDKMK","KGLHDDMK","KGLHDA
MK","KGLOBFLG","KGLOBSTA","KGLOBTYP","KGLOBTYD","KGLOBHMK","KGLOBHS0","KGLOBHS1","KGLOBHS2","KGLOBHS3","KGLOBHS4","KGLOBHS5","KGLOBHS6","KGLOBHS7","KG
LOBHD0","KGLOBHD1","KGLOBHD2","KGLOBHD3","KGLOBHD4","KGLOBHD5","KGLOBHD6","KGLOBHD7","KGLOBPC0","KGLOBPC6","KGLOBTP0","KGLOBT00","KGLOBT01","KGLOBT02"
,"KGLOBT03","KGLOBT04","KGLOBT05","KGLOBT35","KGLOBT06","KGLOBT07","KGLOBT08","KGLOBT09","KGLOBT10","KGLOBT11","KGLOBT12","KGLOBT13","KGLOBT14","KGLOB
T15","KGLOBT16","KGLOBT17","KGLOBT18","KGLOBT19","KGLOBT20","KGLOBT21","KGLOBT22","KGLOBT23","KGLOBT24","KGLOBT25","KGLOBT26","KGLOBT28","KGLOBT29","K
GLOBT30","KGLOBT31","KGLOBT27","KGLOBT32","KGLOBT33","KGLOBWAP","KGLOBWCC","KGLOBWCL","KGLOBWUI","KGLOBWDW","KGLOBT42","KGLOBT43","KGLOBT44","KGLOBT45
","KGLOBT46","KGLOBT47","KGLOBT49","KGLOBT50","KGLOBT52","KGLOBT53","KGLOBTL0","KGLOBTL1","KGLOBTS0","KGLOBTS1","KGLOBTN0","KGLOBTN1","KGLOBTN2","KGLO
BTN3","KGLOBTN4","KGLOBTN5","KGLOBTS2","KGLOBTS3","KGLOBTS5","KGLOBTT0","KGLOBCCE","KGLOBCCEH","KGLOBCLA","KGLOBCLC","KGLOBCCC","KGLOBTS4","KGLOBCBCA"
,"KGLOBT48","KGLOBDSO","KGLOBDEX","KGLOBDPX","KGLOBDLD","KGLOBDIV","KGLOBDPS","KGLOBDDR","KGLOBDDW","KGLOBDBF","KGLOBDRO","KGLOBDCP","KGLOBDEL","KGLOB
DFT","KGLOBDEF","KGLOBDUI","KGLOBDCL","KGLOBDAP","KGLOBDCC","KGLOBDPL","KGLOBDJV","KGLOBACS","KGLOBTS6","KGLOBTS7","KGLOBT54","KGLOBT55","KGLOBT56","K
GLOBT57","KGLOBDCO","KGLOBDCI","KGLOBDRR","KGLOBDRB","KGLOBDWR","KGLOBDWB","KGLOBT58","KGLOBDOR","KGLHDMTX","KGLHDMVL","KGLHDMSP","KGLHDMGT","KGLHDDMT
X","KGLHDDVL","KGLHDDSP","KGLHDDGT","KGLHDBID","KGLHDBMTX","KGLHDBVL","KGLHDBSP","KGLHDBGT","KGLOBT59","KGLOBDCU","KGLOBPROP","KGLNACON","KGLNAO1N","K
GLHDMAR","KGLMPFLG","KGLOBAQP","KGLOBT60","KGLOBT61","KGLIMSCANS","KGLIMBYTESUNCOMP","KGLIMBYTESINMEM" FROM X$KGLCURSOR
2 3 4 5 6 7 8 9 10 11 12 13 14 15 
SQL> /
X","KGLHDDVL","KGLHDDSP","KGLHDDGT","KGLHDBID","KGLHDBMTX","KGLHDBVL","KGLHDBSP","KGLHDBGT","KGLOBT59","KGLOBDCU","KGLOBPROP","KGLNACON","KGLNAO1N","K
*
ERROR at line 13:
ORA-00904: "KGLHDMAR": invalid identifier

Describing the source tables used by the views, the columns that generated the error do not exist there. What?!

Looking up MOS – found this note, Invalid x_$ Objects After Upgrade (Doc ID 361757.1)

Although this note does not mention these views explicitly there is a part of that says that

"This is NOT a dictionary corruption, but the result of an unsupported operation.

.....

These x_$ views do not compile because they are based on x$ tables which have changed (columns dropped) and/or do not exist (X$ table dropped).

New (default) 10g & 11g databases do not have views with the prefix x_$.

Please note that this solution can be applied to any and all x_$ views found invalid after upgrade/migration.
These tables are built on x$ tables in the previous release and the same rules still apply."

So these are save to ignore or drop them.

Hope this helps.

Elisson Almeida

Waiting for ASM to startup after upgrading to Oracle 18c

Hi,
Something strange happened to me and I would like to share with you. I was upgrading one of my Virtual Machine which was an Oracle Restart – SIHA,  running 11.2.0.4 to 18c on Oracle Linux 6.9.

The upgrade went fine using the GUI no issues nor strange log messages, but the issue happened when I rebooted the VM. For some reason (which I don’t knew) the startup was hanging when trying to start HAS and then it outputted the error below:

PRCR-1070 :Failed to check if resource ora.asm is registered
CRS-0184 : Cannot communicate with the CRS daemon
Waiting for ASM to startup

Looking up on MOS I found this note – Oracle Linux 6 server hangs on boot with error: Waiting for ASM to startup (Doc ID 2495023.1)

The startup was hanging waiting on acfssihamount but as ASM was not up yet and it could not start.

To fix it you need to boot OS in rescue mode and execute

chroot /mnt/sysimage

Once you do that you will be able to change the startup configuration settings using chkconfig and disable acfssihamount

chkconfig --list acfssihamount
chkconfig acfssihamount off
chkconfig --list acfssihamount

After this exit out the rescue mode and the system should boot normally.

Hope it helps!

Elisson Almeida

Value FROM v$option where parameter = ‘Unified Auditing’

Hi all,
So, just a quick post to clarify once it cost me a few mins to explain this to a client and, indeed, Oracle could have made it easier.

So, facing the info below, doesn’t mean that you don’t have Unified Auditing enabled:

SELECT value FROM v$option WHERE parameter = 'Unified Auditing';
VALUE
-----------------
FALSE

It does mean you just don’t have PURE Unified auditing, but a MIXED configuration, where old fashioned Auditing is also accepted.

See, this way, the MIXED Auditing is by default it is enabled in 12c. The idea is that you use apart from traditional auditing all the features of unified auditing. Once we are comfortable with the unified concept, we can migrate exiting audit setup to unified policy , we can enable pure auditing. This serves as a good mediator for an easy and hassle free switch to the preferred Unified auditing.

HOWEVER, let’s agree that Oracle could have made the value column a little bit m ore meaningful, if this was the case. The actual values actually mean:

FALSE – > MIXED AUDTING
TRUE –> PURE AUDITING

OK, but what if I want to enable ONLY the PURE Auditing, disabling the ‘Classic’ Auditing and changing this value on v$option to TRUE? Well, then you need to relink library for your instance. Note, for RAC linking need to be done all the nodes.

Steps to Change to PURE Unified Auditing Only:

SELECT value FROM v$option WHERE parameter = 'Unified Auditing';
VALUE
-----------------
FALSE

sqlplus / as sysdba <<
shutdown immediate;
exit;

cd $ORACLE_HOME/rdbms/lib
 
make -f ins_rdbms.mk unaiaud_on ioracle
 
sqlplus / as sysdba<<
startup
 
SELECT value FROM v$option WHERE parameter = 'Unified Auditing';
VALUE
-----------------
TRUE

Reference: https://docs.oracle.com/database/121/DBSEG/auditing.htm#DBSEG824

Hope it helps, cheers!

Oracle New Visual Online Tools

Hi all,
So, after a long time of same old documentation style, Oracle started sharing some nice and visual pages for some specifics. Here are 2 examples:

1. Oracle Move to Cloud  (http://www.oracle.com/goto/move)
This one was shown to me by Ricardo Gonzalez, and it’s a GREAT tool if you are planning migrations to Oracle Cloud in general.

The part I like most is where you select the source and destination formats/versions you want and all recommended migration methods are suggested. Of course additional validation on database are required for each method, but it can potentially remind you of things you may be forgetting.

NewScreenshot 2019-05-08 às 16.56.46

Something else I like very much as well is the section with executive summaries of each migration tool. It’s basically ready for a print screen to put in the slides you are going to present to your boss… 🙂

NewScreenshot 2019-05-08 às 16.57.16.png

Also, you have handy some links to papers for the most different migration scenarios, as per:

NewScreenshot 2019-05-08 às 16.57.24

2. Oracle Database Features: (https://apex.oracle.com/database-features/)
I actually got it from this post by Portilho, and found it great! See how it looks like:

NewScreenshot 2019-05-08 às 16.52.14

And by selecting any of the features you can see more info in a very simple way, as per below. By the way, with one click you can be directed to the documentation (the old fashioned one):

NewScreenshot 2019-05-08 às 16.55.08

 

Hope you enjoy those as much as I did and hope Oracle continue providing those sort of interfaces to us!

Cheers!

Oracle: “Invisible” Locks!

Have you ever heard:
“My session was killed. I’m trying again. There is no locks. But My SQL doesn’t run!”

By default, when a session is disconnected from Oracle, the SQLs uncommitted are undone (rollback). So, in case of a long routing there is a probably a rollback in place.

How Oracle rollback take that long? Well, it’s Oracle mechanism. The RDBMS basically assume you know what your are doing and start writing the new blocks so the commit will be very quick. Which, however, is bad in case of rollbacks.

How to check if this is your case?

select ktuxeusn, to_char(sysdate,'DD-MON-YYYY HH24:MI:SS') "Time", ktuxesiz, ktuxesta
from x$ktuxe
where ktuxecfl = 'DEAD'
4 /

KTUXEUSN Time KTUXESIZ KTUXESTA
---------- -------------------- ---------- ----------------
31 07-APR-2018 16:48:53 17705 ACTIVE
48 07-APR-2018 16:48:53 108856 ACTIVE

Hm.. What can I do? Not much really, we need to wait the rollback to complete.
One thing that can be done to increase the rollback is changing the fast_start_parallel_rollback parameter, as per:

SQL> show parameters roll

NAME TYPE VALUE
------------------------------------ ----------- ------------------------------
fast_start_parallel_rollback string LOW
rollback_segments string
transactions_per_rollback_segment integer 5
SQL> alter system set fast_start_parallel_rollback=HIGH;

System altered.

Hope it helps!
Cheers!