Category: Errors and Bugs

Database, Errors and Bugs

Automatic Killing Inactive Sessions with Resource Manager

 ,

Hello All!

So, your are manually (or via script) killing idle sessions on your database?

As consequence, your users are getting error “ORA-00028: your session has been killed.” and getting angry on you?

What about doing it automatically in a much more graceful way and be seen as a nicer DBA? You can do it using Resource Manager.

Are you already using Resource Manager on your database? Yes – Great!

No – Shame on you. Read this and put RM in place ASAP. This is a great tool for the database to manage the database resources, plus no additional licensing is needed. So go for it!

Ok, but what is the trick?
It’s the limit max_idle_time. You can use it either on existing groups or subgroups of your plan (or subplan) or switch to “KILL” groups with this you can even use the same criteria you’d use for any script to perform this action. For this you might user the parameter new_switch_group and have a created a different group only for those kills.

Have this option it’s nicer, if you want to avoid killing sessions on database, by the way, as you can always switch a session for the killing group manually, not demanding it to fill the requirements to automatic switching.

How? Quick example:

Creating new Plan Groups with MAX_IDLE_TIME:

begin
dbms_resource_manager.create_plan_directive( plan => '&RM_PLAN', group_or_subplan =>
'LONG_RUN', comment => 'Limit idle time to 5 minutes', max_idle_time => 300);
dbms_resource_manager.create_plan_directive( plan => '&RM_PLAN', group_or_subplan =>
'SHORT_RUN', comment => 'Limit idle time to 1 minute', max_idle_time => 60);
end;
/

Cool!

And what would be the error for the user that get’s the session killed?

ORA-3113 End of file on communication channel

Much nicer: Now you are a nice DBA and don’t kill sessions anymore.
You automated it!

Some additional recommendations:

  • Use this solution for Databases above 11.2.0.4 or 12.1.0.2, due some known bugs:
    • Bug 9523768 – IDLE SESSIONS AREN’T ACTUALLY KILLED IMMEDIATELY (affecting 11.2.0.1 to 11.2.0.3)
    • Bug 13837378 – ALTER SYSTEM KILL SESSION IMMEDIATE DOES NOT KILL QC SESSIONS (11.2.0.1 to 12.1.0.1)
    • Bug 8891495 – NON-IDLE SESSIONS ARE KILLED (11.2.0.1 only)
  • Some MOS references on it:
    • How To Automatic Kill Inactive Sessions using Resource Manager (Doc ID 1935739.1)
    • Using Resource Manager to Detect and Kill Idle Sessions (Doc ID 1557657.1)

Hope it helps!

Database, Errors and Bugs, Migration | Upgrades | Patching, Performance

Advise: Setting underScore/Hidden Parameters

 ,

Please be aware no hidden parameters (parameters starting with “_”) should not be set unless recommended by Oracle explicitly or in case of a perfect match to the case.

This is explained on MOS Best Practices and Guidelines for Setting underScore/Hidden Parameter to Workaround and/or Resolve Customers Problems (Doc ID 2524119.1)

So, in case of any diagnostics you have to lead to setting it, SUGGESTED:

  • Get the explicit approval/agreement from Oracle.
  • Include evidence on the root cause for this setting. A comment should work, something like:
alter system set "_replace_virtual_columns"=false Comment= 'Workaround Doc ID 1512347.1' SID= '*' scope= both;

Cheers!

Database, Errors and Bugs

ORA-20001: Statistics Advisor: Invalid task name for the current user

 ,

Hi all,
So, I start seeing on my alert log:

2019-12-26T00:42:03.543268-05:00
Errors in file /u01/app/oracle/diag/rdbms/cdb1/CDB1/trace/CDB1_j000_89793.trc:
ORA-12012: error on auto execute of job "SYS"."ORA$AT_OS_OPT_SY_36807"
ORA-20001: Statistics Advisor: Invalid task name for the current user
ORA-06512: at "SYS.DBMS_STATS", line 47207
ORA-06512: at "SYS.DBMS_STATS_ADVISOR", line 882
ORA-06512: at "SYS.DBMS_STATS_INTERNAL", line 20059
ORA-06512: at "SYS.DBMS_STATS_INTERNAL", line 22201
ORA-06512: at "SYS.DBMS_STATS", line 47197

After some research I found Bug 22879263 – BETA 12.2 – ORA-12012 , ORA-20001 AND ORA-06512 FROM ORA$AT_OS_OPT_SY_88 JOB.

This was suppose to be fixed on 12.2.0.1, however I’m still getting. Anyway, I executed the below and it got fixed:

EXEC dbms_stats.init_package();

Ref: Recurring ORA-12012, ORA-20001, ORA-06512 In Container Database (Doc ID 2420581.1)

Hope it helps!

Database, Errors and Bugs

Diagsnap Causing Node Eviction on 12c

 ,

Hi all,

So, you know how it is when we are having node evictions over a RAC. Lot’s of sessions getting killed, some effects for the clients and applications, also boss bossing in our shoulders to have a quick resolution over this. Plus it’s never an easy thing to drill down and understand.

Troubleshooting node reboots/evictions within Grid Infrastructure (GI) often is difficult due to the lack of Network and OS level resource information. To help circumvent this situation the diagsnap feature has been developed and integrated with Grid Infrastructure. Diagsnap is triggered to collect Network and OS level resource information when a given node is about to get evicted or when Grid Infrastructure is about to crash.

The diagsnap feature is enabled automatically starting from 12.1.0.2 Oct2017 PSU and 12.2.0.1 Oct2017 RU.
For more information about the diagsnap feature, refer to the Document 2345654.1 “What is diagsnap resource in 12c GI and above?”

However, after a lot of research and SR logs sending and interaction with Oracle to investigate a node eviction, we ended up finding a match to Bug 25785073 – OCSSD hangs while DIAGSNAP takes pstack causing a node reboot(Doc ID 25785073.8).

So Diagsnap is not helping, it is the cause of the issues.

After some research, seems this is not the only bug related to it. See some more:

  • Bug 27182006 – Auxiliary commands generated by DIAGSNAP spin CPU(Doc ID 27182006.8)
  • Bug 24692439 – Auxiliary commands generated by DIAGSNAP consumes high CPU(Doc ID 24692439.8)
  • Bug 23101338 – Disable diagsnap after 12.1.0.2.160419 GI PSU patch was installed(Doc ID 23101338.8)
  • Bug 28462215 – The Process diagsnap.pl is Restarted Every 2 Minutes(Doc ID 28462215.8)

Well,

At least for the 12Cs, I’m disabling it in all my environments:

– Check if diagsnap is disabled or not (DIAGSNAP=Disable if disabled, nothing if enabled)

$ egrep '^DIAGSNAP|^PSTACK' /u01/app/12.2.0.1/grid/crf/admin/crf$(hostname -s).ora

– Disable diagsnap (this will disable diagsnap on all nodes)

$ /u01/app/12.2.0.1/grid/bin/oclumon manage -disable diagsnap

– Check that diagsnap is disabled (DIAGSNAP=Disable if disabled, nothing if enabled) — to be done on each node

$ egrep '^DIAGSNAP|^PSTACK' /u01/app/12.2.0.1/grid/crf/admin/crf$(hostname -s).ora

Hope it helps!
Cheers!

Database, Errors and Bugs

Automatic SQL Tuning Advisor Raising ORA-00600: internal error code, arguments: [qksvcReplaceVC0]

 ,

Hi all,

So I got to receive frequently this error, always on same hour, from a database:

ORA-00600: internal error code, arguments: [qksvcReplaceVC0], [], [], [], [], [], [], [], [], [], [], []

Not much was required for matching it to the Automatic SQL Tuning Advisor.

This only seems to happen during execution of Automatic SQL Tuning Advisor. Several bugs have been logged for the issue but have not been resolved as the error is not reproducible at will. For example:

Bug 17401718: ORA-600 [QKSVCREPLACEVC0] USING SQL TUNING ADVISOR
Bug 16491690: ORA-600 [QKSVCREPLACEVC0] WHEN AUTOMATIC SQL TUNING ADVISOR EXECUTED
Bug 13959984: ORA-00600 [QKSVCREPLACEVC0]

How to fix it? Apply the patches!

To workaround it?

A few options:

1. Setting “_replace_virtual_columns” to false.

You can set this parameter at both session (where automatic SQL Tuning Advisor starts)
and system level with the following commands-

SQL> alter session set "_replace_virtual_columns"=false;

SQL> alter system set "_replace_virtual_columns"=false

2. Since it is only failing in the SQL Tuning Advisor auto task and has no effect on the database the error can be ignored.
You can disable that auto task and just run it manually when required:

–check auto job status

SQL> select client_name,status from dba_autotask_task;

SQL> select client_name,status from dba_autotask_client;

SQL> select client_name, operation_name, status from dba_autotask_operation;

–disable SQL Tuning Advisor job

SQL> exec dbms_auto_task_admin.disable ('sql tuning advisor', null, null);

-OR-

SQL> exec dbms_auto_task_admin.disable (client_name => 'sql tuning advisor', operation => null, window_name => null);

–enable SQL Tuning Advisor job

SQL> exec dbms_auto_task_admin.enable ('sql tuning advisor', null, null);

-OR-

SQL> exec dbms_auto_task_admin.enable (client_name => 'sql tuning advisor', operation => null, window_name => null);

 

Hope it helps!

Database, Errors and Bugs

ORA-02056: 2PC: k2lcom: bad two-phase command number rdonly from coord

 , ,

Hi all,

So, you found this error?

Well, I received this in a client environment. After checking MOS ORA-02056: 2PC: K2gCheckGlobalCommit: Bad Two-phase Command Number 1 From Coord (Doc ID 2498134.1), which points to a known issue when dealing with committing transactions as per Bug 22016049 I noticed my problem could be different…

My real root cause is my transaction being done on a remote object and when checking on remote database I found pending distributed transactions.

For more informations about distributed transactions, please check: https://docs.oracle.com/database/121/ADMIN/ds_txns.htm#ADMIN12211

GUID-D521B4E9-E916-4B02-8B0E-4FAF4DC61423-default

So, if you getting this situation, before planning and applying the patch I suggest you:

  • Check if your process is not working on remote databases.
  • Check for any possible distributed remote transactions.

How? Easy check (on remote db):

select * from dba_2pc_pending;

I hope it helps!

PS NOTE: I just found this very good and complete post by Rodrigo Mufalani on the same topic: ORA-01591: lock held by in-doubt distributed transaction

Have a look for an Action Plan!
Cheers!

Database, Errors and Bugs

ORA-04030: out of process memory when trying to allocate % bytes (kkoutlCreatePh,ub1 : kkoabr)

 , , ,

Hi all,

So, I started seen this on a client environment. Researching on the  case after no crear reference on MOS, I noticed some high PGA allocation as per below.

SYS@proddb AS SYSDBA PROD> select pid, serial#,category, allocated/1024/1024 MB, used/1024/1024 MB_used, max_allocated/1024/1024 MB_MAX_ALLOCATED_ON_PGA
2 from v$process_memory where pid=852;

PID SERIAL# CATEGORY MB MB_USED MB_MAX_ALLOCATED_ON_PGA
---------- ---------- --------------- ---------- ---------- -----------------------
852 91 SQL .086807251 .00806427 .672416687
852 91 PL/SQL .087730408 .078926086 .126182556
852 91 Freeable .5625 0
852 91 Other 2.25187302 2.25187302

Seems a match MOS ORA-04030 Error With High “kkoutlCreatePh” (Doc ID 1618444.1).

The solution? Simply disabled the following parameter:

"_b_tree_bitmap_plans"=false

 

Trace for additional info:

Oracle Database 12c Enterprise Edition Release 12.1.0.2.0 - 64bit Production
With the Partitioning, Real Application Clusters, Automatic Storage Management, OLAP,
Advanced Analytics and Real Application Testing options
ORACLE_HOME = /u01/app/oracle/product/12.1.0/db_1
System name: Linux
Node name: proddb.local
Release: 4.1.12-94.8.3.el7uek.x86_64
Version: #2 SMP Wed Apr 25 19:57:32 PDT 2018
Machine: x86_64
Instance name: proddb
Redo thread mounted by this instance: 1
Oracle process number: 854
Unix process pid: 28899, image: oracle@proddb.local

*** 2019-01-09 11:20:09.130
*** SESSION ID:(5429.55092) 2019-01-09 11:20:09.130
*** CLIENT ID:() 2019-01-09 11:20:09.130
*** SERVICE NAME:(XXXXX) 2019-01-09 11:20:09.130
*** MODULE NAME:(JDBC Thin Client) 2019-01-09 11:20:09.130
*** CLIENT DRIVER:(jdbcthin) 2019-01-09 11:20:09.130
*** ACTION NAME:() 2019-01-09 11:20:09.130

[TOC00000]
Jump to table of contents
Dump continued from file: /u01/app/oracle/diag/rdbms/proddb/proddb/trace/proddb_ora_28899.trc
[TOC00001]
ORA-04030: out of process memory when trying to allocate 34392040 bytes (kkoutlCreatePh,ub1 : kkoabr)

[TOC00001-END]
[TOC00002]
========= Dump for incident 2710965 (ORA 4030) ========
[TOC00003]
----- Beginning of Customized Incident Dump(s) -----
=======================================
TOP 10 MEMORY USES FOR THIS PROCESS
---------------------------------------
72% 18 GB, 1356 chunks: "free memory " SQL
kxs-heap-c ds=0x7fa8df330220 dsprt=0x7fa8df49dbe0
27% 6809 MB, 3366 chunks: "permanent memory " SQL
kxs-heap-c ds=0x7fa8df330220 dsprt=0x7fa8df49dbe0
0% 100 MB, 894 chunks: "permanent memory " SQL
kkoutlCreatePh ds=0x7fa8cce16708 dsprt=0x7fa8df330220
0% 23 MB, 589515 chunks: "chedef : qcuatc "
TCHK^2a9688d9 ds=0x7fa8df33feb8 dsprt=0x7fa8df49c9e0
0% 18 MB, 150124 chunks: "opndef: qcopCreateOpnViaM "
TCHK^2a9688d9 ds=0x7fa8df33feb8 dsprt=0x7fa8df49c9e0
0% 16 MB, 214829 chunks: "logdef: qcopCreateLog "
TCHK^2a9688d9 ds=0x7fa8df33feb8 dsprt=0x7fa8df49c9e0
0% 11 MB, 241 chunks: "free memory "
top call heap ds=0x7fa8df49dbe0 dsprt=(nil)
0% 9643 KB, 4623 chunks: "qkkele " SQL
kxs-heap-c ds=0x7fa8df330220 dsprt=0x7fa8df49dbe0
0% 6534 KB, 58578 chunks: "optdef: qcopCreateOptInte "
TCHK^2a9688d9 ds=0x7fa8df33feb8 dsprt=0x7fa8df49c9e0
0% 4134 KB, 15399 chunks: "kccdef : qcsvwsci "
TCHK^2a9688d9 ds=0x7fa8df33feb8 dsprt=0x7fa8df49c9e0

=======================================
PRIVATE MEMORY SUMMARY FOR THIS PROCESS
---------------------------------------
******************************************************
PRIVATE HEAP SUMMARY DUMP
25 GB total:
25 GB commented, 794 KB permanent
12 MB free (0 KB in empty extents),
24 GB, 1 heap: "kxs-heap-c " 67 KB free held
------------------------------------------------------
Summary of subheaps at depth 1
25 GB total:
203 MB commented, 6809 MB permanent
18 GB free (0 KB in empty extents),
6398 MB, 9243 chunks: "allocator state " 6398 MB free held
3758 MB, 4623 chunks: "qkkele " 3749 MB free held
3650 MB, 4623 chunks: "qkkkey " 3650 MB free held

=========================================
REAL-FREE ALLOCATOR DUMP FOR THIS PROCESS
-----------------------------------------

Dump of Real-Free Memory Allocator Heap [0x7fa8df317000]
mag=0xfefe0001 flg=0x5000003 fds=0x0 blksz=65536
blkdstbl=0x7fa8df317010, iniblk=524288 maxblk=524288 numsegs=318
In-use num=2965 siz=641597440, Freeable num=0 siz=0, Free num=254 siz=3586195456

==========================================
INSTANCE-WIDE PRIVATE MEMORY USAGE SUMMARY
------------------------------------------

Dumping Work Area Table (level=1)
=====================================

Global SGA Info
---------------

global target: 102400 MB
auto target: 62376 MB
max pga: 2048 MB
pga limit: 4096 MB
pga limit known: 0
pga limit errors: 0

pga inuse: 33104 MB
pga alloc: 35238 MB
pga freeable: 1225 MB
pga freed: 433664026 MB
pga to free: 0 %
broker request: 0

pga auto: 12 MB
pga manual: 0 MB

pga alloc (max): 35238 MB
pga auto (max): 12284 MB
pga manual (max): 2 MB

# workareas : 0
# workareas(max): 551
Database, Errors and Bugs, Performance

Oracle Script: hcheck.sql

 , ,

Hi all,
Just documenting here, in case you don’t know this script:

Script to Check for Known Problems in oracle8i, oracle9i, oracle10g, Oracle 11g and Oracle 12c( Doc ID 136697.1 )

SQL> @hcheck.sql
HCheck Version 07MAY18 on 12-AUG-2019 21:51:32
----------------------------------------------
Catalog Version 12.2.0.1.0 (1202000100)
db_name: R360PD
Is CDB?: NO

                                   Catalog       Fixed
Procedure Name                     Version    Vs Release    Timestamp
Result
------------------------------ ... ---------- -- ---------- --------------
------
.- LobNotInObj                 ... 1202000100 <=  *All Rel* 08/12 21:51:32 PASS
.- MissingOIDOnObjCol          ... 1202000100 <=  *All Rel* 08/12 21:51:33 PASS
.- SourceNotInObj              ... 1202000100 <=  *All Rel* 08/12 21:51:33 PASS
.- OversizedFiles              ... 1202000100 <=  *All Rel* 08/12 21:51:33 PASS
.- PoorDefaultStorage          ... 1202000100 <=  *All Rel* 08/12 21:51:33 PASS
.- PoorStorage                 ... 1202000100 <=  *All Rel* 08/12 21:51:33 PASS
.- TabPartCountMismatch        ... 1202000100 <=  *All Rel* 08/12 21:51:33 PASS
.- OrphanedTabComPart          ... 1202000100 <=  *All Rel* 08/12 21:51:34 PASS
.- MissingSum$                 ... 1202000100 <=  *All Rel* 08/12 21:51:38 PASS
.- MissingDir$                 ... 1202000100 <=  *All Rel* 08/12 21:51:38 PASS
.- DuplicateDataobj            ... 1202000100 <=  *All Rel* 08/12 21:51:39 PASS
.- ObjSynMissing               ... 1202000100 <=  *All Rel* 08/12 21:55:56 PASS
.- ObjSeqMissing               ... 1202000100 <=  *All Rel* 08/12 21:55:57 PASS
.- OrphanedUndo                ... 1202000100 <=  *All Rel* 08/12 21:55:57 PASS
.- OrphanedIndex               ... 1202000100 <=  *All Rel* 08/12 21:55:57 PASS
.- OrphanedIndexPartition      ... 1202000100 <=  *All Rel* 08/12 21:55:57 PASS
.- OrphanedIndexSubPartition   ... 1202000100 <=  *All Rel* 08/12 21:55:57 PASS
.- OrphanedTable               ... 1202000100 <=  *All Rel* 08/12 21:56:00 PASS
.- OrphanedTablePartition      ... 1202000100 <=  *All Rel* 08/12 21:56:00 PASS
.- OrphanedTableSubPartition   ... 1202000100 <=  *All Rel* 08/12 21:56:00 PASS
.- MissingPartCol              ... 1202000100 <=  *All Rel* 08/12 21:56:04 PASS
.- OrphanedSeg$                ... 1202000100 <=  *All Rel* 08/12 21:56:04 PASS
.- OrphanedIndPartObj#         ... 1202000100 <=  *All Rel* 08/12 21:56:15 PASS
.- DuplicateBlockUse           ... 1202000100 <=  *All Rel* 08/12 21:56:15 PASS
.- FetUet                      ... 1202000100 <=  *All Rel* 08/12 21:56:17 PASS
.- Uet0Check                   ... 1202000100 <=  *All Rel* 08/12 21:56:17 PASS
.- SeglessUET                  ... 1202000100 <=  *All Rel* 08/12 21:56:18 PASS
.- BadInd$                     ... 1202000100 <=  *All Rel* 08/12 21:56:19 PASS
.- BadTab$                     ... 1202000100 <=  *All Rel* 08/12 21:56:21 PASS
.- BadIcolDepCnt               ... 1202000100 <=  *All Rel* 08/12 21:56:21 PASS
.- ObjIndDobj                  ... 1202000100 <=  *All Rel* 08/12 21:56:21 PASS
.- TrgAfterUpgrade             ... 1202000100 <=  *All Rel* 08/12 21:56:22 PASS
.- ObjType0                    ... 1202000100 <=  *All Rel* 08/12 21:56:22 FAIL

HCKE-0036: Bad OBJ$ entry with TYPE#=0 (Doc ID 1361015.1)
OBJ$ OBJ#=212606505 TYPE#=0 NAME=COMMENTS NAMESPACE=66 Dblink=EMCPROD
OBJ$ OBJ#=212606501 TYPE#=0 NAME=CONTACT NAMESPACE=66 Dblink=EMCPROD
OBJ$ OBJ#=212606496 TYPE#=0 NAME=HT360_CURRENCY_REMOTE NAMESPACE=66
Dblink=EMCPROD
OBJ$ OBJ#=74768696 TYPE#=0 NAME=HT360_USER_ACCESS_MV NAMESPACE=66 Dblink=MDMPROD
OBJ$ OBJ#=183148682 TYPE#=0 NAME=HT360_USER_SUBS_PERM_REMOTE NAMESPACE=66
Dblink=EMCPROD
OBJ$ OBJ#=186229204 TYPE#=0 NAME=HT360_USER_SUBS_PERM_REMOTE NAMESPACE=66
Dblink=EMCPROD
OBJ$ OBJ#=115573869 TYPE#=0 NAME=HT360_USER_SUBS_PERM_REMOTE NAMESPACE=66
Dblink=EMCPROD
OBJ$ OBJ#=212606499 TYPE#=0 NAME=HT360_USER_SUBS_PERM_REMOTE NAMESPACE=66
Dblink=EMCPROD
OBJ$ OBJ#=212606631 TYPE#=0 NAME=TC_HT_SERVICE_DEL_ATTRIBUTES_V NAMESPACE=66
Dblink=EBSPROD
OBJ$ OBJ#=186229477 TYPE#=0 NAME=GDS360_PROPERTY_REMOTE NAMESPACE=66
Dblink=MDMPROD
OBJ$ OBJ#=148134057 TYPE#=0 NAME=MEDIA_ROI_INS NAMESPACE=66 Dblink=EMCPROD
OBJ$ OBJ#=212606497 TYPE#=0 NAME=MEDIA_ROI_INS NAMESPACE=66 Dblink=EMCPROD
OBJ$ OBJ#=75622441 TYPE#=0 NAME=CONTACT_REMOTE NAMESPACE=66 Dblink=EMCPROD
OBJ$ OBJ#=197171249 TYPE#=0 NAME=CONTACT_REMOTE NAMESPACE=66 Dblink=EMCPROD
OBJ$ OBJ#=148134085 TYPE#=0 NAME=CAMPAIGN_DMP NAMESPACE=66 Dblink=EMCPROD
OBJ$ OBJ#=212606500 TYPE#=0 NAME=CAMPAIGN_DMP NAMESPACE=66 Dblink=EMCPROD
OBJ$ OBJ#=148134088 TYPE#=0 NAME=INSERTION_CODE NAMESPACE=66 Dblink=EMCPROD
OBJ$ OBJ#=212606503 TYPE#=0 NAME=INSERTION_CODE NAMESPACE=66 Dblink=EMCPROD
OBJ$ OBJ#=148134080 TYPE#=0 NAME=INSERTION_DMP NAMESPACE=66 Dblink=EMCPROD
OBJ$ OBJ#=212606504 TYPE#=0 NAME=INSERTION_DMP NAMESPACE=66 Dblink=EMCPROD
OBJ$ OBJ#=148134083 TYPE#=0 NAME=INSERTION_ELEMENT NAMESPACE=66 Dblink=EMCPROD
OBJ$ OBJ#=212606502 TYPE#=0 NAME=INSERTION_ELEMENT NAMESPACE=66 Dblink=EMCPROD
OBJ$ OBJ#=172793128 TYPE#=0 NAME=SUPPLIER_CONTACT_REMOTE NAMESPACE=66
Dblink=EMCPROD
OBJ$ OBJ#=197171212 TYPE#=0 NAME=SUPPLIER_CONTACT_REMOTE NAMESPACE=66
Dblink=EMCPROD
OBJ$ OBJ#=183148695 TYPE#=0 NAME=PROPERTY_MAPPING NAMESPACE=66 Dblink=MDMPROD
OBJ$ OBJ#=154666744 TYPE#=0 NAME=PROPERTY_MAPPING NAMESPACE=66 Dblink=MDMPROD

.- BadOwner                    ... 1202000100 <=  *All Rel* 08/12 21:56:22 PASS
.- StmtAuditOnCommit           ... 1202000100 <=  *All Rel* 08/12 21:56:23 PASS
.- BadPublicObjects            ... 1202000100 <=  *All Rel* 08/12 21:56:23 PASS
.- BadSegFreelist              ... 1202000100 <=  *All Rel* 08/12 21:56:23 PASS
.- BadDepends                  ... 1202000100 <=  *All Rel* 08/12 21:56:25 PASS
.- CheckDual                   ... 1202000100 <=  *All Rel* 08/12 21:56:26 PASS
.- ObjectNames                 ... 1202000100 <=  *All Rel* 08/12 21:56:26 WARN

HCKW-0018: OBJECT name clashes with SCHEMA name (Doc ID 2363142.1)
Schema=TC_MONITOR JOB=TC_MONITOR.TC_MONITOR
Schema=TCDWSTAGE DATABASE LINK=PUBLIC.TCDWSTAGE

.- BadCboHiLo                  ... 1202000100 <=  *All Rel* 08/12 21:56:27 PASS
.- ChkIotTs                    ... 1202000100 <=  *All Rel* 08/12 21:56:32 PASS
.- NoSegmentIndex              ... 1202000100 <=  *All Rel* 08/12 21:56:32 PASS
.- BadNextObject               ... 1202000100 <=  *All Rel* 08/12 21:56:32 PASS
.- DroppedROTS                 ... 1202000100 <=  *All Rel* 08/12 21:56:33 PASS
.- FilBlkZero                  ... 1202000100 <=  *All Rel* 08/12 21:56:33 PASS
.- DbmsSchemaCopy              ... 1202000100 <=  *All Rel* 08/12 21:56:33 PASS .- OrphanedIdnseqObj           ... 1202000100 >  1201000000 08/12 21:56:33 PASS
.- OrphanedIdnseqSeq           ... 1202000100 >  1201000000 08/12 21:56:33 PASS
.- OrphanedObjError            ... 1202000100 >  1102000000 08/12 21:56:33 PASS
.- ObjNotLob                   ... 1202000100 <=  *All Rel* 08/12 21:56:34 PASS
.- MaxControlfSeq              ... 1202000100 <=  *All Rel* 08/12 21:56:34 PASS .- SegNotInDeferredStg         ... 1202000100 >  1102000000 08/12 21:56:34 PASS
.- SystemNotRfile1             ... 1202000100 >   902000000 08/12 21:56:35 PASS
.- DictOwnNonDefaultSYSTEM     ... 1202000100 <=  *All Rel* 08/12 21:56:35 PASS
.- OrphanTrigger               ... 1202000100 <=  *All Rel* 08/12 21:56:35 PASS
.- ObjNotTrigger               ... 1202000100 <=  *All Rel* 08/12 21:56:35 PASS 

---------------------------------------
12-AUG-2019 21:56:36  Elapsed: 304 secs
---------------------------------------
Found 26 potential problem(s) and 2 warning(s)
Contact Oracle Support with the output and trace file
to check if the above needs attention or not

PL/SQL procedure successfully completed.

Statement processed.

Complete output is in trace file:
/u01/app/oracle/diag/rdbms/r360pdlo/r360pd3/trace/r360pd3_ora_310651_HCHECK.trc

 

useful, right?

Database, Errors and Bugs

ORA-02019 While SELECT From A View Owned By Another User Using Dblink

 , , , ,

Quick case today.

This week I had a client experiencing ORA-02019 while SELECT from a View with dblink and CONNECT BY PRIOR … START WITH into SELECT.
The situation involved Views on a DB which need to be accessed by a groups of users from another DB using proxy user and a DB link but encounter this ORA error.

The root cause?

ORA-02019 while performing select on a view or while selecting a view owned by another user, with dblink, is a match to Bug 26558437 – DATABASE LINK FAILS WITH ORA-2019 WHEN SELECT ANOTHER USER VIEW.

But MOS doesn’t have workaround besides applying patch, as usual. What we did and solved on our case?

We created a materialized view refreshed every 15 mins (solution supported this delay0 using the DBLink if the view owner.
In this case, the other users instead of executing the query on the view, will be actually querying the table created (and refreshed) by the mview code, which would be only be executed by the mview owner.

By the way, 2 good side effects:
1) Once view was executed more often then the period of refresh, this solution is also saving some efforts database wise, once executing select from mview is way better then the view code, besides not using the network on dblink
2) If the remote database get slow, or down, the data would be still available from last mview refresh.

Conclusions:
1) Use MATERIALIZED VIEW!
2) MOS not always give you all the steps. Sometimes you can easily solve your problem by thinking a little bit more on the root cause problem.

Cheers

Database, Errors and Bugs

ORA-00600: internal error code, arguments: [kghfrempty:ds]

 , , , ,

Hi all,
Annoying thing, just because that sounded veeery unsual to me. I would expect a different error.

I have seen this “[kghfrempty:ds]” in past a couple times, and if you did see it too you may have noticed, this is almost as much generic as the ORA-600 itself. It happens because this is usually in the top of KGH errors in general. Some examples:

1.

kghnerror <- kghfrempty <- kghfrempty_ex <- qerhjFreeSpace
<- qerhjFetch <- qersoProcessULS <- qersoFetch <- opifch2 <- kpoal8
<- opiodr <- kpoodrc <- rpiswu2 <- kpoodr <- upirtrc
<- kpurcsc <- kpufch0 <- kpufch <- OCIStmtFetch2 <- qksanExecSql <- qksanAnalyzeSql

2.

kghfrempty <- kghfrempty_partial <- kghgex <- kghfnd <- kghalo <- kghgex <- kghalf <- ktmrProcessCRClone 
<- ktmbRead <- ktmrget_int <- ktmrget <- kdmsTransGet <- kdst_fetch_imc  <- kdsttgr  <- qertbFetch

And as being it can be related either to analyze bugs (as per stack 1), In-Memory Column Store – IMC (as per stack 2), create views, rollback over cluster, trigger creation, use function based indexes, or several other generic situations (there is no pattern, as you can see).

So what do to if you get this message? Well, generically, read all the MOS notes available and try to match you case. IF you match, pray to have a bug for your current version, or open an SR to either have a backport or start a new investigation.

A few examples of things to check:

  • ALERT: Bug 13605839 ORA-600 [ktbsdp1] ORA-600 [kghfrempty:ds] ORA-600 [kdBlkCheckError]. Corruption in Rollback with Clusterwide Global Transactions in RAC (Doc ID 1527740.1)
  • SQL Analyze Fails with ORA-600[kghfrempty:ds] (Doc ID 2247180.1)
    Bug 21556276 – dataguide createviewonpath hit ora 600 [kghfrempty:ds] (Doc ID 21556276.8)
  • Bug 20878625 – ORA-600[kghfrempty:ds] with IMC enabled (Doc ID 20878625.8)
  • Bug 19212166 – ORA-600 [KGHFREMPTY:DS] With Cache Line Alignment Enabled (Doc ID 19212166.8)
  • Bug 6797925 – ORA-600 [kghfrempty:ds] During Creation Of Trigger In XML Schema (Doc ID 6797925.8)
  • 11.1: ORA-600 [17147] AND [kghfrempty:ds] On Select Using Functional Indexes (Doc ID 884882.1)
  • ORA-600 [kghfrempty:ds] And ORA-600 [kghrcdepth:ds] While Gathering Statistics (Doc ID 1480132.1)

BUUT I just noticed some undocumented scenario causing this and it might help you on fixing this as well.
I noticed I was getting this error for different queries but always from same application. So I addressed a complete review on application layer and noticed the JDBC 7 on this application. Happens we had just upgraded from 12.1 to 19c.

SO, after upgrading jdbc, it started to work again properly and no MOS SR was required. So lesson learned: When you receive this kghfrempty:ds], speacially after migration, also check for your application JDBC version.

For compatibility Matrix: Starting With Oracle JDBC Drivers – Installation, Certification, and More! ( Doc ID 401934.1 )

Hope it helps, cheers!