ORA-12012: error on auto execute of job “SYS”.”BSLN_MAINTAIN_STATS_JOB”

Hi all,
Evaluating a database I detected it was failing to execute the default scheduler job SYS.BSLN_MAINTAIN_STATS_JOB. This job is an Oracle defined automatic moving window baseline statistics computation job, that runs only in weekends.
Below the last stack error in the alert log:

2016-04-24 00:00:10.064000 +00:00
Errors in file /db/u1001/oracle/diag/rdbms/MYDB/MYDB/trace/MYDB_j000_15675.trc:
ORA-12012: error on auto execute of job "SYS"."BSLN_MAINTAIN_STATS_JOB"
ORA-06502: PL/SQL: numeric or value error
ORA-06512: at "DBSNMP.BSLN_INTERNAL", line 2073
ORA-06512: at line 1
2016-04-26 15:54:07.480000 +00:00

And the full tracefile:

Trace file /db/u1001/oracle/diag/rdbms/MYDB/MYDB/trace/MYDB_j000_15675.trc
Oracle Database 11g Enterprise Edition Release 11.2.0.3.0 - 64bit Production
With the Partitioning, OLAP, Data Mining and Real Application Testing options
ORACLE_HOME = /u01/app/oracle/product/11.2
System name:    Linux
Node name:      prddb09
Release:        2.6.18-164.el5
Version:        #1 SMP Tue Aug 18 15:51:48 EDT 2009
Machine:        x86_64
Instance name: MYDB
Redo thread mounted by this instance: 1
Oracle process number: 151
Unix process pid: 15675, image: oracle@prddb09 (J000)
*** 2016-04-24 00:00:10.064
*** SESSION ID:(586.10305) 2016-04-24 00:00:10.064
*** CLIENT ID:() 2016-04-24 00:00:10.064
*** SERVICE NAME:(SYS$USERS) 2016-04-24 00:00:10.064
*** MODULE NAME:(DBMS_SCHEDULER) 2016-04-24 00:00:10.064
*** ACTION NAME:(BSLN_MAINTAIN_STATS_JOB) 2016-04-24 00:00:10.064
ORA-12012: error on auto execute of job "SYS"."BSLN_MAINTAIN_STATS_JOB"
ORA-06502: PL/SQL: numeric or value error
ORA-06512: at "DBSNMP.BSLN_INTERNAL", line 2073
ORA-06512: at line 1

According the notes below, the recommended action is to recreate the DBSNMP component:
Bug 10110625 – DBSNMP.BSLN_INTERNAL reports ORA-6502 running BSLN_MAINTAIN_STATS_JOB (Doc ID 10110625.8)
ORA-12012: Error on Auto Execute of job SYS.BSLN_MAINTAIN_STATS_JOB (Doc ID 1413756.1)
KEWBMBTA: Maintain BSLN Thresholds Failed, Check For Details. (Doc ID 1490391.1)

However, it’s a process that can affect other mechanisms. So, I found the follow note with the same error pointing to a privilege issue:
Ora-06508: Pl/Sql: Could Not Find Program Unit Being Called: “DBSNMP.BSLN_INTERNAL” (Doc ID 1323597.1)

But after granting the privilege as workaround suggested, the fail remais…

MYDB> select * from dba_tab_privs where table_name='DBMS_JOB';
GRANTEE                        OWNER                          TABLE_NAME                     GRANTOR                        PRIVILEGE
------------------------------ ------------------------------ ------------------------------ ------------------------------ -----------
APEX_030200                    SYS                            DBMS_JOB                       SYS                            EXECUTE
SYSMAN                         SYS                            DBMS_JOB                       SYS                            EXECUTE
EXFSYS                         SYS                            DBMS_JOB                       SYS                            EXECUTE
PUBLIC                         SYS                            DBMS_JOB                       SYS                            EXECUTE
SQL> GRANT EXECUTE ON sys.dbms_job to DBSNMP;
Grant succeeded.
MYDB> select * from dba_tab_privs where table_name='DBMS_JOB';
GRANTEE                        OWNER                          TABLE_NAME                     GRANTOR                        PRIVILEGE
------------------------------ ------------------------------ ------------------------------ ------------------------------ --------------
SYSMAN                         SYS                            DBMS_JOB                       SYS                            EXECUTE
APEX_030200                    SYS                            DBMS_JOB                       SYS                            EXECUTE
EXFSYS                         SYS                            DBMS_JOB                       SYS                            EXECUTE
DBSNMP                         SYS                            DBMS_JOB                       SYS                            EXECUTE
PUBLIC                         SYS                            DBMS_JOB                       SYS                            EXECUTE
SQL> EXEC DBMS_SCHEDULER.RUN_JOB('BSLN_MAINTAIN_STATS_JOB');
BEGIN DBMS_SCHEDULER.RUN_JOB('BSLN_MAINTAIN_STATS_JOB'); END;
*
ERROR at line 1:
ORA-06502: PL/SQL: numeric or value error
ORA-06512: at "DBSNMP.BSLN_INTERNAL", line 2073
ORA-06512: at line 1
ORA-06512: at "SYS.DBMS_ISCHED", line 185
ORA-06512: at "SYS.DBMS_SCHEDULER", line 486
ORA-06512: at line 1

After that, while I was quering on DBSNMP, I realized another instance name active in DBSNMP.BSLN_BASELINES.
I guess this database was created with another instance name and then renamed without DBNID.

MYDB> select * from DBSNMP.BSLN_BASELINES;
DBID INSTANCE_NAME    BASELINE_ID BSLN_GUID                        TI A STATUS
---------- ---------------- ----------- -------------------------------- -- - ---------
4092499541 MYDB                       0 75B49690F8B4742084990643EEFFB6AA HX Y ACTIVE
4092499541 oldname                    0 415373CD9959B77AAEE1804F06D88B60 NW Y ACTIVE

So, I deleted the row and the job started to run successfully:

MYDB> DELETE FROM DBSNMP.BSLN_BASELINES WHERE INSTANCE_NAME ='oldname';
1 row deleted.
MYDB> commit;
Commit complete.
SQL> EXEC DBMS_SCHEDULER.RUN_JOB('BSLN_MAINTAIN_STATS_JOB');
PL/SQL procedure successfully completed.

Execution logs:

MYDB> select *
2    from (select owner, job_name, log_date, status, run_duration
3            from dba_scheduler_job_run_details a
4           where job_name = 'BSLN_MAINTAIN_STATS_JOB'
5           order by log_date)
6   where rownum < 10;
OWNER                          JOB_NAME                  LOG_DATE                            STATUS          RUN_DURATION
------------------------------ ------------------------- ----------------------------------- --------------- ---------------
SYS                            BSLN_MAINTAIN_STATS_JOB   03/04/16 00:00:08,484972 +00:00     FAILED          +000 00:00:08
SYS                            BSLN_MAINTAIN_STATS_JOB   10/04/16 00:00:07,943598 +00:00     FAILED          +000 00:00:07
SYS                            BSLN_MAINTAIN_STATS_JOB   17/04/16 00:00:08,486526 +00:00     FAILED          +000 00:00:08
SYS                            BSLN_MAINTAIN_STATS_JOB   24/04/16 00:00:10,067848 +00:00     FAILED          +000 00:00:09
SYS                            BSLN_MAINTAIN_STATS_JOB   29/04/16 13:58:10,779201 +00:00     FAILED          +000 00:00:01
SYS                            BSLN_MAINTAIN_STATS_JOB   29/04/16 14:01:04,162900 +00:00     SUCCEEDED       +000 00:00:00

I hope it help you too!

Matheus.

Failure unregister integrated extract

Some times it’s impossible to unregister Integrated Extract, however it need to exclude to avoid RMAN failures.

Follow below to hack GoldenGate registration:

SQL> select CAPTURE_NAME from dba_capture;

CAPTURE_NAME
------------------------------
OGG$CAP_IE_CAPT

GGSCI (myhost as ggate@foodb) 13> unregister extract IE_CAPT database

ERROR OGG-08222 EXTRACT IE_CAPT must be registered with the database to perform this operation.

Try it:

SQL> select 'exec DBMS_CAPTURE_ADM.DROP_CAPTURE ('''||capture_name||''');' from dba_capture;

'EXECDBMS_CAPTURE_ADM.DROP_CAPTURE('''||CAPTURE_NAME||''');'
----------------------------------------------------------------------
exec DBMS_CAPTURE_ADM.DROP_CAPTURE ('OGG$CAP_IE_CAPT');

SQL> exec DBMS_CAPTURE_ADM.DROP_CAPTURE ('OGG$CAP_IE_CAPT');

PL/SQL procedure successfully completed.

Maiquel.

ERROR OGG-02636 when creating a integrated extract in Goldengate 12C on a Puggable database 12C

While creating an integrated extract in Goldengate 12C on a Puggable database 12C I came across the follow error, stating that the needed catalog name is mandatory  and was not being informed.

ERROR OGG-02636 Oracle GoldenGate Capture for Oracle, ext1.prm: The TABLE specification ‘TABLE table_name’ for the source table ‘table_name’ does not include a catalog name. The database requires a catalog name.

There is two ways to solve this case: The first , besides less indicated, is to add the name of pluggable database (Catalog) before the owner name on the table maps, for example:

GGSCI (host1.net) 1> edit param ext1

–Tables
TABLE PDB_NAME.SCHEMA_OWNER.TABLE_NAME;

Not really enjoying this solution and after searching for long hours without any other result , our friend Maiquel DC indicated a parameter that identifies the catalog name for all tables in the extract ;

Add the following parameter in extract  configuration file:

GGSCI (host1.net) 1> edit param ext1

–Parameters
SOURCECATALOG PDB_NAME

 

Thats all folks.
Dieison.

RMAN Raise ORA-19913 ORA-28365 On Restore from Cloud Backup

First I think was some error with Database Backup To Cloud, when testing. Then I realized it was a simple mistake by myself.

Let me show you. First trying to restore datafile:

[oracle@mydbsrvr archivelogs]$ rman target /
RMAN> restore datafile 6;
Starting restore at 03-MAY-2016 20:00:30
using channel ORA_SBT_TAPE_1
allocated channel: ORA_DISK_1
channel ORA_DISK_1: SID=178 device type=DISK
channel ORA_SBT_TAPE_1: starting datafile backup set restore
channel ORA_SBT_TAPE_1: specifying datafile(s) to restore from backup set
channel ORA_SBT_TAPE_1: restoring datafile 00006 to /db/u1001/test/cloud_test/test_restore.dbf
channel ORA_SBT_TAPE_1: reading from backup piece 0sr4mdun_1_1
RMAN-00571: ===========================================================
RMAN-00569: =============== ERROR MESSAGE STACK FOLLOWS ===============
RMAN-00571: ===========================================================
RMAN-03002: failure of restore command at 05/03/2016 20:00:34
ORA-19870: error while restoring backup piece 0sr4mdun_1_1
ORA-19913: unable to decrypt backup
ORA-28365: wallet is not open

Ok, it might happen because I forgot to set encryption password:

RMAN> SET ENCRYPTION ON IDENTIFIED BY "matheusdba" only;
executing command: SET encryption
RMAN> restore datafile 6;
Starting restore at 03-MAY-2016 20:00:30
using channel ORA_SBT_TAPE_1
allocated channel: ORA_DISK_1
channel ORA_DISK_1: SID=178 device type=DISK
channel ORA_SBT_TAPE_1: starting datafile backup set restore
channel ORA_SBT_TAPE_1: specifying datafile(s) to restore from backup set
channel ORA_SBT_TAPE_1: restoring datafile 00006 to /db/u1001/test/cloud_test/test_restore.dbf
channel ORA_SBT_TAPE_1: reading from backup piece 0sr4mdun_1_1
RMAN-00571: ===========================================================
RMAN-00569: =============== ERROR MESSAGE STACK FOLLOWS ===============
RMAN-00571: ===========================================================
RMAN-03002: failure of restore command at 05/03/2016 20:00:34
ORA-19870: error while restoring backup piece 0sr4mdun_1_1
ORA-19913: unable to decrypt backup
ORA-28365: wallet is not open

It hapen again?
This point I suspect some kind of bug… But it was my mistake and is not related to Cloud, but to Encryption use. To undestand:
For Backup: Use ENCRYPTION
For Restore/Recover: Use DECRYPTION

Obviously, but take me a minute to realize…

Setting decryption, and problem solved:

RMAN> set DECRYPTION identified by "matheusdba";
executing command: SET decryption
RMAN> restore datafile 6;
Starting restore at 03-MAY-2016 20:00:58
using channel ORA_SBT_TAPE_1
using channel ORA_DISK_1
channel ORA_SBT_TAPE_1: starting datafile backup set restore
channel ORA_SBT_TAPE_1: specifying datafile(s) to restore from backup set
channel ORA_SBT_TAPE_1: restoring datafile 00006 to /db/u1001/test/cloud_test/test_restore.dbf
channel ORA_SBT_TAPE_1: reading from backup piece 0sr4mdun_1_1
channel ORA_SBT_TAPE_1: piece handle=0sr4mdun_1_1 tag=TAG20160503T193239
channel ORA_SBT_TAPE_1: restored backup piece 1
channel ORA_SBT_TAPE_1: restore complete, elapsed time: 00:00:03
Finished restore at 03-MAY-2016 20:01:02

See ya!
Matheus.

ERROR OGG-00446 – Unable to lock file “*” (error 11, Resource temporarily unavailable).

GoldenGate 12c was running over NFS filesystem and had unexpected stop then when it try starts take OGG-00446.

ERROR OGG-00446 Oracle GoldenGate Capture for Oracle, e_crm01.prm: Unable to lock file “/mnt/ggate/dirchk/MYCRM.cpe” (error 11, Resource temporarily unavailable).

Here the solution:
Move the file /mnt/ggate/dirchk/MYCRM.cpe to
/mnt/ggate/dirchk/MYCRM.cpe_backup

Then copy /mnt/ggate/dirchk/MYCRM.cpe_backup to
/mnt/ggate/dirchk/MYCRM.cpe

I can’t understand why Oracle keeps this stupid bugs on 12c.

C’est La Vie!
Maiquel.

MySQL: Unable to connect to database ‘xxx’ on server ‘xxx’ on port xx with user ‘root’

Quick tip:

# Problem:

MySQL: Unable to connect to database 'xxx' on server 'xxx' on port xx with user 'root' - Access denied for user 'root'@'xxxxx'

 

Solution:

GRANT ALL PRIVILEGES ON *.* TO root@'xxxxx' IDENTIFIED BY '$PASSWORD' WITH GRANT OPTION;
FLUSH PRIVILEGES;

 

Have a nice week!
Matheus.

Tip for the Future: Segmentation fault because of LD_LIBRARY_PATH

More than once I forgot to set LD_LIBRARY_PATH in new environments and sometimes I faced awkward errors. The most common is “Segmentation Fault”.
Today a lost almost 15 minutes searching about Segmentation Fault related to Datapump on 11.2, then I realized I forgot the LD_LIBRARY_PATH again…

Other day, in a Upgrade from 11.2.0.3.6 to 11.2.0.4.2 I get stuck in lots of errors on upgrade process. Bullshit again, after a few minutes of errors and searching I founded a post, somewhere, talking about the variables setting.

So, Matheus from the Future: Check if LB_LIBRARY_PATH and other variables are setted for the right Oracle Home.

I expect this post save me from this same pain in the future. 😛
Thanks.

Matheus.

Explain ORA-XXX on SQL*Plus

For those when the error is unkown/rare, SQL*Plus helps us. It’s just call “oerr” from OS.

See the Linux example (made on RHEL):

SQL>  !oerr ora 01652
01652, 00000, "unable to extend temp segment by %s in tablespace %s"
// *Cause:  Failed to allocate an extent of the required number of blocks for
//          a temporary segment in the tablespace indicated.
// *Action: Use ALTER TABLESPACE ADD DATAFILE statement to add one or more
//          files to the tablespace indicated.

Pretty cool, han?

Have a nice week!
Matheus.

ORA-01994: GRANT failed: password file missing or disabled

Quick tip:

[oracle@server ~]$ orapwd file=$ORACLE_HOME/dbs/orapw$ORACLE_SID password=$password entries=$num_users force=y

 

KB: http://docs.oracle.com/cd/B28359_01/server.111/b28310/dba007.htm#ADMIN12478

# OBS 1
“If you are running multiple instances of Oracle Database using Oracle Real Application Clusters, the environment variable for each instance should point to the same password file.”

# OBS 2
REMOTE_LOGIN_PASSWORDFILE need to be in EXCLUSIVE to alter user with sysdba.

# OBS 3
Users can be chacked on V$PWFILE_USERS.

# OBS 4
Entries represent the quantity of users on orapwd/with sysdba.

Matheus.

11.2.0.1: ORA-00600: internal error code, arguments: [7005], [0], [], [], [], [], [], [], [], [], [], []

# Error

Errors in file /oracle/diag/rdbms/mydb/mydb_1/trace/mydb_1_ora_972.trc (incident=195818):
ORA-00600: internal error code, arguments: [7005], [0], [], [], [], [], [], [], [], [], [], []
Incident details in: /oracle/diag/rdbms/mydb/mydb_1/incident/incdir_195818/mydb_1_ora_972_i195818.trc

#Cause
The query causing this error uses a CONTAINS clause on alphanumerical column using bind variables. This is a perfect match with note ORA-0600 [7005] on a Select Query Using Contains Clause (Doc ID 1176276.1), referencing the unpublished Bug 8770557 ORA-600 [7005] While Running Text Queries.
The symptoms includes this two key factors:
– presence of CONTAINS clause
– use of bind variables

# Solution
Apply 11.2.0.2 patchset or higher, where this issue is fixed or Apply one off Patch 8770557 if available for your version / platform.

See ya!
Matheus.