OEM: HttpConnectionException:Response code:404

Hello all,
So, a client’s OEM Information Report was running and email being sent, but all queries (Report sections) resulting on error:

Unexpected error getting the HTTP response stream while generating report: oracle.sysman.eml.ip.publishJob.HttpConnectionException:Response code:404

 

Weird?
I had a silar issue, as per documented on this post: OEM Information Reports: ORA-00600 [kpndbcon-svchpnotNULL]

This is happening due TCP timeout while executing Report after 5 minutes. By default, Idle Timeout set for Profile parameter (TCP) for the virtual server in F5 is 300 seconds, for example.
I simulated manually execution of each SQL on report and noticed report takes around 8 minutes to run.

What I did? I took some of the slow queries and transformed to a Materialized View with daily refresh.

This was valid in my case because the report runs only once a day. Remember to always check on your own situation. Perhaps an hourly refresh or so is better for you.

Another possibility is to engage the LB admin to modify this parameter by setting a new protocol profile for the virtual server or modifying the existing one.

NOTE: After modifying SLB Values, you may need to restart OMS :

emctl stop oms -all
emctl start oms

Reference?
MOS oracle.sysman.eml.ip.publishJob.HttpConnectionException:Response code:404″ Errors for reports run from EM (Doc ID 2061361.1)

Oracle: Explicitly Lock a Table

Hello all,
So I client had a process that kept need big chunks locks on a table. Problem is that due application use this lock occasionally failed (lock timeout) and the whole process fail. So how to guarantee this?

Using explicit locks for the table before starting the procedure, with command below:

LOCK TABLE GREPORA.GREP_TABLE IN SHARE MODE;

This lock is released as soon as the session commit or rollback transaction in the current session. Killing session means an implicit rollback, as usual.

The SHARE permits concurrent queries but prohibits updates to the locked table.

Nice right?
Hope it helps you too!

Adding new PDB to a CDB with Standby

On Oracle 12c we got a new feature on RMAN which allows you to restore database files, over the network, from a physical standby database by using the FROM SERVICE clause of the RESTORE command.

So in a Data Guard environment in case you add a new Pluggable database (PDB) you don’t need to duplicate the entire Container database (CDB) again. You can restore only the new PDB to the physical standby. Cool right, less work, faster work. I would prefer this as RMAN would do all its magic and we would not need to manual copies to renames.

On the practical side, this would be done in case you need to add a new PDB in your Data Guard configuration.

Stop the archive log apply on the standby using DG broker

validate database verbose PRIMARY;
validate database verbose STANDBY;
edit database 'STANDBY' set state='apply-off';
On the standby database restore the new PDB

rman target /
run{
allocate channel prim1 type disk;
set newname for pluggable database NEW_PDB to new;
restore pluggable database NEW_PDB from service PRIMARY;
switch datafile all;
}

– Enable recovery on the new PDB, if you running an Active DataGuard you need to stop it and start the standby database in mount, also if you get ORA-1113 make sure you are in MOUNT stage.

sqlplus / as sysdba
shtudown immediate
startup mount
alter session set container=NEW_PDB;
alter pluggable database enable recovery;
exit

– Enable archive log apply

edit database 'STANDBY' set state='apply-on';
validate database verbose PRIMARY;
validate database verbose STANDBY;

More information you can find on the MOS Note Making Use Deferred PDB Recovery and the STANDBYS=NONE Feature with Oracle Multitenant (Doc ID 1916648.1)

Hope it helps,
Elisson Almeida

OEM Report: Last 6 month Database Space Usage and Growth

Hello All!
So I had worked in some very useful reports to have in OEM. In next weeks I’ll share some code you may like… 🙂

To create it? Enterprise -> Reports -> Information Reports. There are several nice default reports there you me like.

Now, let’s go to the first Report, as per title:

1. First you Select the Database:

Select_DB.png

2. Then you see the report:

Report3

That’s nice, right?
Here is a report for another database with actual 640GB average growth per month (its expected, once DBSize is over 60TB):

Growth

So, Mat, can you share the queries? Of course:

Continue reading

Oracle Trigger Follows Clause: Simultaneous Ordered Triggers

Hi All!
I was reviewing some features in Oracle and, basically, every single time I review them I find something new. Seems Oracle Databases’ features are near to infinite and we frequently find some that can really add value to our solutions.

So here is something you may not be aware and is really cool:

Follows Clause

Oracle allows more than one trigger to be created for the same timing point, but up to version 11g is not possible to establish the execution order of execution. The Oracle 11g trigger syntax now includes the FOLLOWS clause to guarantee execution order for triggers defined with the same timing point.

The example below establish that second_trigger is executed after first_trigger before inserting each row in table_example.

CREATE OR REPLACE TRIGGER second_trigger
BEFORE INSERT ON table_example
FOR EACH ROW
FOLLOWS first_trigger
BEGIN NULL; END;
/

Nice, han?
Cheers!

12cR1 – Stop CRS: OC4J Fails to Stop

Hi all!
So, I started to to see this error to stop a 12.1.0.2 CRS :

CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on 'grepora-srv-01'
CRS-2673: Attempting to stop 'ora.crsd' on 'grepora-srv-01'
CRS-2790: Starting shutdown of Cluster Ready Services-managed resources on 'grepora-srv-01'
CRS-2679: Attempting to clean 'ora.oc4j' on 'grepora-srv-01'
CRS-2680: Clean of 'ora.oc4j' on 'grepora-srv-01' failed
CRS-2799: Failed to shut down resource 'ora.oc4j' on 'grepora-srv-01'
CRS-2794: Shutdown of Cluster Ready Services-managed resources on 'grepora-srv-01' has failed
CRS-2675: Stop of 'ora.crsd' on 'grepora-srv-01' failed
CRS-2799: Failed to shut down resource 'ora.crsd' on 'grepora-srv-01'
CRS-2795: Shutdown of Oracle High Availability Services-managed resources on 'grepora-srv-01' has failed
CRS-4687: Shutdown command has completed with errors.

When running a crsctl stat res -t it will show something like this

ora.oc4j 1 ONLINE UNKNOWN grpora-srv-01 CLEANING

Solution? Follow The rootupgrade.sh fails because the oc4j resource fails to stop (Doc ID 1921629.1):

1) Stop the OC4J resource, as grid user:

 srvctl stop oc4j -f

Note: Even if the oc4j does not stop, continue with step 2
2) Disable the OC4J resource, as grid user:

 srvctl disable oc4j

3) Confirm the OC4J resource is NOT running, as grid user:

 srvctl status oc4j

If the OC4J resource is still running, go to the node where oc4j is running and recycle GI by issuing the following as root:

 crsctl stop crs -f
 crsctl start crs

Why?
oc4j is used only for Quality of Service Management (QoS feature), as per:
https://docs.oracle.com/database/121/APQOS/install_config.htm#APQOS151
http://docs.oracle.com/cd/E11882_01/server.112/e24611.pdf

Searched on MOS and only found notes saying we cannot remove default services (it is not supported). But as a DBA, in the middle of the night with environment down, this may be a solution for you, specially if you are not using QoS.

Hope it helps. Cheers!

Check Ports in use by OEM

So, how to quickly check which ports are indeed in use by OEM?

Easy:

[oracle@greporasrv ~]$ cat $OMS_HOME/install/portlist.ini
Enterprise Manager Upload Http Port=4889
Enterprise Manager Upload Http SSL Port=4903
OHS Http Port=9788
BI Publisher Http Port=9701
Enterprise Manager Central Console Http SSL Port=7802
Node Manager Http SSL Port=7403
BI Publisher Http SSL Port=9803
OHS Http SSL Port=9851
Managed Server Http Port=7202
Oracle Management Agent Port=3872
Enterprise Manager Central Console Http Port=7788
Admin Server Http SSL Port=7102
Managed Server Http SSL Port=7301

Hope it helps!

Regular Expressions with SUBEXPR

Hi All!

Did you know since 11g the REGEXP_INSTR and REGEXP_SUBSTR functions include a new SUBEXPR parameter that limits the pattern match to a specific subexpression in the search pattern?

Also, a new function, REGEXP_COUNT, returns the number of times the search pattern appears in source string.

Below a simple example of each one:

SQL> SELECT REGEXP_INSTR('1234567890', '(123)(4(56)(78))', 1, 1, 0, 'i', 3) FROM dual;

REGEXP_INSTR('1234567890','(123)(4(56)(78))',1,1,0,'I',3)
---------------------------------------------------------
                                                        5

SQL> SELECT REGEXP_SUBSTR('1234567890', '(123)(4(56)(78))', 1, 1, 'i', 3) FROM dual;

RE
--
56

SQL> SELECT REGEXP_COUNT('123 123 123 123', '123', 1, 'i') FROM dual;

REGEXP_COUNT('123123123123','123',1,'I')
----------------------------------------
                                       4

Cheers!

ORA-31600: invalid input value DATABASE LINK for parameter OBJECT_TYPE in function GET_DDL When Dropping DBLink

So while back there was an error while trying to drop a public database link

SQL> drop public database link "BOB.DATABASE_LINK_1";
drop public database link "BOB.DATABASE_LINK_1"
*
ERROR at line 1:
ORA-00604: error occurred at recursive SQL level 1
ORA-31600: invalid input value DATABASE LINK for parameter OBJECT_TYPE in
function GET_DDL
ORA-06512: at "SYS.DBMS_METADATA", line 5805
ORA-06512: at "SYS.DBMS_METADATA", line 8344
ORA-06512: at line 14

And the even strange part is that executing DBMS_METADATE.get_ddl worked against the database link

SQL> SELECT dbms_metadata.get_ddl('DB_LINK','BOB.DATABASE_LINK_1','SYS') FROM dual;

CREATE DATABASE LINK "BOB.DATABASE_LINK_1"
CONNECT TO "NYU" IDENTIFIED BY VALUES '*******'
USING 'ORCL';

The solution relied on MOS note Cannot drop a database link after changing the global_name ORA-02024 (Doc ID 382994.1) all though the symptoms in the note were not a match to the problem above the solution did the trick.
Which is to backup the table sys.link$ as CTAS and delete the row corresponding to the database link which you need to drop.

Take a complete consistent backup of the database or use CTAS can be used to backup sys.link$:

$sqlplus /nolog
connect / as sysdba

SQL> create table backup_link$ as select * from sys.link$:

Delete the DBLINK as follows:

$sqlplus /nolog
connect / as sysdba

SQL> delete sys.link$ where name='db_link_name>';

SQL>commit;

Verify if the operation was correctly proceeded:

select db_link, username, host from user_db_links;

Hope it helps!