Tag: cpu

Database, Migration | Upgrades | Patching

Opatchauto Failing on “CheckActiveFilesAndExecutables” during Prerequisite Check

 , , ,

Hi all,
So, very recently when applying the 2021 January CPU in a client environment, the following happened:

[root@dbserver01 32226239]# $ORACLE_HOME/OPatch/opatchauto apply

OPatchauto session is initiated at Sun Mar 14 03:00:06 2021

System initialization log file is /u01/app/oracle/product/19c/grid/cfgtoollogs/opatchautodb/systemconfig2021-03-14_03-00-08AM.log.

Session log file is /u01/app/oracle/product/19c/grid/cfgtoollogs/opatchauto/opatchauto2021-03-14_03-00-13AM.log
The id for this session is 1J89

Executing OPatch prereq operations to verify patch applicability on home /u01/app/oracle/product/19c/db
Patch applicability verified successfully on home /u01/app/oracle/product/19c/db


Executing patch validation checks on home /u01/app/oracle/product/19c/db
Patch validation checks successfully completed on home /u01/app/oracle/product/19c/db


Verifying SQL patch applicability on home /u01/app/oracle/product/19c/db
SQL patch applicability verified successfully on home /u01/app/oracle/product/19c/db


Executing OPatch prereq operations to verify patch applicability on home /u01/app/oracle/product/19c/grid
Patch applicability verified successfully on home /u01/app/oracle/product/19c/grid


Executing patch validation checks on home /u01/app/oracle/product/19c/grid
Patch validation checks successfully completed on home /u01/app/oracle/product/19c/grid


Preparing to bring down database service on home /u01/app/oracle/product/19c/db
Successfully prepared home /u01/app/oracle/product/19c/db to bring down database service


Bringing down database service on home /u01/app/oracle/product/19c/db
Following database has been stopped and will be restarted later during the session: er1pprd,obiee
Database service successfully brought down on home /u01/app/oracle/product/19c/db


Performing prepatch operations on CRS - bringing down CRS service on home /u01/app/oracle/product/19c/grid
Prepatch operation log file location: /u01/app/oracle/product/crsdata/dbserver01/crsconfig/hapatch_2021-03-14_03-06-15AM.log
CRS service brought down successfully on home /u01/app/oracle/product/19c/grid


Start applying binary patch on home /u01/app/oracle/product/19c/db
Failed while applying binary patches on home /u01/app/oracle/product/19c/db

Execution of [OPatchAutoBinaryAction] patch action failed, check log for more details. Failures:
Patch Target : dbserver01->/u01/app/oracle/product/19c/db Type[sidb]
Details: [
---------------------------Patching Failed---------------------------------
Command execution failed during patching in home: /u01/app/oracle/product/19c/db, host: dbserver01.
Command failed: /u01/app/oracle/product/19c/db/OPatch/opatchauto apply /ora02/soft/jan21cpu/32126842/32226239 -oh /u01/app/oracle/product/19c/db -target_type oracle_database -binary -invPtrLoc /u01/app/oracle/product/19c/grid/oraInst.loc -jre /u01/app/oracle/product/19c/grid/OPatch/jre -persistresult /u01/app/oracle/product/19c/db/opatchautocfg/db/sessioninfo/sessionresult_dbserver01_sidb_2.ser -analyzedresult /u01/app/oracle/product/19c/db/opatchautocfg/db/sessioninfo/sessionresult_analyze_dbserver01_sidb_2.ser
Command failure output:
==Following patches FAILED in apply:

Patch: /ora02/soft/jan21cpu/32126842/32226239/32218454
Log: /u01/app/oracle/product/19c/db/cfgtoollogs/opatchauto/core/opatch/opatch2021-03-14_03-17-58AM_1.log
Reason: Failed during Patching: oracle.opatch.opatchsdk.OPatchException: Prerequisite check "CheckActiveFilesAndExecutables" failed.

After fixing the cause of failure Run opatchauto resume

]
OPATCHAUTO-68061: The orchestration engine failed.
OPATCHAUTO-68061: The orchestration engine failed with return code 1
OPATCHAUTO-68061: Check the log for more details.
OPatchAuto failed.

OPatchauto session completed at Sun Mar 14 03:19:25 2021
Time taken to complete the session 19 minutes, 19 seconds

opatchauto failed with error code 42

OK, going by parts, let's see what we have on the refered log:

[Mar 14, 2021 3:33:32 AM] [INFO] Start fuser command /sbin/fuser /u01/app/oracle/product/19c/grid/bin/expdp at Sat Mar 14 03:33:32 PDT 2021
[Mar 14, 2021 3:33:32 AM] [INFO] Finish fuser command /sbin/fuser /u01/app/oracle/product/19c/grid/bin/expdp at Sat Mar 14 03:33:32 PDT 2021
[Mar 14, 2021 3:33:32 AM] [INFO] Following active executables are not used by opatch process :


Following active executables are used by opatch process :
/u01/app/oracle/product/19c/grid/lib/libclntsh.so.19.1
[Mar 14, 2021 3:33:32 AM] [INFO] Prerequisite check "CheckActiveFilesAndExecutables" failed.
The details are:


Following active executables are not used by opatch process :


Following active executables are used by opatch process :
/u01/app/oracle/product/19c/grid/lib/libclntsh.so.19.1
[Mar 14, 2021 3:33:33 AM] [INFO] UtilSession failed: Prerequisite check "CheckActiveFilesAndExecutables" failed.
[Mar 14, 2021 3:33:33 AM] [SEVERE] OUI-67073:UtilSession failed: Prerequisite check "CheckActiveFilesAndExecutables" failed.
[Mar 14, 2021 3:33:33 AM] [INFO] Finishing UtilSession at Sat Mar 14 03:33:33 PDT 2021
[Mar 14, 2021 3:33:33 AM] [INFO] Log file location: /u01/app/oracle/product/19c/grid/cfgtoollogs/opatchauto/core/opatch/oapatch_2021-03-14_03-06-15AM.log

This is an interesting situation.

After some validations making sure no service is online, the path is writable, oracle and root have the required privilege and access, I found some relevant Oracle notes:

  • 19c Installation Fails with error “libclntsh.so: file format not recognized; treating as linker script” (Doc ID 2631283.1): Pointing to file corruption
  • While Applying a Weblogic Patch, opatch Fails with “Prerequisite check “CheckActiveFilesAndExecutables” failed” Error (Doc ID 2705809.1): Not a DB note and pointing to other processes using the files.
  • Opatch failure due to “CheckActiveFilesAndExecutables” as Remote registry service holding files (Doc ID 2462952.1): Remote registry holding the binaries.
  • Prerequisite Check “Checkactivefilesandexecutables” Failed (Doc ID 1281644.1): Patch requisite miss on 10g
  • Failed to apply PSU due to CheckActiveFilesAndExecutables check failure (Doc ID 2506432.1): SQLPlus holding the binaries.
  • [OCI]: Database System Patching Failed With Error “DCS-10001:Internal Error Encountered: Failure : Failed To Apply” And Opatch Log Shows “Prerequisite check “CheckActiveFilesAndExecutables” failed” (Doc ID 2687607.1): My case is not an OCI and not in RAC.

So, no matches at all.

However, this last note gave me the hints I needed. From Doc ID 2687607.1, for RAC environments:

/u01/app/19.0.0.0/grid/crs/install/rootcrs.sh -unlock
/u01/app/19.0.0.0/grid/crs/install/rootcrs.sh -init
/u01/app/19.0.0.0/grid/crs/install/rootcrs.sh -prepatch
/u01/app/19.0.0.0/grid/crs/install/rootcrs.sh -postpatch

So, in my case, a Standalone On-Premise Database (and GI):

/ora01/app/oracle/product/19c/grid/crs/install/roothas.sh -unlock
/ora01/app/oracle/product/19c/grid/crs/install/roothas.sh -init
/ora01/app/oracle/product/19c/grid/crs/install/roothas.sh -prepatch
[ Apply the patch! ]
/ora01/app/oracle/product/19c/grid/crs/install/roothas.sh -postpatch

Check the output:

[root@dbserver01 jan21cpu]# /u01/app/oracle/product/19c/grid/crs/install/roothas.sh -unlock
Using configuration parameter file: /u01/app/oracle/product/19c/grid/crs/install/crsconfig_params
The log of current session can be found at:
/u01/app/oracle/product/crsdata/dbserver01/crsconfig/haunlock__2021-03-14_04-00-35AM.log
2021/03/14 04:01:01 CLSRSC-347: Successfully unlock /u01/app/oracle/product/19c/grid
[root@dbserver01 jan21cpu]# /u01/app/oracle/product/19c/grid/crs/install/roothas.sh -init
Using configuration parameter file: /u01/app/oracle/product/19c/grid/crs/install/crsconfig_params
The log of current session can be found at:
/u01/app/oracle/product/crsdata/dbserver01/crsconfig/roothas_2021-03-14_04-01-09AM.log
[root@dbserver01 jan21cpu]# /u01/app/oracle/product/19c/grid/crs/install/roothas.sh -prepatch
Using configuration parameter file: /u01/app/oracle/product/19c/grid/crs/install/crsconfig_params
The log of current session can be found at:
/u01/app/oracle/product/crsdata/dbserver01/crsconfig/hapatch_2021-03-14_04-01-16AM.log
2021/03/14 04:01:27 CLSRSC-347: Successfully unlock /u01/app/oracle/product/19c/grid
2021/03/14 04:01:27 CLSRSC-671: Pre-patch steps for patching GI home successfully completed.

And now resuming the Opatchauto:

[root@dbserver01 jan21cpu]# cd 32126842/32226239/
[root@dbserver01 32226239]# $ORACLE_HOME/OPatch/opatchauto resume

OPatchauto session is initiated at Sun Mar 14 04:02:07 2021
Session log file is /u01/app/oracle/product/19c/grid/cfgtoollogs/opatchauto/opatchauto2021-03-14_04-02-10AM.log
Resuming existing session with id 1J89

Start applying binary patch on home /u01/app/oracle/product/19c/db
Binary patch applied successfully on home /u01/app/oracle/product/19c/db


Start applying binary patch on home /u01/app/oracle/product/19c/grid

Binary patch applied successfully on home /u01/app/oracle/product/19c/grid


Performing postpatch operations on CRS - starting CRS service on home /u01/app/oracle/product/19c/grid
Postpatch operation log file location: /u01/app/oracle/product/crsdata/dbserver01/crsconfig/hapatch_2021-03-14_04-27-58AM.log
CRS service started successfully on home /u01/app/oracle/product/19c/grid


Preparing home /u01/app/oracle/product/19c/db after database service restarted
No step execution required.........


Trying to apply SQL patch on home /u01/app/oracle/product/19c/db
SQL patch applied successfully on home /u01/app/oracle/product/19c/db

OPatchAuto successful.

--------------------------------Summary--------------------------------

Patching is completed successfully. Please find the summary as follows:

Host:dbserver01
SIDB Home:/u01/app/oracle/product/19c/db
Version:19.0.0.0.0
Summary:

==Following patches were SKIPPED:

Patch: /ora02/soft/jan21cpu/32126842/32226239/32218663
Reason: This patch is not applicable to this specified target type - "oracle_database"

Patch: /ora02/soft/jan21cpu/32126842/32226239/29340594
Reason: This patch is not applicable to this specified target type - "oracle_database"

Patch: /ora02/soft/jan21cpu/32126842/32226239/32240590
Reason: This patch is not applicable to this specified target type - "oracle_database"


==Following patches were SUCCESSFULLY applied:

Patch: /ora02/soft/jan21cpu/32126842/32226239/32218454
Log: /u01/app/oracle/product/19c/db/cfgtoollogs/opatchauto/core/opatch/opatch2021-03-14_04-02-36AM_1.log

Patch: /ora02/soft/jan21cpu/32126842/32226239/32222571
Log: /u01/app/oracle/product/19c/db/cfgtoollogs/opatchauto/core/opatch/opatch2021-03-14_04-02-36AM_1.log


Host:dbserver01
SIHA Home:/u01/app/oracle/product/19c/grid
Version:19.0.0.0.0
Summary:

==Following patches were SUCCESSFULLY applied:

Patch: /ora02/soft/jan21cpu/32126842/32226239/29340594
Log: /u01/app/oracle/product/19c/grid/cfgtoollogs/opatchauto/core/opatch/opatch2021-03-14_04-11-34AM_1.log

Patch: /ora02/soft/jan21cpu/32126842/32226239/32218454
Log: /u01/app/oracle/product/19c/grid/cfgtoollogs/opatchauto/core/opatch/opatch2021-03-14_04-11-34AM_1.log

Patch: /ora02/soft/jan21cpu/32126842/32226239/32218663
Log: /u01/app/oracle/product/19c/grid/cfgtoollogs/opatchauto/core/opatch/opatch2021-03-14_04-11-34AM_1.log

Patch: /ora02/soft/jan21cpu/32126842/32226239/32222571
Log: /u01/app/oracle/product/19c/grid/cfgtoollogs/opatchauto/core/opatch/opatch2021-03-14_04-11-34AM_1.log

Patch: /ora02/soft/jan21cpu/32126842/32226239/32240590
Log: /u01/app/oracle/product/19c/grid/cfgtoollogs/opatchauto/core/opatch/opatch2021-03-14_04-11-34AM_1.log

OPatchauto session completed at Sun Mar 14 04:31:48 2021
Time taken to complete the session 29 minutes, 43 seconds

And here is the relevant point: This has been happening to me on several environments and servers across the recent weeks. Always for 2021 January CPU.
My guess is that this might have something to do with this CPU binaries set or, most likely, with the latest OPatch version:

[oracle@dbserver01 ~]$ $ORACLE_HOME/OPatch/opatch version
OPatch Version: 12.2.0.1.24

I hope it helps you as well!

Cloud Services | Exadata | Linux, Database, Oracle Engineered Systems | Exadata

Reduce Exadata Core Count

 , , ,

Ok, so I was preparing for a DC services migration with a client and this would involve resizing the CPU count of Exadatas for better attending those services. This way, one of the steps will require reduce CPU counts in one of the sites to be aligned with the license terms.

Checking for the steps to accomplish that, I found references to change CPU and core count, but always described in the case of increasing allocation. As per 2.7 Increasing the Number of Active Cores on Database Servers. But not so much about reducing, as this seems to be unusual…

Also considering that the planned change would be within the minimum number requirement: 2.1 Restrictions for Capacity-On-Demand on Oracle Exadata Database Machine.

Reviewing on MOS, we found the When Attempting to Change the Number of Cores, Errors With: DBM-10004 – Decreasing the Number of Active Cores is not Supported ( Doc ID 2177634.1 ), pointing to use the clause “FORCE” on “ALTER DBSERVER pendingCoreCount =x” command.

And this worked. I just disabled the iaasMode to play safe. Have a look:

[root@grepora01~]# dbmcli
DBMCLI: Release  - Production on Mon Jan 05 01:10:12 EEST 2019
Copyright (c) 2007, 2014, Oracle.  All rights reserved.
DBMCLI> LIST DBSERVER attributes coreCount
	 36/44
DBMCLI> ALTER DBSERVER pendingCoreCount = 24 force
DBM-10022: At least 26 physical cores need to be active in order to support IaaS.
DBMCLI> ALTER DBSERVER iaasMode = "off"
DBServer exadb01 successfully altered
DBMCLI> ALTER DBSERVER pendingCoreCount = 24 force
DBServer grepora01 successfully altered. Please reboot the system to make the new pendingCoreCount effective.
DBMCLI> LIST DBSERVER attributes pendingCoreCount
24/44

–> Restart the server
After restarting, it should look like:

DBMCLI> LIST DBSERVER attributes coreCount
	 24/44
DBMCLI> LIST DBSERVER attributes pendingCoreCount

Hope this helps!

Linux

RHEL: Figuring out CPUs, Cores and Hyper-Threading

 , , , , ,

Hi all!
It’s a recurrent subject, right? But no one is 100% sure to how figure this out… So, let me quickly show you my way:

– Physical CPUs (sockets):

[root@mysrvr ~]# grep -i "physical id" /proc/cpuinfo | sort -u | wc -l
2
[root@mysrvr ~]# dmidecode -t processor |grep CPU
        Version: Intel(R) Xeon(R) CPU X5570 @ 2.93GHz
        Version: Intel(R) Xeon(R) CPU X5570 @ 2.93GHz

So, 2 physical CPUs.

– Physical Cores

[root@mysrvr ~]# egrep -e "core id" -e ^physical /proc/cpuinfo|xargs -l2 echo|sort -u
physical id : 0 core id : 0
physical id : 0 core id : 1
physical id : 0 core id : 2
physical id : 0 core id : 3
physical id : 1 core id : 0
physical id : 1 core id : 1
physical id : 1 core id : 2
physical id : 1 core id : 3

Each one of Physical Processors has 4 cores.
So, there is two quad-cores. This way, we have 8 cores at all.

– Logical CPUs

[root@mysrvr ~]# grep -i "processor" /proc/cpuinfo | sort -u | wc -l
16

Ok, so we have cores in double.
This means we have Hyper-Threading (technology by Intel Processors).

Not so hard, right?

Those links are similar and quite cool to understand the concepts:
https://access.redhat.com/discussions/480953
https://www.redhat.com/archives/redhat-list/2011-August/msg00009.html
http://www.intel.com/content/www/us/en/architecture-and-technology/hyper-threading/hyper-threading-technology.html

Matheus.

Database, Tools

PL/SQL Developer Taking 100% of Database CPU

 , , ,

When using PL/SQL Developer (Allround Automations), a internal query is taking a lot of cpu cycles on database server (100% of a CPU).
Is this your problem? Please check if the query is like this:

select s.synonym_name object_name, o.object_type
from sys.all_synonyms s,
sys.all_objects o
where s.owner in ('PUBLIC', user)
and o.owner = s.table_owner
and o.object_name = s.table_name
and o.object_type in ('TABLE', 'VIEW', 'PACKAGE','TYPE', 'PROCEDURE', 'FUNCTION', 'SEQUENCE')

It’s caused by the Describe Context Option of Code Assistant. To disable it:
Tools > Preferences > Code Assistant and disable the “Describe Context” option.

More“PL/SQL Developer Taking 100% of Database CPU”

Database, Migration | Upgrades | Patching

PSUs for Databases 10.2.0.4 and above

 , , , ,

Hi all!
I’m researching about and decided to share the list of all Oracle Patch Set Updates (PSU) for Databases 10.2.0.4 and above, until now.
It’s always good to know the last PSU for every version to better fit on our patching policy.

Note: To understand database version numbers: https://docs.oracle.com/cd/B28359_01/server.111/b28310/dba004.htm

I expect it be useful for you too!

Patches for 12.1.0.2
Patch 21359755 – DATABASE PATCH SET UPDATE 12.1.0.2.5
Patch 21523234 – GRID INFRASTRUCTURE PATCH SET UPDATE 12.1.0.2.5 (OCT2015)
Patch 20831110 – DATABASE PATCH SET UPDATE 12.1.0.2.4 (INCLUDES CPUJUL2015)
Patch 20996835 – GRID INFRASTRUCTURE PATCH SET UPDATE 12.1.0.2.4 (JUL2015)
Patch 20299023 – DATABASE PATCH SET UPDATE 12.1.0.2.3 (APR2015)
Patch 20485724 – GRID INFRASTRUCTURE PATCH SET UPDATE 12.1.0.2.3 (APR2015)
Patch 19769480 – DATABASE PATCH SET UPDATE 12.1.0.2.2 (JAN2015)
Patch 19954978 – GRID INFRASTRUCTURE PATCH SET UPDATE 12.1.0.2.2 (JAN2015)
Patch 19303936 – DATABASE PATCH SET UPDATE 12.1.0.2.1 (OCT2014)
Patch 19392646 – GRID INFRASTRUCTURE PATCH SET UPDATE 12.1.0.2.1 (OCT2014)

Patches for 12.1.0.1
Patch 21352619 – DATABASE PATCH SET UPDATE 12.1.0.1.9
Patch 21551666 – GRID INFRASTRUCTURE PSU 12.1.0.1.9 (OCT2015 – INCLUDES DB PSU 12.1.0.1.9)
Patch 20831107 – DATABASE PATCH SET UPDATE 12.1.0.1.8 (INCLUDES CPUJUL2015)
Patch 20996901 – GRID INFRASTRUCTURE PATCH SET UPDATE 12.1.0.1.8 (JUL2015)
Patch 20299016 – DATABASE PATCH SET UPDATE 12.1.0.1.7
Patch 20485762 – GRID INFRASTRUCTURE PATCH SET UPDATE 12.1.0.1.7 (APR2015)
Patch 19769486 – DATABASE PATCH SET UPDATE 12.1.0.1.6
Patch 19971324 – GRID INFRASTRUCTURE PATCH SET UPDATE 12.1.0.1.6 (JAN2015)
Patch 19121550 – DATABASE PATCH SET UPDATE 12.1.0.1.5
Patch 19392372 – GRID INFRASTRUCTURE PATCH SET UPDATE 12.1.0.1.5 (OCT2014)
Patch 18522516 – DATABASE PATCH SET UPDATE 12.1.0.1.4
Patch 18705901 – GRID INFRASTRUCTURE SYSTEM PATCH 12.1.0.1.4 (Exadata)
Patch 18705972 – GRID INFRASTRUCTURE SYSTEM PATCH 12.1.0.1.4
Patch 18031528 – DATABASE PATCH SET UPDATE 12.1.0.1.3
Patch 18139660 – GRID INFRASTRUCTURE SYSTEM PATCH 12.1.0.1.3 (Exadata)
Patch 18413105 – GRID INFRASTRUCTURE SYSTEM PATCH 12.1.0.1.3
Patch 17552800 – DATABASE PATCH SET UPDATE 12.1.0.1.2
Patch 17735306 – GRID INFRASTRUCTURE SYSTEM PATCH 12.1.0.1.2
Patch 17027533 – DATABASE PATCH SET UPDATE 12.1.0.1.1
Patch 17272829 – GRID INFRASTRUCTURE PSU 12.1.0.1.1

Patches for 11.2.0.4
Patch 21352635 – DATABASE PATCH SET UPDATE 11.2.0.4.8 (INCLUDES CPUOCT2015)
Patch 21523375 – GRID INFRASTRUCTURE PATCH SET UPDATE 11.2.0.4.8 (OCT2015)
Patch 20760982 – DATABASE PATCH SET UPDATE 11.2.0.4.7 (INCLUDES CPUJUL2015)
Patch 20996923 – GRID INFRASTRUCTURE PATCH SET UPDATE 11.2.0.4.7 (JUL2015)
Patch 20299013 – DATABASE PATCH SET UPDATE 11.2.0.4.6 (INCLUDES CPUAPR2015)
Patch 20485808 – GRID INFRASTRUCTURE PATCH SET UPDATE 11.2.0.4.6 (APR2015)
Patch 19769489 – DATABASE PATCH SET UPDATE 11.2.0.4.5 (INCLUDES CPUJAN2015)
Patch 19955028 – GRID INFRASTRUCTURE PATCH SET UPDATE 11.2.0.4.5 (JAN2015)
Patch 19121551 – DATABASE PATCH SET UPDATE 11.2.0.4.4 (INCLUDES CPUOCT2014)
Patch 19380115 – GRID INFRASTRUCTURE SYSTEM PATCH 11.2.0.4.4
Patch 18522509 – DATABASE PATCH SET UPDATE 11.2.0.4.3 (INCLUDES CPUJUL2014)
Patch 18706472 – GRID INFRASTRUCTURE SYSTEM PATCH 11.2.0.4.3
Patch 18031668 – DATABASE PATCH SET UPDATE 11.2.0.4.2 (INCLUDES CPUAPR2014)
Patch 18139609 – GRID INFRASTRUCTURE SYSTEM PATCH 11.2.0.4.2
Patch 17478514 – DATABASE PATCH SET UPDATE 11.2.0.4.1 (INCLUDES CPUJAN2014)

Patches for 11.2.0.3
Patch 20760997 – DATABASE PATCH SET UPDATE 11.2.0.3.15 (INCLUDES CPUJUL2015)
Patch 20996944 – GRID INFRASTRUCTURE PATCH SET UPDATE 11.2.0.3.15 (JUL2015)
Patch 20299017 – DATABASE PATCH SET UPDATE 11.2.0.3.14 (INCLUDES CPUAPR2015)
Patch 20485830 – GRID INFRASTRUCTURE PATCH SET UPDATE 11.2.0.3.14 (APR2015)
Patch 19769496 – DATABASE PATCH SET UPDATE 11.2.0.3.13 (INCLUDES CPUJAN2015)
Patch 19971343 – GRID INFRASTRUCTURE PSU 11.2.0.3.13 (JAN2015)
Patch 19121548 – DATABASE PATCH SET UPDATE 11.2.0.3.12 (INCLUDES CPUOCT2014)
Patch 19440385 – GRID INFRASTRUCTURE PSU 11.2.0.3.12 (INCLUDES DB PSU 11.2.0.3.12)
Patch 18522512 – DATABASE PATCH SET UPDATE 11.2.0.3.11 (INCLUDES CPUJUL2014)
Patch 18706488 – GRID INFRASTRUCTURE PSU 11.2.0.3.11 (INCLUDES DB PSU 11.2.0.3.11)
Patch 18031683 – DATABASE PATCH SET UPDATE 11.2.0.3.10
Patch 18139678 – GRID INFRASTRUCTURE PSU 11.2.0.3.10 (INCLUDES DB PSU 11.2.0.3.10)
Patch 17540582 – DATABASE PATCH SET UPDATE 11.2.0.3.9 (INCLUDES CPUJAN2014)
Patch 17735354 – GRID INFRASTRUCTURE PSU 11.2.0.3.9 (INCLUDES DB PSU 11.2.0.3.9)
Patch 16902043 – DATABASE PATCH SET UPDATE 11.2.0.3.8 (INCLUDES CPUOCT2013)
Patch 17272731 – GRID INFRASTRUCTURE PSU 11.2.0.3.8 (INCLUDES DB PSU 11.2.0.3.8)
Patch 16619892 – DATABASE PATCH SET UPDATE 11.2.0.3.7 (INCLUDES CPUJUL2013)
Patch 16742216 – GRID INFRASTRUCTURE PSU 11.2.0.3.7 (INCLUDES DB PSU 11.2.0.3.7)
Patch 16056266 – DATABASE PATCH SET UPDATE 11.2.0.3.6 (INCLUDES CPUAPR2013)
Patch 16083653 – GRID INFRASTRUCTURE PSU 11.2.0.3.6 (INCLUDES DB PSU 11.2.0.3.6)
Patch 14727310 – DATABASE PATCH SET UPDATE 11.2.0.3.5 (INCLUDES CPUJAN2013)
Patch 14727347 – GRID INFRASTRUCTURE PSU 11.2.0.3.5 (INCLUDES DB PSU 11.2.0.3.5)
Patch 14275605 – DATABASE PATCH SET UPDATE 11.2.0.3.4 (INCLUDES CPUOCT2012)
Patch 14275572 – GRID INFRASTRUCTURE PSU 11.2.0.3.4 (INCLUDES DB PSU 11.2.0.3.4)
Patch 13923374 – DATABASE PATCH SET UPDATE 11.2.0.3.3 (INCLUDES CPU JUL2012)
Patch 13919095 – GRID INFRASTRUCTURE PSU 11.2.0.3.3 (INCLUDES DB PSU 11.2.0.3.3)
Patch 13696216 – DATABASE PATCH SET UPDATE 11.2.0.3.2 (INCLUDES CPU APR2012)
Patch 13696251 – GRID INFRASTRUCTURE PSU 11.2.0.3.2 (INCLUDES DB PSU 11.2.0.3.2)
Patch 13343438 – DATABASE PATCH SET UPDATE 11.2.0.3.1 (INCLUDES CPU JAN2012)

Patches for 11.2.0.2
Patch 17082367 – DATABASE PATCH SET UPDATE 11.2.0.2.12 (INCLUDES CPUOCT2013)
Patch 17272753 – GRID INFRASTRUCTURE PSU 11.2.0.2.12 (INCLUDES DB PSU 11.2.0.2.12)
Patch 16619893 – DATABASE PATCH SET UPDATE 11.2.0.2.11 (INCLUDES CPUJUL2013)
Patch 16742320 – GRID INFRASTRUCTURE PSU 11.2.0.2.11 (INCLUDES DB PSU 11.2.0.2.11)
Patch 16056267 – DATABASE PATCH SET UPDATE 11.2.0.2.10 (INCLUDES CPUAPR2013)
Patch 16166868 – GRID INFRASTRUCTURE PSU 11.2.0.2.10 (INCLUDES DB PSU 11.2.0.2.10)
Patch 14727315 – DATABASE PATCH SET UPDATE 11.2.0.2.9 (INCLUDES CPUJAN2013)
Patch 14841385 – GRID INFRASTRUCTURE PSU 11.2.0.2.9 (INCLUDES DB PSU 11.2.0.2.9)
Patch 14275621 – DATABASE PATCH SET UPDATE 11.2.0.2.8 (INCLUDES CPUOCT2012)
Patch 14390437 – GRID INFRASTRUCTURE PSU 11.2.0.2.8 (INCLUDES DB PSU 11.2.0.2.8)
Patch 13923804 – DATABASE PATCH SET UPDATE 11.2.0.2.7 (INCLUDES CPU JUL2012)
Patch 14192201 – GRID INFRASTRUCTURE PSU 11.2.0.2.7 (INCLUDES DB PSU 11.2.0.2.7)
Patch 13696224 – DATABASE PATCH SET UPDATE 11.2.0.2.6 (INCLUDES CPU APR2012)
Patch 13696242 – GRID INFRASTRUCTURE PSU 11.2.0.2.6 (INCLUDES DB PSU 11.2.0.2.6)
Patch 13343424 – DATABASE PATCH SET UPDATE 11.2.0.2.5 (INCLUDES CPU JAN2012)
Patch 12827726 – DATABASE PSU 11.2.0.2.4 (INCLUDES CPUOCT2011)
Patch 12419331 – DATABASE PSU 11.2.0.2.3 (INCLUDES CPUJUL2011)
Patch 11724916 – DATABASE PSU 11.2.0.2.2 (INCLUDES CPUAPR2011)
Patch 10248523 – DATABASE PSU 11.2.0.2.1 (Patch)

Patches for 11.2.0.1
Patch 12419378 – DATABASE PSU 11.2.0.1.6 (INCLUDES CPUJUL2011)
Patch 11724930 – DATABASE PSU 11.2.0.1.5 (INCLUDES CPUAPR2011)
Patch 10248516 – DATABASE PSU 11.2.0.1.4 (INCLUDES CPUJAN2011)
Patch 9952216 – DATABASE PSU 11.2.0.1.3 (INCLUDES CPUOCT2010)
Patch 9655006 – GI PSU 11.2.0.1.2 (INCLUDES DATABASE PSU 11.2.0.1.2)
Patch 9654983 – DATABASE PSU 11.2.0.1.2 (INCLUDES CPUJUL2010)
Patch 9352237 – DATABASE PSU 11.2.0.1.1 (Patch)

Patches for 11.1.0.7
Patch 20761024 – DATABASE PATCH SET UPDATE 11.1.0.7.24 (INCLUDES CPUJUL2015)
Patch 20299012 – DATABASE PATCH SET UPDATE 11.1.0.7.23 (INCLUDES CPUAPR2015)
Patch 19769499 – DATABASE PATCH SET UPDATE 11.1.0.7.22 (INCLUDES CPUJAN2015)
Patch 19152553 – DATABASE PATCH SET UPDATE 11.1.0.7.21 (INCLUDES CPUOCT2014)
Patch 18522513 – DATABASE PATCH SET UPDATE 11.1.0.7.20 (INCLUDES CPUJUL2014)
Patch 18031726 – DATABASE PATCH SET UPDATE 11.1.0.7.19 (INCLUDES CPUAPR2014)
Patch 17465583 – DATABASE PATCH SET UPDATE 11.1.0.7.18 (INCLUDES CPUJAN2014)
Patch 17082366 – DATABASE PATCH SET UPDATE 11.1.0.7.17 (INCLUDES CPUOCT2013)
Patch 16619896 – DATABASE PATCH SET UPDATE 11.1.0.7.16 (INCLUDES CPUJUL2013)
Patch 16056268 – DATABASE PATCH SET UPDATE 11.1.0.7.15 (INCLUDES CPUAPR2013)
Patch 14739378 – DATABASE PATCH SET UPDATE 11.1.0.7.14 (INCLUDES CPUJAN2013)
Patch 14275623 – DATABASE PATCH SET UPDATE 11.1.0.7.13 (INCLUDES CPUOCT2012)
Patch 13923474 – DATABASE PATCH SET UPDATE 11.1.0.7.12 (INCLUDES CPU JUL2012)
Patch 13621679 – DATABASE PATCH SET UPDATE 11.1.0.7.11 (INCLUDES CPU APR2012)
Patch 13343461 – DATABASE PATCH SET UPDATE 11.1.0.7.10 (INCLUDES CPU JAN2012)
Patch 12827740 – DATABASE PSU 11.1.0.7.9 (INCLUDES CPUOCT2011)
Patch 12419384 – DATABASE PSU 11.1.0.7.8 (INCLUDES CPUJUL2011)
Patch 11724953 – TRACKING BUG FOR 11.1.0.7.7 CRS PSU
Patch 11724936 – DATABASE PSU 11.1.0.7.7 (INCLUDES CPUAPR2011)
Patch 10248531 – DATABASE PSU 11.1.0.7.6 (INCLUDES CPUJAN2011)
Patch 9952228 – DATABASE PSU 11.1.0.7.5 (INCLUDES CPUOCT2010)
Patch 9654987 – DATABASE PSU 11.1.0.7.4 (INCLUDES CPUJUL2010)
Patch 9352179 – DATABASE PSU 11.1.0.7.3 (INCLUDES CPUAPR2010)
Patch 9209238 – DATABASE PSU 11.1.0.7.2 (INCLUDES CPUJAN2010)
Patch 8833297 – DATABASE PSU 11.1.0.7.1 (INCLUDES CPUOCT2009)

Patches for 10.2.0.5
Patch 16619894 – DATABASE PATCH SET UPDATE 10.2.0.5.12 (INCLUDES CPUJUL2013)
Patch 16056270 – DATABASE PATCH SET UPDATE 10.2.0.5.11 (INCLUDES CPUAPR2013)
Patch 14727319 – DATABASE PATCH SET UPDATE 10.2.0.5.10 (INCLUDES CPUJAN2013)
Patch 14275629 – DATABASE PATCH SET UPDATE 10.2.0.5.9 (INCLUDES CPUOCT2012)
Patch 13923855 – DATABASE PATCH SET UPDATE 10.2.0.5.8 (INCLUDES CPU JUL2012)
Patch 13632743 – DATABASE PATCH SET UPDATE 10.2.0.5.7 (INCLUDES CPU APR2012)
Patch 13343471 – DATABASE PATCH SET UPDATE 10.2.0.5.6 (INCLUDES CPU JAN2012)
Patch 12827745 – DATABASE PSU 10.2.0.5.5 (INCLUDES CPUOCT2011)
Patch 12419392 – DATABASE PSU 10.2.0.5.4 (INCLUDES CPUJUL2011)
Patch 11724962 – DATABASE PSU 10.2.0.5.3 (INCLUDES CPUAPR2011)
Patch 9952245 – TRACKING BUG FOR 10.2.0.5.2 CRS PSU
Patch 10248542 – DATABASE PSU 10.2.0.5.2 (INCLUDES CPUJAN2011)
Patch 9952230 – DATABASE PSU 10.2.0.5.1 (INCLUDES CPUOCT2010)

Patches for 10.2.0.4
Patch 16619897 – DATABASE PSU 10.2.0.4.17 (REQUIRES PRE-REQUISITE 10.2.0.4.4|INCLUDES CPUJUL2013)
Patch 16056269 – DATABASE PSU 10.2.0.4.16 (REQUIRES PRE-REQUISITE 10.2.0.4.4|INCLUDES CPUAPR2013)
Patch 14736542 – DATABASE PSU 10.2.0.4.15 (REQUIRES PRE-REQUISITE 10.2.0.4.4|INCLUDES CPUJAN2013)
Patch 14275630 – DATABASE PSU 10.2.0.4.14 (REQUIRES PRE-REQUISITE 10.2.0.4.4|INCLUDES CPUOCT2012)
Patch 13923851 – DATABASE PSU 10.2.0.4.13 (REQUIRES PRE-REQUISITE 10.2.0.4.4|INCLUDES CPUJUL2012)
Patch 12879933 – DATABASE PSU 10.2.0.4.12 (REQUIRES PRE-REQUISITE 10.2.0.4.4|INCLUDES CPUAPR2012)
Patch 12879929 – DATABASE PATCH SET UPDATE 10.2.0.4.11 (PRE-REQ 10.2.0.4.4|INCLUDES CPUJAN2012)
Patch 12827778 – DATABASE PSU 10.2.0.4.10 (REQUIRES PRE-REQUISITE 10.2.0.4.4|INCLUDES CPUOCT2011)
Patch 12419397 – DATABASE PSU 10.2.0.4.9 (REQUIRES PRE-REQUISITE 10.2.0.4.4|INCLUDES CPUJUL2011)
Patch 11724977 – DATABASE PSU 10.2.0.4.8 (REQUIRES PRE-REQUISITE 10.2.0.4.4|INCLUDES CPUAPR2011)
Patch 10248636 – DATABASE PSU 10.2.0.4.7 (REQUIRES PRE-REQUISITE 10.2.0.4.4|INCLUDES CPUJAN2011)
Patch 9952234 – DATABASE PSU 10.2.0.4.6 (REQUIRES PRE-REQUISITE 10.2.0.4.4|INCLUDES CPUOCT2010)
Patch 9654991 – DATABASE PSU 10.2.0.4.5 (REQUIRES PRE-REQUISITE 10.2.0.4.4|INCLUDES CPUJUL2010)
Patch 9294403 – TRACKING BUG FOR THE 10.2.0.4.4 CRS PSU
Patch 9352164 – DATABASE PSU 10.2.0.4.4 (INCLUDES CPUAPR2010)
Patch 9119284 – DATABASE PSU 10.2.0.4.3 (INCLUDES CPUJAN2010)
Patch 8833280 – DATABASE PSU 10.2.0.4.2 (INCLUDES CPUOCT2009)
Patch 8576156 – DATABASE PSU 10.2.0.4.1 (INCLUDES CPUJUL2009)

Database, Errors and Bugs, Linux, RAC | Data Guard

VKTM Hang – High CPU Usage

 , , ,

Today a database (RHEL 6, single instance, 11.2.0.4) suddently started to “explode” CPU on VKTM process (100% CPU).
After some minutes lost (completely) in support.oracle.com (there was just a few notes about binary permissions on Solaris), I decided to make a McGayver by myself. 🙂

By Oracle words: “VKTM acts as a time publisher for an Oracle instance. VKTM publishes two sets of time: a wall clock time using a seconds interval and a higher resolution time (which is not wall clock time) for interval measurements. The VKTM timer service centralizes time tracking and offloads multiple timer calls from other clients.”

This way, my solution:

SQL> alter system set "_high_priority_processes"='LMS*' scope=spfile;
System altered.

And restart the database, of course.
So, VKTM is no more a “priority” process. The problem was solved. 🙂

Another possibility is to disable VKTM (undocumented parameter “_disable_vktm” – boolean). But I wanted to keep it running, changing less as possible of database configuration, just reducing priority.

KB:
Master Note: Troubleshooting Oracle Background Processes (Doc ID 1509616.1)
Great post about hidden parameters: http://oracleinaction.com/undocumented-params-11g/
Oficial one: http://www.orafaq.com/parms/index.htm

Hugs!
Matheus.

Database, Performance, RAC | Data Guard

High CPU usage by LMS and Node Evictions: Solved by Setting “_high_priority_processes”

 , , , , , , ,

Another thing that may help you in environments with highly interdependent applications:

Our env has high interconnect network block changing, and, as a consequence, high CPU usage by Global Cache Services (GCS)/Lock Manager Server Process (LMS).

This way, for each little latency in the interconnect interface, we were having a node eviction and all the impacts to the legacy application you can imagine (without gridlink or any solution to make the relocation ‘transparent’, as is usual to legacy application) and, of course, the business impact.

Oracle obviously suggested that we reduce the block concurrency over the cluster nodes grouping the application by affinity. But, it’s just no applicable to our env… 🙁

When nothing seemed to help, the workaround came from here: Top 5 Database and/or Instance Performance Issues in RAC Environment (Doc ID 1373500.1).

Here is our change:

boesing@proddb> alter system set "_high_priority_processes"='LMS*|LGWR|VKTM' scope=spfile sid='*';
System altered.

No magic, but the problem stopped to happen. After that, we’re having some warnings about clock synchronization over the cluster nodes on CRS alerts. Like this:

CRS-2409:The clock on host proddb1 is not synchronous with the mean cluster time. No action has been taken as the Cluster Time Synchronization. Service is running in observer mode.

I believe it happens because VKTM lost priority. But it’s OK: The node evictions has stopped! 😀

Matheus.