kernel.panic_on_oops: New Oracle 12c Installation Requirement

Hi all,
Do you know what mean the parameters on installing 12c?

This parameter controls the kernel’s behaviour when an oops or bug is encountered:

  • 0: try to continue operation
  • 1: panic immediately.  If the `panic’ sysctl is also non-zero then the machine will be rebooted.

OOPS is a deviation from correct behavior of the Linux kernel, one that produces a certain error log.
The better-known kernel panic condition results from many kinds of oops, but other instances of an oops event may allow continued operation with compromised reliability.

This is recommended in a system where we want to have node evicted in case of any hardware failure or any other issue.

To adjust as recommended by Oracle?
1. Put an entry in sysctl.conf for having it permanent:

kernel.panic_on_oops = 1

2. Refresh running command:

sysctl -p

KB: https://www.kernel.org/doc/Documentation/sysctl/kernel.txt

Matheus.

Installing and Configuring ASMLIb on Oracle Linux 7

Hi all!
For those are familiar with RHEL/OEL 4 and 5, there is some differences to start ASMLib on OEL 6 and 7.

spanner.png
So, a quick guide to install (done on OEL 7), start and configure:

1. Install the ASMLib kernel module package as root using the following command:

yum install kmod-oracleasm

2. Install the ASMLib library package and utilities package

yum install oracleasm-support oracleasmlib oracleasm-`uname -r`

It’s possible some package to not found. For example:

No package oracleasmlib available.

So, you can download rpm libs from here and install via rpm:

[root@dbsrv01 oracle]# rpm -Uvh ~/oracleasmlib-2.0.12-1.el6.x86_64.rpm
Preparing...                          ################################# [100%]
Updating / installing...
1:oracleasmlib-2.0.12-1.el6        ################################# [100%]

Ok, now, lets configure/start services:

[root@dbsrv01 ~]# /etc/init.d/oracleasm configure

Nothing happen? Ok, let’s try to start it:

[root@dbsrv01 ~]# /etc/init.d/oracleasm start
Starting oracleasm (via systemctl):  Job for oracleasm.service failed because the control process exited with error code. See "systemctl status oracleasm.service" and "journalctl -xe" for details.
[FAILED]

Hmmm… Are these commands correct?

[root@dbsrv01 ~]# /etc/init.d/oracleasm
Usage: /etc/init.d/oracleasm {configure|createdisk|deletedisk|querydisk|listdisks|scandisks|status}

Ok… So, what to do?

Take a look:

[root@dbsrv01 ~]# oracleasm init
Creating /dev/oracleasm mount point: /dev/oracleasm
Loading module "oracleasm": oracleasm
Configuring "oracleasm" to use device physical block size
Mounting ASMlib driver filesystem: /dev/oracleasm

Victory!
Now, let’s configure:

[root@dbsrv01 ~]# oracleasm configure
ORACLEASM_UID=
ORACLEASM_GID=
ORACLEASM_SCANBOOT=true
ORACLEASM_SCANORDER=""
ORACLEASM_SCANEXCLUDE=""
ORACLEASM_USE_LOGICAL_BLOCK_SIZE="false"

It shows, but how configure?

Just put “-i” clause, like:

[root@dbsrv01 ~]# oracleasm configure -i
Configuring the Oracle ASM library driver.
This will configure the on-boot properties of the Oracle ASM library
driver.  The following questions will determine whether the driver is
loaded on boot and what permissions it will have.  The current values
will be shown in brackets ('[]').  Hitting  without typing an
answer will keep that current value.  Ctrl-C will abort.
Default user to own the driver interface []: grid
Default group to own the driver interface []: oinstall
Scan for Oracle ASM disks on boot (y/n) [y]: y
Writing Oracle ASM library driver configuration: done

And you can list again:

[root@dbsrv01 ~]# oracleasm configure
ORACLEASM_UID=grid
ORACLEASM_GID=oinstall
ORACLEASM_SCANBOOT=true
ORACLEASM_SCANORDER=""
ORACLEASM_SCANEXCLUDE=""
ORACLEASM_USE_LOGICAL_BLOCK_SIZE="false"
[root@dbsrv01 ~]# oracleasm status
Checking if ASM is loaded: yes
Checking if /dev/oracleasm is mounted: yes

To add a disk, the same process can be followed on earlier versions:

[root@dbsrv01 ~]# oracleasm createdisk SDD /dev/sdd1
Writing disk header: done
Instantiating disk: done
[root@dbsrv01 ~]# oracleasm listdisks
SDD

For all commands:

[root@dbsrv01 ~]# oracleasm -h
Usage: oracleasm [--exec-path=]  [  ]
oracleasm --exec-path
oracleasm -h
oracleasm -V
The basic oracleasm commands are:
configure        Configure the Oracle Linux ASMLib driver
init             Load and initialize the ASMLib driver
exit             Stop the ASMLib driver
scandisks        Scan the system for Oracle ASMLib disks
status           Display the status of the Oracle ASMLib driver
listdisks        List known Oracle ASMLib disks
querydisk        Determine if a disk belongs to Oracle ASMlib
createdisk       Allocate a device for Oracle ASMLib use
deletedisk       Return a device to the operating system
renamedisk       Change the label of an Oracle ASMlib disk
update-driver    Download the latest ASMLib driver

And to see arguments for each one:

[root@dbsrv01 ~]# oracleasm configure -h
Usage: oracleasm-configure [-l ] [-i|-I] [-e|-d] [-u ] [-g ] [-b|-p] [-s y|n] [[-o ] ...] [[-x ] ...]

Have a nice day!
See ya!
Matheus.

Kludge: Mounting ACFS Thought Shellscript

Just the script. The history is here.
This is a “workaround” script. As always, is recommended to use Oracle Restart, like I posted here.

#!/bin/sh
$GRID_HOME/bin/srvctl add filesystem -d /dev/asm/dbhome-270 -g 'DGHOME' -v DBHOME -m /oracle/db -u oracle
if [ $? = "0" -o $? = "2" ]; then
$GRID_HOME/bin/srvctl start filesystem -d /dev/asm/dbhome-270
if [ $? = "0" ]; then
chown oracle:oinstall /oracle/db
chmod 775 /oracle/db
$GRID_HOME/bin/srvctl status filesystem -d /dev/asm/dbhome-270
exit 0
fi
$GRID_HOME/bin/srvctl status filesystem -d /dev/asm/dbhome-270
fi

There is a good post ACFS and ACFS restart scripting:
https://levipereira.wordpress.com/2011/07/28/oracle-acfs-filesystem-managed-by-ohas-on-oracle-restart/

See ya!

Matheus.

ORA-15186: ASMLIB error function

Almost a month away… My bad!
Here I go again, with a quick tip, that a passed today. Our kernel was ‘changed’ without advise and this began to happen:

ORA-15186: ASMLIB error function = [asm_init], error = [18446744073709551611], mesg = [Driver not installed]
ERROR: error ORA-15186 caught in ASM I/O path

The solution was is basically update the asmlibs, that is based on kernel version. For RHEL, the solution is well decribed here:

http://www.oracle.com/technetwork/server-storage/linux/asmlib/rhel6-1940776.html
https://access.redhat.com/solutions/315643

Just to remember: After the kernel change, a relink of your Oracle Home is higly recommended.

Have a nice day!
Matheus.

Create SPFILE on ASM from PFILE on Filesystem

Some basics, right?
Another thing that is not usual and everytime I do, someone be surprised: “shu” alias for “shutdown”:

SQL> create spfile='+DGDATA/MYDB/spfilemydb.ora' from pfile='/oracle/product/11.2/dbs/init_mydb.ora';
File created.
SQL> shu immediate;
Database closed.
Database dismounted.
ORACLE instance shut down.

The Bourleson Master also wrote about it. Take a look on a better detailed post about this subject: http://www.dba-oracle.com/concepts/pfile_spfile.htm.

Matheus.

ASM: Disk Imbalance Query

It can be useful if you work frequently with OEM metrics…

# OEM Query

SELECT file_num, MAX(extent_count) max_disk_extents, MIN(extent_count)
min_disk_extents
, MAX(extent_count) - MIN(extent_count) disk_extents_imbalance
FROM (SELECT number_kffxp file_num, disk_kffxp disk_num, COUNT(xnum_kffxp)
extent_count
FROM x$kffxp
WHERE group_kffxp = 1
AND disk_kffxp != 65534
GROUP BY number_kffxp, disk_kffxp
ORDER BY number_kffxp, disk_kffxp)
GROUP BY file_num
HAVING MAX(extent_count) - MIN(extent_count) > 5
ORDER BY disk_extents_imbalance DESC;

# MatheusDBA Query

select max(free_mb) biggest, min(free_mb) lowest, avg(free_mb) AVG,
trunc(GREATEST ((avg(free_mb)*100/max(free_mb)),(min(free_mb)*100/avg(free_mb))),2)||'%' as balanced,
trunc(100-(GREATEST ((avg(free_mb)*100/max(free_mb)),(min(free_mb)*100/avg(free_mb)))),2)||'%' as inbalanced
from v$asm_disk
where group_number in
(select group_number from v$asm_diskgroup where name = upper('&DG'));

I made my own query for two reasons:
1) I didn’t have the OEM query in the time i made it.
2) My query measures the inbalance with the avg of the disks (if everydisk would balanced, how would be the difference), rather than the real/present difference between the disk with the maximum and the minimum usage…

So, you can chose the one you need… 🙂

Matheus.

GoldenGate: RAC One Node Archivelog Missing

The situation:

We have a GoldenGate on Allow Mode running some extracts on RAC One Node Database (reading the archivelogs). And then, suddenly, the instance crashes (network lost contact to the server) and the other instance (thread) was auto started by CRS. To the database no problems: The other node redologs was used during the startup recover and every thing is ok.

The application running with Weblogic serverpool and gridlink just had a little contention and continued the operation thought the started instance. The Goldengate switch was manually made, but some sequences was lost. What we found? the sequences was in the old thread’s redologfiles. It should be backed up if fast_start_mttr_target was different to zero. Buuut, the world is not so beautiful:

raconenodedb> show parameters mttr
NAME TYPE VALUE
------------------------------------
fast_start_mttr_target integer 0

How we solved?
Simple solution: identified the group/thread and made a cp from ASM. The copied redolog was used as archivelog on goldengate and everything was ok.

raconenodedb> select sequence#,group#,thread# from v$log where thread#=2 order by 1;
SEQUENCE# GROUP# THREAD#
---------- ---------- ----------
39636 6 2
39637 7 2
39638 8 2
39639 9 2
39640 10 2
ASMCMD> cp group_10.288.859482805 /oracle/grup10_thread2
copying +DGDATA/MYDB/ONLINELOG/group_10.288.859482805 -> /oracle/grup10_thread2

Easy like that.

Matheus.