Managing database changes with Sqitch

Nowadays everybody talks about continuous integration, automated test tools, and stuff like that. But almost every time it’s about testing applications. Let’s talk a little about Sqitch, “a database change manager” as they use to call themselves.

Let’s say you’re starting a new project and want to ensure every database change are managed and secured with tests? Just like with an app, you can write tests to ensure everything works and you can have a “revert” method for the changes too. Suppose you are changing a field type and start creating a backup column (fallback ;)), but something goes wrong on the meantime and for some reason, you can’t do a simple rollback… revert it and be happy.

What if your CI says that the application it’s fine and can be released to the production server but “a wild bug appears” and you NEED to revert to a previous version of your service, easy huh? With Docker and one command line, the service is up and running again. But what about the database changes your team did? Just call the revert and be happy :).

Have I mentioned that you can have multiples connection URI at the project and have different environments to apply the changes with a single command?!. Let’s see a few commands to create a simple MySQL database…

Create a folder, initialize GIT in it and initialize the Sqitch project.

mkdir awesomeapp
cd awesomeapp
git init .
sqitch init flipr --uri https://github.com/theory/sqitch-mysql-intro/ --engine mysql

Now you have a ready to go sqitch structure. Now let’s do some basic creates.

sqitch add createcustomeruser -n 'Creation of Customer table'

Add the create table script to the file inside the “deploy” dir.

CREATE TABLE customer (
id INTEGER PRIMARY KEY,
name VARCHAR(50)
);

Add the verify script, can be a simple select. Add the revert script, guess what’s the command, and after all, you can run the commands below and see the result.

sqitch deploy db:mysql://root@/awesomeapp
sqitch verify db:mysql://root@/awesomeapp
sqitch revert db:mysql://root@/awesomeapp

This was a quick explanation about Sqitch so go ahead and read more at their docs and tutorials, they are pretty good.

12c New Feature: Unused Privileges

Did you know 12c now has a mechanism to examine which privileges are actually being used by an user, module or for the database as a whole, as opposed to merely the privileges granted?
This is a great security improvement in order give users the privileges that they need precisely without granting too much.

Here’s how is works?

1. First you have to set up a capture process using the new package DBMS_PRIVILEGE_CAPTURE.

# Example:

BEGIN 
DBMS_PRIVILEGE_CAPTURE.CREATE_CAPTURE( 
        name         => 'all_priv_analysis_pol', 
        description  => 'database-wide policy to analyze all privileges', 
        type         => DBMS_PRIVILEGE_CAPTURE.G_DATABASE); 
END;
/

This starts the capture process database wide. Let it run for a bit.

2. Now you can either generate a report:

# Example

BEGIN 
DBMS_PRIVILEGE_CAPTURE.GENERATE_RESULT( 
        name         => 'all_priv_analysis_pol'); 
END;
/

Or examine the new views:

DBA_USED_PRIVS 
DBA_UNUSED_PRIVS 
DBA_USED_OBJPRIVS 
DBA_UNUSED_OBJPRIVS

3. To turn off you proceed with the following:

BEGIN 
DBMS_PRIVILEGE_CAPTURE.DROP_CAPTURE( 
        name         => 'all_priv_analysis_pol'); 
END;
/

So now you ensure that you use the best practice of least privilege for users.

Cheers!

11g Feature: Fine Grained Dependency Tracking (FGDT)

Hi All!
I was reviewing some features in Oracle and, basically, every single time I review them I find something new. Seems Oracle Databases’ features are near to infinite and we frequently find some that can really add value to our solutions.

So I decided to make a serie of posts with really quick notes about each one of them.
You can see all posts in this serie in my page of posts and some others more.

Ready? Here it goes:

11g Feature: Fine Grained Dependency Tracking (FGDT)

In previous versions, object dependencies were managed at the object level. This way, altering an object automatically invalidated all dependent objects.
Oracle 11g has more granular dependency management, so only changes that directly affect an object will cause an invalidation.

This is not a Feature under control of Dev or DBA, but I judged important to mention here, as per has important impact to development and deployment processes.

More details can be found here: http://www.orafaq.com/node/2683

11g Feature: Fine-Grained Access Control (FGAC) on Network Services

Hi All!
I was reviewing some features in Oracle and, basically, every single time I review them I find something new. Seems Oracle Databases’ features are near to infinite and we frequently find some that can really add value to our solutions.

So I decided to make a serie of posts with really quick notes about each one of them.
You can see all posts in this serie in my page of posts and some others more.

Ready? Here it goes:

Fine-Grained Access Control (FGAC) on Network Services

Oracle supplies PL/SQL utility packages such as UTL_TCP, UTL_SMTP, UTL_MAIL, UTL_HTTP and UTL_INADDR to access to network services. In 11g Oracle have enhanced security available.
Rather than public being granted execute privileges on these packages, now it’s needed to create an ACCESS CONTROL LIST (ACL) in order to use these packages. Some ACL Related Data Dictionary VIEWS are DBA_NETWORK_ACLS and [DBA/USER]_NETWORK_ACL_PRIVILEGES.

> To create ACL:

SQL> begin
DBMS_NETWORK_ACL_ADMIN.create_acl(
acl => 'example.xml',
description=>'EXEMPLE ACL',
principal=>'EXAMPLE',                        
is_grant=>TRUE,
privilege=>'connect');
End;
/

> Once the ACL is created, additional user or privileges can be added using the DBMS_NETWORK_ACL_ADMIN.add_privileges procedure:

SQL> BEGIN
  DBMS_NETWORK_ACL_ADMIN.add_privilege ( 
    acl         =>  'example.xml', 
    principal   => 'SCOTT',
    is_grant    => FALSE, 
    privilege   => 'connect', 
    position    => NULL, 
    start_date  => NULL,
    end_date    => NULL);
  COMMIT;
END;
/

* DBMS_NETWORK_ACL_ADMIN.delete_privileges can be usedto drop privileges and DBMS_NETWORK_ACL_ADMIN.drop_acl to drop ACL.

> To assign ACL to a Network Host:

SQL> begin
DBMS_NETWORK_ACL_ADMIN.assign_acl(
acl => 'example.xml',
host=>'grepora');
End;
/

See you next week!

11g Feature: PLS-00436 Restriction in FORALL Statements Removed

Hi All!
I was reviewing some features in Oracle and, basically, every single time I review them I find something new. Seems Oracle Databases’ features are near to infinite and we frequently find some that can really add value to our solutions.

So I decided to make a serie of posts with really quick notes about each one of them.
You can see all posts in this serie in my page of posts and some others more.

Ready? Here it goes:

11g Feature: PLS-00436 Restriction in FORALL Statements Removed

In 11g, the PLS-00436 restriction has been removed, meaning individual elements of a collection can be referenced with SET and WHERE clauses in a FORALL construction.

Please check the following example, setting text of my_table to ‘Line x’ where x is the line number (also ID column):

DECLARE
  TYPE t_test IS TABLE OF my_table%ROWTYPE;
  l_test t_test;
BEGIN
  SELECT * BULK COLLECT INTO l_test FROM my_table;
  
  FOR i IN l_test.first .. l_test.last LOOP
    l_test(i).text := 'Line ' || i;
  END LOOP;

  FORALL i IN l_test.first .. l_test.last
    UPDATE my_table SET text = l_test(i).text WHERE id = l_test(i).id;
  COMMIT;
END;
/

How to Setup Automatic Startup and Shutdown of an Oracle Database on Linux (Not Using Oracle-Restart)

Ok, we all have done it several times. I, myself, made some scripts to do it in past. However, do you know there is an official way/script for that?
You can accomplish this with dbstart and dbshut scripts, which are located in the $ORACLE_HOME/bin directory.
This is documented for 12.1 in Stopping and Starting Oracle Software .

Of course that, if you have Oracle Clusterware configured, you can use Oracle Restart and SRVCTL tool, and Clusterware automatically starts and stops the Oracle database instances and listeners. Which is way better.
This post refers to official procedure in case you haven’t Clusterware configured.

Quick Guide:

Continue reading

Oracle Database Lock Performance / False Locks

Hello all!
This is a very interesing one… what about database taking too long or even showing locks that don’t really exists?
I faced some weird situations related to the size of DBMS_LOCK_ALLOCATED.

After some research, seems root cause is descripted in MOS DBMS_LOCK_ALLOCATED Table Keeps Growing Eventually Depleting Tablespace (Doc ID 1054343.6).

The issue is:
– DBMS_LOCK_ALLOCATED is the table that keep all locks on database.
– This table keeps growing even though many of the locks are probably no longer used.
– Upon checking the EXPIRED column, it reveals that the locks’ expiration date is a year or more in the future.

Besides the mentioned “storage space issue” in MOS Doc, there is also the situation that we have actually 1073741823 “lockhandlers” available. And what if this ‘not releasing’ cause we run out of handlers? Unlike, but possible.
Having a big table can also cause bad performance on DBMS_LOCK.REQUEST, that is the basic mechanism for locks (passing the handler, lockmode, timeout and release instructions).

Continue reading