Kafka-Compatible APIs: Oracle 23ai Meets the Streaming World

Oracle 23ai introduces Kafka-compatible APIs for Transactional Event Queues (TxEventQ) — Oracle’s messaging and event streaming infrastructure. This means your existing Kafka producer and consumer code can talk to Oracle Database without any changes.

What are Transactional Event Queues?

TxEventQ is Oracle’s built-in messaging system — stored inside the Oracle Database, with full ACID guarantees. Unlike standalone Kafka, messages in TxEventQ participate in database transactions. If a transaction rolls back, the message is also rolled back.

The Kafka bridge:

Oracle 23ai exposes TxEventQ over the Kafka protocol. Kafka clients (using the standard Kafka Java client, Confluent libraries, or any Kafka-compatible client) connect to Oracle’s Kafka endpoint and produce/consume messages transparently.

# Standard Kafka client configuration
bootstrap.servers=oracle-db-host:9092
security.protocol=SASL_SSL
sasl.mechanism=PLAIN
# (Oracle credentials used for SASL auth)

Creating a Kafka-compatible topic (TxEventQ):

BEGIN
    DBMS_AQADM.CREATE_TRANSACTIONAL_EVENT_QUEUE(
        queue_name    => 'order_events',
        queue_payload_type => DBMS_AQADM.JMS_TYPE
    );
    DBMS_AQADM.START_QUEUE('order_events');
END;

Why this matters:

  • Transactional safety: A Kafka consumer acknowledging a message and updating a database row are one atomic operation — no dual-write problem.
  • Simplified architecture: Replace your Kafka cluster + database for use cases that require exactly-once semantics between messaging and data storage.
  • Migration path: Gradually migrate Kafka-based microservices to Oracle without rewriting clients.

This bridges two worlds that have traditionally required complex integration middleware. For Oracle-centric architectures, it’s a compelling alternative to external Kafka clusters.

Priority Transactions in Oracle 23ai

Oracle 23ai introduces Priority Transactions — a mechanism to assign priorities to DML transactions so that higher-priority work can preempt lower-priority work when lock contention occurs.

In a busy OLTP system, a long-running batch job holding a lock can block critical customer-facing transactions. Until now, the only options were to kill the batch session manually or accept the latency. Priority Transactions automate this.

Setting a transaction priority:

-- High priority (e.g., customer checkout)
EXEC DBMS_TRANSACTION.SET_PRIORITY('HIGH');

-- Low priority (e.g., background batch)
EXEC DBMS_TRANSACTION.SET_PRIORITY('LOW');

How preemption works:

When a HIGH priority transaction attempts to acquire a lock held by a LOW priority transaction, Oracle evaluates the conflict. If the high-priority transaction has waited longer than a configurable threshold, Oracle can:

  1. Abort the low-priority transaction (rollback and release locks)
  2. Notify the low-priority session so it can self-abort cleanly
  3. Defer the decision based on age and priority differential

Configuration:

ALTER SYSTEM SET TRANSACTION_PRIORITY_POLICY = 'PREEMPTIVE';
ALTER SYSTEM SET PRIORITY_TRANSACTION_WAIT_TIMEOUT = 5; -- seconds

Monitoring:

SELECT sid, serial#, priority, wait_class, seconds_in_wait
FROM   v$session
WHERE  wait_class = 'Concurrency'
ORDER BY priority DESC, seconds_in_wait DESC;

Real-world impact: In e-commerce and financial systems where a payment transaction must complete quickly, Priority Transactions ensure that a poorly-timed batch report doesn’t cause a customer-facing timeout. This is the kind of feature that previously required complex application-layer retry logic.

Lock-Free Reservations: High-Concurrency DML Without Conflicts

One of Oracle 23ai’s most interesting concurrency features is Lock-Free Reservations. It solves a classic high-concurrency problem: multiple sessions competing to update the same row — like decrementing inventory counts in a flash sale.

The classic problem:

-- Session 1 and Session 2 both try to decrement inventory
UPDATE products SET stock_qty = stock_qty - 1 WHERE product_id = 42;

With thousands of concurrent orders, Sessions 2 through N all block waiting for Session 1’s lock. This creates a hot row contention bottleneck.

Lock-Free Reservation solution:

-- Define the column as a reservation target
ALTER TABLE products MODIFY (
    stock_qty NUMBER RESERVABLE CONSTRAINT stock_non_negative CHECK (stock_qty >= 0)
);

Once a column is declared RESERVABLE, Oracle handles concurrent updates to it using a reservation mechanism rather than row-level locking. Each session “reserves” a portion of the value without blocking others.

How it works under the hood:

Oracle maintains a pending reservation table alongside the main table. Each concurrent update is stored as a reservation. The actual column value is the base value plus the sum of all pending reservations. When a session commits, its reservation is rolled into the base value.

The net effect:

  • Multiple sessions can concrement stock_qty simultaneously without blocking each other
  • The CHECK constraint (stock_qty >= 0) is still enforced — the sum of all reservations cannot drive the value below zero
  • ACID guarantees are maintained

Ideal use cases: Inventory management, ticket booking, seat reservations, resource quotas — any scenario where many sessions update the same numeric column concurrently.

This is a sophisticated concurrency improvement that eliminates a well-known Oracle scalability bottleneck without requiring application-level workarounds.

SQL Firewall: Protecting Your Database with SQL Whitelisting

Oracle SQL Firewall is one of the most important security additions in Oracle 23ai. It’s a kernel-level feature (not an add-on product) that inspects every SQL statement sent to the database and compares it against a learned whitelist of approved statements.

How it works:

  1. Learning phase: SQL Firewall observes your application’s normal SQL traffic and builds a whitelist of allowed statements, users, and connection paths.
  2. Enforcement phase: Any SQL statement not in the whitelist is either blocked, logged, or both.

Setting up SQL Firewall:

-- Enable SQL Firewall
EXEC DBMS_SQL_FIREWALL.ENABLE;

-- Start learning for a specific user
EXEC DBMS_SQL_FIREWALL.START_CAPTURE('APP_USER');

-- (Let the application run normally for the learning period)

-- Stop learning and create the allow-list
EXEC DBMS_SQL_FIREWALL.STOP_CAPTURE('APP_USER');
EXEC DBMS_SQL_FIREWALL.CREATE_ALLOW_LIST('APP_USER');

-- Enable enforcement
EXEC DBMS_SQL_FIREWALL.ENABLE_ALLOW_LIST(
    username     => 'APP_USER',
    enforce      => DBMS_SQL_FIREWALL.ENFORCE_SQL,
    block        => TRUE
);

What it catches:

  • SQL injection attempts that generate SQL not in the whitelist
  • Unauthorized queries run by rogue application code or DBA accounts
  • Unexpected data exfiltration queries (SELECT * FROM users when the app only does parameterized lookups)
  • Suspicious connection paths (e.g., a user who normally connects from the app server suddenly connecting from a different IP)

Monitoring violations:

SELECT username, sql_text, action_taken, timestamp
FROM   dba_sql_firewall_violations
ORDER BY timestamp DESC;

SQL Firewall is especially valuable for organizations subject to PCI-DSS, HIPAA, or SOX compliance. It provides a last line of defense at the database level — even if the application layer is compromised.