Kafka - Oracle

> Data Integration Tool (ETL/ELT) > Kafka (Event Hub)

1 - About

The connector for oracle

There is actually two given by Oracle:

  • the OGG (Oracle GoldenGate ) Kafka Handler (Supported),
  • or an open source Kafka Connect handler

And others:

3 - Installation

Advertising

4 - Implementation

It takes change data capture operations from a source trail file and generates:

The data structs are serialized via configured converters then enqueued onto Kafka topics.

4.1 - Topic Name

The topic name used corresponds to the fully qualified source table name as obtained from the GoldenGate trail file.

Individual operations consist of:

  • Insert and update operation (The corresponding struct includes the after change data)
  • and delete operations (Delete operations include the before change data.)

4.2 - Primary Key Update

A primary key update is a special case for an update where one or more of the primary key(s) is/are changed. The primary key update represents a special case in that without the before image data it is not possible to determine what row is actually changing when only in possession of the after change data.

The default behavior of a primary key update is to ABEND in the Kafka Connect formatter. However, the formatter can be configured to simply treat these operations as regular updates or to treat them as deletes and then an insert which is the closest big data modeling to the substance of the transaction.

5 - Documentation / Reference

Advertising