debezium / debezium/dbz

IBMi Connector: Support for Multi-member Physical Files and Journal Resolution via SQL Alias

Open
#1,924 1 comment 0 reactions 0 assignees View on GitHub
component/ibmi-connector type/enhancement
Dominant language
HTML
Stars
6
Forks
8
Avg merge
2d 19h
Merged PRs (30d)
1

Description

Hi Debezium team!

While investigating Debezium capabilities for legacy IBM i environments, we've identified a behavior regarding DDS-defined Physical Files with multiple members that prevents CDC. We are almost certain that this is not a direct limitation of the Debezium connector itself, but rather a long-standing historical constraint originating from the underlying JDBC JT400 driver's design and its interaction with the IBM i database engine.

**Current Behavior:**

**Member Access:** The connector defaults to the first member (*FIRST). There is currently no documented way to iterate or specify additional members in the configuration.

**Naming Convention**: Attempting to use standard IBM i member syntax (e.g., LIBRARY/FILE(MEMBER)) in the table.include.list results in a lookup failure. We already tried changing the naming convention to system and other related parameters without success.

**SQL Alias Workaround**: As a workaround, we created a SQL ALIAS pointing to a specific member name. The streaming phase (CDC) fails immediately when the connector attempts to locate the journal associated with the object.

**Error Log observed:**

java.lang.IllegalStateException: unable to retrieve journal details
Caused by: java.lang.IllegalStateException: Journal not found for SCHEMA.ALIAS_NAME error File ALIAS_NAME not allowed with SYSTEM(*LCL).

**Technical Observation:**
In our humble opinion, the error File ALIAS_NAME not allowed with SYSTEM(*LCL) suggests that the connector's internal mechanism for retrieving journal entries is incompatible with SQL ALIAS objects. Since an Alias seems to be a DDM file pointing to a local member, the underlying system API used by the connector appears to reject it, expecting a Physical (*FILE PF) or Logical (*FILE LF) file instead.

**Questions & Request**:
Is there a recommended workaround to capture changes (CDC) from specific members of a multi-member Physical File?

If it is not possible to support member declaration via regex/syntax in the configuration, would you please enhance the journal resolution logic to support SQL ALIAS tables

**Environment**:
Debezium version: 3.5.0.Final
Driver: jt400.jar (JTOpen) version 21.0.6
Platform: IBM i V7R5

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.