airbytehq / airbytehq/airbyte

[source-mssql] Sync fails with CData Connect Server due to unfiltered indexes metadata query in memoizedClusteredIndexKeys

Offen
#78,529 3 Kommentare 0 Reaktionen 0 zugewiesene Personen Auf GitHub ansehen
area/connectors autoteam community connectors/source/mssql hyd-review needs-triage team/extensibility team/use type/bug
Vorherrschende Sprache
Python
Sterne
22.1k
Forks
5.3k
PR-Merge-Kennzahlen
PR-Kennzahlen ausstehend

Beschreibung

### Connector Name

source-mssql

### Connector Version

4.4.7

### What step the error happened?

During the sync

### Relevant information

The `source-mssql` connector (v4.0.0+) fails on every sync when connecting through CData Connect Server's TDS/virtual SQL Server endpoint. The connector issues an unfiltered query against the `indexes` system metadata table, which CData requires to be filtered by `TableName` in the `WHERE` clause. This makes the CData + Airbyte integration (documented on CData's official docs) completely non-functional on connector v4.x.

### Environment

- **Connector**: `source-mssql` v4.4.7
- **Airbyte platform**: Self-hosted
- **Source**: CData Connect Server (TDS/virtual MSSQL endpoint) fronting Odoo
- **Replication mode**: Full Refresh + Overwrite
- **CData docs for this integration**: https://docs.cloud.cdata.com/en/Clients/Airbyte-Client

### Steps to Reproduce

1. Set up a CData Connect Server connection to any data source (e.g. Odoo)
2. Configure `source-mssql` in Airbyte pointing to CData's TDS endpoint (`tds.cdata.com`, port `14333`) as documented by CData
3. Complete schema discovery successfully (connection test passes)
4. Trigger a sync on any stream in Full Refresh mode
5. Sync fails immediately before reading any data

### Error

```
java.lang.RuntimeException: SQL Server clustered index discovery query failed:
Insufficient filtering condition in WHERE clause for system table 'indexes'
at io.airbyte.integrations.source.mssql.MsSqlSourceMetadataQuerier$memoizedClusteredIndexKeys$2.invoke(MsSqlSourceMetadataQuerier.kt:316)
at io.airbyte.integrations.source.mssql.MsSqlSourceMetadataQuerier.getMemoizedClusteredIndexKeys(MsSqlSourceMetadataQuerier.kt:262)
at io.airbyte.integrations.source.mssql.MsSqlSourceMetadataQuerier.getOrderedColumnForSync(MsSqlSourceMetadataQuerier.kt:386)
at io.airbyte.integrations.source.mssql.MsSqlServerJdbcPartitionFactory.getOrderedColumnAsList(MsSqlServerJdbcPartitionFactory.kt:69)
at io.airbyte.integrations.source.mssql.MsSqlServerJdbcPartitionFactory.coldStart(MsSqlServerJdbcPartitionFactory.kt:102)

Caused by: com.microsoft.sqlserver.jdbc.SQLServerException:
Insufficient filtering condition in WHERE clause for system table 'indexes'
```

### Root Cause

In `MsSqlSourceMetadataQuerier.kt:271`, the connector executes an unfiltered query against the `indexes` metadata table to discover clustered index keys for read ordering (`memoizedClusteredIndexKeys`). This query has no `WHERE TableName = ?` filter.

CData's virtual SQL Server engine requires that `indexes` (and other system metadata tables) be queried with a mandatory table-scoped filter in the `WHERE` clause. Without it, CData throws the error above.

This call happens on **every sync, for every table, regardless of replication mode** — including Full Refresh, where clustered index discovery should not be necessary.

### Expected Behavior

The clustered index discovery query should either:
- Include a `WHERE` clause scoped to the specific table being synced (e.g. `WHERE TableName = 'res_users'`), or
- Be skipped entirely for Full Refresh mode where read ordering by clustered index is not required

### Additional Context

- CData explicitly documents Airbyte + MSSQL connector as a supported integration: https://docs.cloud.cdata.com/en/Clients/Airbyte-Client
- Connection test and schema discovery pass successfully — only sync fails
- Affects all tables, not specific ones

### Relevant log output

```shell
INFO MsSqlSourceMetadataQuerier$memoizedClusteredIndexKeys$2(invoke):266 Querying SQL Server system tables for all clustered index keys for catalog discovery.

WARN RootReader$FeedExceptionHandler(handleException):152 canceled feed 'Odoo.res_currency' due to thrown exception java.lang.RuntimeException: SQL Server clustered index discovery query failed: Insufficient filtering condition in WHERE clause for system table 'indexes'
at io.airbyte.integrations.source.mssql.MsSqlSourceMetadataQuerier$memoizedClusteredIndexKeys$2.invoke(MsSqlSourceMetadataQuerier.kt:316)
at io.airbyte.integrations.source.mssql.MsSqlSourceMetadataQuerier.getMemoizedClusteredIndexKeys(MsSqlSourceMetadataQuerier.kt:262)
at io.airbyte.integrations.source.mssql.MsSqlSourceMetadataQuerier.getOrderedColumnForSync(MsSqlSourceMetadataQuerier.kt:386)
at io.airbyte.integrations.source.mssql.MsSqlServerJdbcPartitionFactory.coldStart(MsSqlServerJdbcPartitionFactory.kt:102)

Caused by: com.microsoft.sqlserver.jdbc.SQLServerException: Insufficient filtering condition in WHERE clause for system table 'indexes'
at com.microsoft.sqlserver.jdbc.SQLServerException.makeFromDatabaseError(SQLServerException.java:356)
at io.airbyte.integrations.source.mssql.MsSqlSourceMetadataQuerier$memoizedClusteredIndexKeys$2.invoke(MsSqlSourceMetadataQuerier.kt:271)

ERROR Failed class io.airbyte.cdk.read.ReadOperation operation execution.
Source process exited with non-zero exit code 1
```

### Contribute

- [ ] Yes, I want to contribute

---
**Internal Tracking:** https://github.com/airbytehq/oncall/issues/12766

Beitragsleitfaden

Beitragsleitfaden öffnen

Bewertung

Dieses Issue wurde noch nicht bewertet.

Neue Issues direkt in Ihr Postfach

Eine kurze Übersicht über anfängerfreundliche GitHub-Issues.