[source-mssql] Sync fails with CData Connect Server due to unfiltered indexes metadata query in memoizedClusteredIndexKeys
- Vorherrschende Sprache
- Python
- Sterne
- 22.1k
- Forks
- 5.3k
- PR-Merge-Kennzahlen
- PR-Kennzahlen ausstehend
Beschreibung
### Connector Name
source-mssql
### Connector Version
4.4.7
### What step the error happened?
During the sync
### Relevant information
The `source-mssql` connector (v4.0.0+) fails on every sync when connecting through CData Connect Server's TDS/virtual SQL Server endpoint. The connector issues an unfiltered query against the `indexes` system metadata table, which CData requires to be filtered by `TableName` in the `WHERE` clause. This makes the CData + Airbyte integration (documented on CData's official docs) completely non-functional on connector v4.x.
### Environment
- **Connector**: `source-mssql` v4.4.7
- **Airbyte platform**: Self-hosted
- **Source**: CData Connect Server (TDS/virtual MSSQL endpoint) fronting Odoo
- **Replication mode**: Full Refresh + Overwrite
- **CData docs for this integration**: https://docs.cloud.cdata.com/en/Clients/Airbyte-Client
### Steps to Reproduce
1. Set up a CData Connect Server connection to any data source (e.g. Odoo)
2. Configure `source-mssql` in Airbyte pointing to CData's TDS endpoint (`tds.cdata.com`, port `14333`) as documented by CData
3. Complete schema discovery successfully (connection test passes)
4. Trigger a sync on any stream in Full Refresh mode
5. Sync fails immediately before reading any data
### Error
```
java.lang.RuntimeException: SQL Server clustered index discovery query failed:
Insufficient filtering condition in WHERE clause for system table 'indexes'
at io.airbyte.integrations.source.mssql.MsSqlSourceMetadataQuerier$memoizedClusteredIndexKeys$2.invoke(MsSqlSourceMetadataQuerier.kt:316)
at io.airbyte.integrations.source.mssql.MsSqlSourceMetadataQuerier.getMemoizedClusteredIndexKeys(MsSqlSourceMetadataQuerier.kt:262)
at io.airbyte.integrations.source.mssql.MsSqlSourceMetadataQuerier.getOrderedColumnForSync(MsSqlSourceMetadataQuerier.kt:386)
at io.airbyte.integrations.source.mssql.MsSqlServerJdbcPartitionFactory.getOrderedColumnAsList(MsSqlServerJdbcPartitionFactory.kt:69)
at io.airbyte.integrations.source.mssql.MsSqlServerJdbcPartitionFactory.coldStart(MsSqlServerJdbcPartitionFactory.kt:102)
Caused by: com.microsoft.sqlserver.jdbc.SQLServerException:
Insufficient filtering condition in WHERE clause for system table 'indexes'
```
### Root Cause
In `MsSqlSourceMetadataQuerier.kt:271`, the connector executes an unfiltered query against the `indexes` metadata table to discover clustered index keys for read ordering (`memoizedClusteredIndexKeys`). This query has no `WHERE TableName = ?` filter.
CData's virtual SQL Server engine requires that `indexes` (and other system metadata tables) be queried with a mandatory table-scoped filter in the `WHERE` clause. Without it, CData throws the error above.
This call happens on **every sync, for every table, regardless of replication mode** — including Full Refresh, where clustered index discovery should not be necessary.
### Expected Behavior
The clustered index discovery query should either:
- Include a `WHERE` clause scoped to the specific table being synced (e.g. `WHERE TableName = 'res_users'`), or
- Be skipped entirely for Full Refresh mode where read ordering by clustered index is not required
### Additional Context
- CData explicitly documents Airbyte + MSSQL connector as a supported integration: https://docs.cloud.cdata.com/en/Clients/Airbyte-Client
- Connection test and schema discovery pass successfully — only sync fails
- Affects all tables, not specific ones
### Relevant log output
```shell
INFO MsSqlSourceMetadataQuerier$memoizedClusteredIndexKeys$2(invoke):266 Querying SQL Server system tables for all clustered index keys for catalog discovery.
WARN RootReader$FeedExceptionHandler(handleException):152 canceled feed 'Odoo.res_currency' due to thrown exception java.lang.RuntimeException: SQL Server clustered index discovery query failed: Insufficient filtering condition in WHERE clause for system table 'indexes'
at io.airbyte.integrations.source.mssql.MsSqlSourceMetadataQuerier$memoizedClusteredIndexKeys$2.invoke(MsSqlSourceMetadataQuerier.kt:316)
at io.airbyte.integrations.source.mssql.MsSqlSourceMetadataQuerier.getMemoizedClusteredIndexKeys(MsSqlSourceMetadataQuerier.kt:262)
at io.airbyte.integrations.source.mssql.MsSqlSourceMetadataQuerier.getOrderedColumnForSync(MsSqlSourceMetadataQuerier.kt:386)
at io.airbyte.integrations.source.mssql.MsSqlServerJdbcPartitionFactory.coldStart(MsSqlServerJdbcPartitionFactory.kt:102)
Caused by: com.microsoft.sqlserver.jdbc.SQLServerException: Insufficient filtering condition in WHERE clause for system table 'indexes'
at com.microsoft.sqlserver.jdbc.SQLServerException.makeFromDatabaseError(SQLServerException.java:356)
at io.airbyte.integrations.source.mssql.MsSqlSourceMetadataQuerier$memoizedClusteredIndexKeys$2.invoke(MsSqlSourceMetadataQuerier.kt:271)
ERROR Failed class io.airbyte.cdk.read.ReadOperation operation execution.
Source process exited with non-zero exit code 1
```
### Contribute
- [ ] Yes, I want to contribute
---
**Internal Tracking:** https://github.com/airbytehq/oncall/issues/12766
Beitragsleitfaden
Bewertung
Dieses Issue wurde noch nicht bewertet.