PostgreSQL Destination v3.0.4 - Invalid timestamp value "{}" causes COPY failures (self-hosted)
- Ngôn ngữ chính
- Python
- Star
- 22.1k
- Fork
- 5.4k
- Merge trung bình
- 5 giờ
- Pull request đã merge (30 ngày)
- 671
Mô tả
## Bug Description
The PostgreSQL destination connector version 3.0.4 is passing an invalid string `"{}"` instead of a proper timestamp value for the `_airbyte_extracted_at` column, causing all connection tests and syncs to fail with a PostgreSQL type conversion error.
## Environment
- **Airbyte Version**: Self-hosted via abctl on Azure VM (Ubuntu)
- **Connector**: Postgres Destination
- **Connector Version**: 3.0.4
- **Deployment**: Self-hosted (abctl)
- **Target Database**: Azure PostgreSQL Flexible Server v16
## Steps to Reproduce
1. Configure a PostgreSQL destination with any valid connection details
2. Test the connection OR run any sync
3. Observe the error during COPY operation
## Expected Behavior
The connector should insert a valid `timestamp with time zone` value for `_airbyte_extracted_at` column.
## Actual Behavior
The connector attempts to insert the literal string `"{}"` as a timestamp value, causing PostgreSQL to reject the COPY operation with:
```
ERROR: invalid input syntax for type timestamp with time zone: "{}"
Where: COPY _airbyte_connection_test_, line 1, column _airbyte_extracted_at: "{}"
```
## Error Logs
```
ERROR main i.a.i.d.p.w.l.PostgresInsertBuffer(flush):66 Unable to flush accumulated data.
org.postgresql.util.PSQLException: ERROR: invalid input syntax for type timestamp with time zone: "{}"
Where: COPY _airbyte_connection_test_84699f86a1fc4af2bfb3ff9106acab84, line 1, column _airbyte_extracted_at: "{}"
at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2733)
at org.postgresql.core.v3.QueryExecutorImpl.processCopyResults(QueryExecutorImpl.java:1311)
at org.postgresql.core.v3.QueryExecutorImpl.endCopy(QueryExecutorImpl.java:1107)
at org.postgresql.core.v3.CopyInImpl.endCopy(CopyInImpl.java:53)
at org.postgresql.copy.CopyManager.copyIn(CopyManager.java:227)
at org.postgresql.copy.CopyManager.copyIn(CopyManager.java:203)
at io.airbyte.integrations.destination.postgres.client.PostgresAirbyteClient.copyFromCsv(PostgresAirbyteClient.kt:367)
at io.airbyte.integrations.destination.postgres.write.load.PostgresInsertBuffer.flush(PostgresInsertBuffer.kt:61)
```
Stack trace shows `bulk-cdk-core-load-0.1.83.jar` is being used.
## Impact
- **Severity**: Critical - PostgreSQL destinations are completely non-functional on v3.0.4
- **Scope**: Affects both connection tests and actual syncs
- **Workaround Attempted**:
- Enabled "Disable Final Tables" option - same error persists
- Attempted to downgrade via UI - operation times out
- Attempted Docker image manipulation - connector still pulls v3.0.4
## Additional Context
- Using self-hosted Airbyte deployed via abctl on Azure VM
- Connection works fine from command line (psql, Python), so it's not a network/firewall issue
- The bug appears in the `PostgresInsertBuffer.flush()` method during COPY operations
## Configuration Sample
```json
{
"host": "my-postgres-server.postgres.database.azure.com",
"port": 5432,
"database": "mydb",
"schema": "myschema",
"username": "myuser",
"password": "********",
"ssl_mode": {
"mode": "require"
},
"tunnel_method": {
"tunnel_method": "NO_TUNNEL"
}
}
```
## Possible Root Cause
The Destinations V2 / Direct Load architecture introduced in v3.0.0 appears to have a bug in timestamp serialization where an empty JSON object literal `"{}"` is being written to the CSV data instead of a properly formatted timestamp string.
## Requested Fix
- Hotfix release for v3.0.4 with corrected timestamp handling
- OR ability to pin/downgrade connector versions via API/UI for self-hosted deployments
Hướng dẫn đóng góp
Đánh giá
Issue này chưa được đánh giá.