airbytehq / airbytehq/airbyte

PostgreSQL Destination v3.0.4 - Invalid timestamp value "{}" causes COPY failures (self-hosted)

Đang mở
#70,844 2 bình luận 1 reaction 0 người được giao Xem trên GitHub
autoteam community team/use
Ngôn ngữ chính
Python
Star
22.1k
Fork
5.4k
Merge trung bình
5 giờ
Pull request đã merge (30 ngày)
671

Mô tả

## Bug Description
The PostgreSQL destination connector version 3.0.4 is passing an invalid string `"{}"` instead of a proper timestamp value for the `_airbyte_extracted_at` column, causing all connection tests and syncs to fail with a PostgreSQL type conversion error.

## Environment
- **Airbyte Version**: Self-hosted via abctl on Azure VM (Ubuntu)
- **Connector**: Postgres Destination
- **Connector Version**: 3.0.4
- **Deployment**: Self-hosted (abctl)
- **Target Database**: Azure PostgreSQL Flexible Server v16

## Steps to Reproduce
1. Configure a PostgreSQL destination with any valid connection details
2. Test the connection OR run any sync
3. Observe the error during COPY operation

## Expected Behavior
The connector should insert a valid `timestamp with time zone` value for `_airbyte_extracted_at` column.

## Actual Behavior
The connector attempts to insert the literal string `"{}"` as a timestamp value, causing PostgreSQL to reject the COPY operation with:

```
ERROR: invalid input syntax for type timestamp with time zone: "{}"
Where: COPY _airbyte_connection_test_, line 1, column _airbyte_extracted_at: "{}"
```

## Error Logs
```
ERROR main i.a.i.d.p.w.l.PostgresInsertBuffer(flush):66 Unable to flush accumulated data.
org.postgresql.util.PSQLException: ERROR: invalid input syntax for type timestamp with time zone: "{}"
Where: COPY _airbyte_connection_test_84699f86a1fc4af2bfb3ff9106acab84, line 1, column _airbyte_extracted_at: "{}"
at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2733)
at org.postgresql.core.v3.QueryExecutorImpl.processCopyResults(QueryExecutorImpl.java:1311)
at org.postgresql.core.v3.QueryExecutorImpl.endCopy(QueryExecutorImpl.java:1107)
at org.postgresql.core.v3.CopyInImpl.endCopy(CopyInImpl.java:53)
at org.postgresql.copy.CopyManager.copyIn(CopyManager.java:227)
at org.postgresql.copy.CopyManager.copyIn(CopyManager.java:203)
at io.airbyte.integrations.destination.postgres.client.PostgresAirbyteClient.copyFromCsv(PostgresAirbyteClient.kt:367)
at io.airbyte.integrations.destination.postgres.write.load.PostgresInsertBuffer.flush(PostgresInsertBuffer.kt:61)
```

Stack trace shows `bulk-cdk-core-load-0.1.83.jar` is being used.

## Impact
- **Severity**: Critical - PostgreSQL destinations are completely non-functional on v3.0.4
- **Scope**: Affects both connection tests and actual syncs
- **Workaround Attempted**:
- Enabled "Disable Final Tables" option - same error persists
- Attempted to downgrade via UI - operation times out
- Attempted Docker image manipulation - connector still pulls v3.0.4

## Additional Context
- Using self-hosted Airbyte deployed via abctl on Azure VM
- Connection works fine from command line (psql, Python), so it's not a network/firewall issue
- The bug appears in the `PostgresInsertBuffer.flush()` method during COPY operations

## Configuration Sample
```json
{
"host": "my-postgres-server.postgres.database.azure.com",
"port": 5432,
"database": "mydb",
"schema": "myschema",
"username": "myuser",
"password": "********",
"ssl_mode": {
"mode": "require"
},
"tunnel_method": {
"tunnel_method": "NO_TUNNEL"
}
}
```

## Possible Root Cause
The Destinations V2 / Direct Load architecture introduced in v3.0.0 appears to have a bug in timestamp serialization where an empty JSON object literal `"{}"` is being written to the CSV data instead of a properly formatted timestamp string.

## Requested Fix
- Hotfix release for v3.0.4 with corrected timestamp handling
- OR ability to pin/downgrade connector versions via API/UI for self-hosted deployments

Hướng dẫn đóng góp

Mở hướng dẫn đóng góp

Đánh giá

Issue này chưa được đánh giá.

Nhận issue mới trong hộp thư của bạn

Bản tóm tắt ngắn những issue GitHub phù hợp với người mới.