[destination-s3] Sync fails on WRITE init: NPE in DestinationStreamFactory when generationId/syncId are null (platform 2.0.0, Helm 2.0.18, AWS cn-north-1)
- Vorherrschende Sprache
- Python
- Sterne
- 22.1k
- Forks
- 5.3k
- PR-Merge-Kennzahlen
- PR-Kennzahlen ausstehend
Beschreibung
### Connector Name
destination-s3
### Connector Version
1.9.7
### What step the error happened?
During the sync
### Relevant information
### Deployment / region
- **Cloud:** AWS **China (Beijing) `cn-north-1`**
- **Orchestration:** EKS + **Helm** chart **`airbyte-v2/airbyte`** version **`2.0.18
[values.dummy.yaml](https://github.com/user-attachments/files/26113975/values.dummy.yaml)
[replication-job-6-attempt-0.log](https://github.com/user-attachments/files/26114020/replication-job-6-attempt-0.log)
`**
- **Container images:** Pulled from **Amazon ECR** in `cn-north-1` (private registry + `imagePullSecrets`), not directly from Docker Hub at runtime.
### Platform images (all use `global.image.tag`, mirrored to ECR)
Full image refs follow pattern: `.dkr.ecr.cn-north-1.amazonaws.com.cn/airbyte/:2.0.0`
| Component |
|-----------|
| `airbyte/bootloader` |
| `airbyte/connector-builder-server` |
| `airbyte/connector-sidecar` |
| `airbyte/container-orchestrator` |
| `airbyte/cron` |
| `airbyte/server` |
| `airbyte/worker` |
| `airbyte/workload-api-server` |
| `airbyte/workload-init-container` |
| `airbyte/workload-launcher` |
Additional override in values: `airbyte/async-profiler:build-c16119df63-24114-master` (connector profiler).
Temporal: `temporalio/auto-setup:1.27.2`
### Helm values (sanitized)
See attached **`values.dummy.yaml`** in this repo (or paste gist) — same shape as production **without** real account IDs, RDS endpoints, or bucket names.
### Connection setup
- **Source:** Postgres (non-CDC), one stream, **Full refresh | Overwrite** (Also has tried with source s3, Source setups works)
- **Destination:** S3 (official `destination-s3`), **CHECK succeeds**
- **Failure:** Sync fails immediately on destination **WRITE** initialization
### Expected
Destination **WRITE** should start successfully: either the **bulk CDK** (`S3V2Destination`) tolerates missing generation/sync ids, or the **platform/orchestrator** always populates `generationId`, `minimumGenerationId`, and `syncId` on `ConfiguredAirbyteStream` for this sync path.
### Actual
- **Destination CHECK** succeeds; **sync** fails immediately on destination **WRITE** init.
- **`ConfigErrorException: Failed to initialize connector operation`** with root cause **`NullPointerException`** at **`DestinationStreamFactory.make(DestinationStream.kt:204)`** (`S3V2Destination` / `S3V2ConfigurationFactory`).
- **`ConfiguredAirbyteStream`** logged by the Postgres source shows **`generationId=, minimumGenerationId=, syncId=`** while **`destinationSyncMode=overwrite`** / resumable full refresh.
- **Sync summary:** `recordsSynced: 0`, `failureOrigin: destination`, `failureType: config_error`, **`connector_command: write`** (example job id **6**).
## Summary (troubleshooting attempts)
**Helm:** tested **`airbyte/airbyte` `1.8.0`** and **`airbyte-v2/airbyte` `2.0.18`** (latter with platform **`global.image.tag` `2.0.0`**). **`destination-s3` tags tested:** **`1.9.7`**, **`1.9.2`**, **`1.8.8`**, **`1.8.6`**.
### Relevant log output
```shell
```
### Contribute
- [ ] Yes, I want to contribute
---
**Internal Tracking:** https://github.com/airbytehq/oncall/issues/11702
Beitragsleitfaden
Bewertung
Dieses Issue wurde noch nicht bewertet.