airbytehq / airbytehq/airbyte

[destination-s3] Sync fails on WRITE init: NPE in DestinationStreamFactory when generationId/syncId are null (platform 2.0.0, Helm 2.0.18, AWS cn-north-1)

Offen
#75,210 11 Kommentare 0 Reaktionen 0 zugewiesene Personen Auf GitHub ansehen
area/connectors autoteam community connectors/destination/s3 needs-triage team/extensibility type/bug
Vorherrschende Sprache
Python
Sterne
22.1k
Forks
5.3k
PR-Merge-Kennzahlen
PR-Kennzahlen ausstehend

Beschreibung

### Connector Name

destination-s3

### Connector Version

1.9.7

### What step the error happened?

During the sync

### Relevant information

### Deployment / region
- **Cloud:** AWS **China (Beijing) `cn-north-1`**
- **Orchestration:** EKS + **Helm** chart **`airbyte-v2/airbyte`** version **`2.0.18

[values.dummy.yaml](https://github.com/user-attachments/files/26113975/values.dummy.yaml)

[replication-job-6-attempt-0.log](https://github.com/user-attachments/files/26114020/replication-job-6-attempt-0.log)

`**
- **Container images:** Pulled from **Amazon ECR** in `cn-north-1` (private registry + `imagePullSecrets`), not directly from Docker Hub at runtime.

### Platform images (all use `global.image.tag`, mirrored to ECR)
Full image refs follow pattern: `.dkr.ecr.cn-north-1.amazonaws.com.cn/airbyte/:2.0.0`

| Component |
|-----------|
| `airbyte/bootloader` |
| `airbyte/connector-builder-server` |
| `airbyte/connector-sidecar` |
| `airbyte/container-orchestrator` |
| `airbyte/cron` |
| `airbyte/server` |
| `airbyte/worker` |
| `airbyte/workload-api-server` |
| `airbyte/workload-init-container` |
| `airbyte/workload-launcher` |

Additional override in values: `airbyte/async-profiler:build-c16119df63-24114-master` (connector profiler).
Temporal: `temporalio/auto-setup:1.27.2`

### Helm values (sanitized)
See attached **`values.dummy.yaml`** in this repo (or paste gist) — same shape as production **without** real account IDs, RDS endpoints, or bucket names.

### Connection setup
- **Source:** Postgres (non-CDC), one stream, **Full refresh | Overwrite** (Also has tried with source s3, Source setups works)
- **Destination:** S3 (official `destination-s3`), **CHECK succeeds**
- **Failure:** Sync fails immediately on destination **WRITE** initialization

### Expected
Destination **WRITE** should start successfully: either the **bulk CDK** (`S3V2Destination`) tolerates missing generation/sync ids, or the **platform/orchestrator** always populates `generationId`, `minimumGenerationId`, and `syncId` on `ConfiguredAirbyteStream` for this sync path.

### Actual
- **Destination CHECK** succeeds; **sync** fails immediately on destination **WRITE** init.
- **`ConfigErrorException: Failed to initialize connector operation`** with root cause **`NullPointerException`** at **`DestinationStreamFactory.make(DestinationStream.kt:204)`** (`S3V2Destination` / `S3V2ConfigurationFactory`).
- **`ConfiguredAirbyteStream`** logged by the Postgres source shows **`generationId=, minimumGenerationId=, syncId=`** while **`destinationSyncMode=overwrite`** / resumable full refresh.
- **Sync summary:** `recordsSynced: 0`, `failureOrigin: destination`, `failureType: config_error`, **`connector_command: write`** (example job id **6**).

## Summary (troubleshooting attempts)

**Helm:** tested **`airbyte/airbyte` `1.8.0`** and **`airbyte-v2/airbyte` `2.0.18`** (latter with platform **`global.image.tag` `2.0.0`**). **`destination-s3` tags tested:** **`1.9.7`**, **`1.9.2`**, **`1.8.8`**, **`1.8.6`**.

### Relevant log output

```shell

```

### Contribute

- [ ] Yes, I want to contribute
---

**Internal Tracking:** https://github.com/airbytehq/oncall/issues/11702

Beitragsleitfaden

Beitragsleitfaden öffnen

Bewertung

Dieses Issue wurde noch nicht bewertet.

Neue Issues direkt in Ihr Postfach

Eine kurze Übersicht über anfängerfreundliche GitHub-Issues.