airbytehq / airbytehq/airbyte

Destination S3: Source Kafka <> Destination S3 -> Kafka 'key' field getting dropped in S3

Aperta
#26,417 3 commenti 0 reazioni 0 assegnatari Vedi su GitHub
area/connectors community connectors/destination/s3 connectors/destinations-files frozen releaseStage/ga team/destinations type/bug
Lingua principale
Python
Stelle
22.1k
Fork
5.3k
Metriche di merge delle PR
Metriche PR in attesa

Descrizione

### Connector Name

destination-s3

### Connector Version

0.4.1

### What step the error happened?

During the sync

### Revelant information

Source Kafka version used: 0.2.3
Destination S3 Version: 0.4.1
We are noticing that kafka message **key** is not getting synced in s3. Schema being replicated in S3:
**_airbyte_ab_id, _airbyte_emitted_at, value, _airbyte_additional_properties**
where value is null and record values are present in _airbyte_additional_properties
![Screenshot 2023-05-23 at 6 48 06 PM](https://github.com/airbytehq/airbyte/assets/7288502/76a0854d-25f2-4991-949b-0ca3a6af6dea)

### Relevant log output

```shell
Will share logs if required - not attaching right now, since it contains sensitive data at INFO level logs - will raise a different PR to fix that
```

### Contribute

- [X] Yes, I want to contribute

Guida per i contributori

Apri la guida per i contributori

Valutazione

Questa issue non è ancora stata valutata.

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.