airbytehq / airbytehq/airbyte

Destination S3: Source Kafka <> Destination S3 -> Kafka 'key' field getting dropped in S3

Aberta
#26,417 3 comentários 0 reações 0 responsáveis Ver no GitHub
area/connectors community connectors/destination/s3 connectors/destinations-files frozen releaseStage/ga team/destinations type/bug
Linguagem predominante
Python
Estrelas
22.1k
Forks
5.4k
Merge médio
5h
PRs com merge (30d)
671

Descrição

### Connector Name

destination-s3

### Connector Version

0.4.1

### What step the error happened?

During the sync

### Revelant information

Source Kafka version used: 0.2.3
Destination S3 Version: 0.4.1
We are noticing that kafka message **key** is not getting synced in s3. Schema being replicated in S3:
**_airbyte_ab_id, _airbyte_emitted_at, value, _airbyte_additional_properties**
where value is null and record values are present in _airbyte_additional_properties
![Screenshot 2023-05-23 at 6 48 06 PM](https://github.com/airbytehq/airbyte/assets/7288502/76a0854d-25f2-4991-949b-0ca3a6af6dea)

### Relevant log output

```shell
Will share logs if required - not attaching right now, since it contains sensitive data at INFO level logs - will raise a different PR to fix that
```

### Contribute

- [X] Yes, I want to contribute

Guia de contribuição

Abrir o guia de contribuição

Avaliação

Esta issue ainda não foi avaliada.

Receba novas issues na sua caixa de entrada

Um resumo curto de issues do GitHub para quem está começando.