Aiven-Open / Aiven-Open/cloud-storage-connectors-for-apache-kafka

[QUESTION] How to properly encode CSV files for easy readability (for BigQuery external tables)?

Open
#260 0 comments 0 reactions 0 assignees View on GitHub
GCS
Dominant language
Java
Stars
58
Forks
39
Avg merge
2d 10h
Merged PRs (30d)
5

Description

I am trying to create files in CSV format for storage on a GCS bucket. It seems with encoding set to Base64, the CSV files remain as such in plaintext, and when I try to create an external table in BigQuery with these files, the data remains as Base64 in the query results.

When I use "none" as the encoding, querying the external table throws an ASCII error (Bad character (ASCII 0) encountered.).

What is the correct way to go about this? I figured it would work if I could use the Avro converter for the key.converter property, but it seems this is not possible.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.