apache / apache/beam

Problem loading ISO-8859-1 into BigQuery using DataFlow

Open
#19,723 0 comments 0 reactions 0 assignees View on GitHub
bug gcp io java P3 text
Dominant language
Java
Stars
8.7k
Forks
4.7k
Avg merge
1d 20h
Merged PRs (30d)
196

Description

Hi all,
I'm trying to load an ISO-8859-1 file into BigQuery using DataFlow. I've built a template with Apache Beam Java. Everything works well but when I check the content of the Bigquery table I see that some characters like 'ñ' or accents 'á','é', etc. haven't been stored propertly, they have been stored as �.

I've tried several charset changing before write into BigQuery. Also, I've created a special ISOCoder passed to the pipeline using the method setCoder(), but nothing works.

Imported from Jira [BEAM-7826](https://issues.apache.org/jira/browse/BEAM-7826). Original Jira may contain additional context.
Reported by: Israel.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.