apache / apache/beam

[SQL] Document/fix text table (TextIO) insert behavior

Open
#19,014 0 comments 0 reactions 0 assignees View on GitHub
dsl improvement P3 sql
Dominant language
Java
Stars
8.7k
Forks
4.7k
Avg merge
1d 20h
Merged PRs (30d)
196

Description

Tables of type `text` are backed by TextIO. We don't do any extra configuration, so it results in behavior like this:

 - create text table with location 'test-file.csv'

 - if the file exists, you can select from it;

 - now insert values into that table;

     - this results in TextIO writing files with names like 'test-file.csv-00000-of-00001';

     - selecting from the original table doesn't allow selecting from the newly created files;

We need to properly document this, possibly log to Stdout.

Imported from Jira [BEAM-4815](https://issues.apache.org/jira/browse/BEAM-4815). Original Jira may contain additional context.
Reported by: kedin.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.