Support additional configuration in BigQueryServices.insertAll()
- Dominant language
- Java
- Stars
- 8.7k
- Forks
- 4.7k
- Avg merge
- 2d 2h
- Merged PRs (30d)
- 205
Description
ignoreUnknownValues is requested in https://issues.apache.org/jira/browse/BEAM-1267
There are additional configurations that could be useful.
TableDataInsertAllRequest content = new TableDataInsertAllRequest();
content.setSkipInvalidRows();
content.setTemplateSuffix();
content.setKind();
I think we can improve the BigQueryServices interface by define it as:
void insertAll(TableReference ref, Collection request);
and, provided a static method to prepare requests:
List makeInsertBatches(List rowList, @Nullable List insertIdList);
Then, client can set additional config in the returned list.
Imported from Jira [BEAM-1305](https://issues.apache.org/jira/browse/BEAM-1305). Original Jira may contain additional context.
Reported by: peihe0@gmail.com.
Contributor guide
Research direction
Start at the BigQueryServices.insertAll interface and inspect its current implementations and callers. Trace how TableDataInsertAllRequest batches are built, then determine how the requested options should be passed; done means insertAll supports the additional request configuration without breaking existing callers.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- backend, data
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100