apache / apache/beam

[Task] (BQ Java): Create an integration test that involves the use of both single and multi partition writes

Open
#23,614 0 comments 0 reactions 0 assignees View on GitHub
gcp io java P2 task
Dominant language
Java
Stars
8.7k
Forks
4.7k
Avg merge
1d 20h
Merged PRs (30d)
196

Description

### What needs to happen?

Need an integration test that fully makes use of [this flatten step in batch loads](https://github.com/ahmedabu98/beam/blob/ef85e6b948c4fd02b85f93e493e3dc50be2299c9/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/BatchLoads.java#L409-L412).

Should look like a file loads write with dynamic destinations, one destination needing copy jobs, one needing only single load job.

### Issue Priority

Priority: 2

### Issue Component

Component: io-java-gcp

Contributor guide

Open the contributing guide

Research direction

Start with the flatten step in sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/BatchLoads.java at the linked lines, then locate the BigQuery Java integration-test entry point for file-load writes. Add coverage using dynamic destinations where one destination requires copy jobs and another requires only a single load job; the test should exercise both single- and multi-partition writes.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
databases, testing
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.