FakeJobsService tries to base64 decode a plain QueryString resulting in an error.
- Dominant language
- Java
- Stars
- 8.7k
- Forks
- 4.7k
- Avg merge
- 1d 20h
- Merged PRs (30d)
- 196
Description
https://github.com/apache/beam/blob/fbeae980e93ea64fc8cc3ad074cbc8aebd157691/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/testing/FakeJobService.java#L488
Not Sure if I am doing something wrong setting up the. FakeBigQueryServices
I have a pipeline reading from BigQuery Table select * from project:dataset.table, the contents is the published to a topic on a queue .
The Pipeline is configured with the FakeBigQueryServices and PubsubTestClient to shield of the real interactions with GCP.
After setting up the expected tables with te expected rows, the pipeline in BigquerIO wants to execute this part:
```
private JobStatus runQueryJob(JobConfigurationQuery query)
throws IOException, InterruptedException {
List rows = FakeBigQueryServices.rowsFromEncodedQuery(query.getQuery());
datasetService.createTable(new Table().setTableReference(query.getDestinationTable()));
datasetService.insertAll(query.getDestinationTable(), rows, null);
return new JobStatus().setState("DONE");
}
```
The query from the job is in plain string: select * from project:dataset.table, but somehow expects a base64 encoded version or encoded results?
resulting in:
```
com.google.common.io.BaseEncoding$DecodingException: Unrecognized character: 0x20"
```
What am I missing?
Contributor guide
Assessment
This issue has not been assessed yet.