apache / apache/beam

FakeJobsService tries to base64 decode a plain QueryString resulting in an error.

Open
#26,935 3 comments 1 reaction 0 assignees View on GitHub
Dominant language
Java
Stars
8.7k
Forks
4.7k
Avg merge
1d 20h
Merged PRs (30d)
196

Description

https://github.com/apache/beam/blob/fbeae980e93ea64fc8cc3ad074cbc8aebd157691/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/testing/FakeJobService.java#L488

Not Sure if I am doing something wrong setting up the. FakeBigQueryServices

I have a pipeline reading from BigQuery Table select * from project:dataset.table, the contents is the published to a topic on a queue .
The Pipeline is configured with the FakeBigQueryServices and PubsubTestClient to shield of the real interactions with GCP.

After setting up the expected tables with te expected rows, the pipeline in BigquerIO wants to execute this part:

```
private JobStatus runQueryJob(JobConfigurationQuery query)
throws IOException, InterruptedException {
List rows = FakeBigQueryServices.rowsFromEncodedQuery(query.getQuery());
datasetService.createTable(new Table().setTableReference(query.getDestinationTable()));
datasetService.insertAll(query.getDestinationTable(), rows, null);
return new JobStatus().setState("DONE");
}
```

The query from the job is in plain string: select * from project:dataset.table, but somehow expects a base64 encoded version or encoded results?

resulting in:
```
com.google.common.io.BaseEncoding$DecodingException: Unrecognized character: 0x20"
```

What am I missing?

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.