apache / apache/beam

BigQueryIO.readTableRows() throws NullPointerException

Open
#21,079 0 comments 0 reactions 0 assignees View on GitHub
bug gcp io java P3
Dominant language
Java
Stars
8.7k
Forks
4.7k
Avg merge
1d 20h
Merged PRs (30d)
196

Description

I tried different methods to extract table content using *BigQueryIO* but in all scenarios, when I try to use *DataflowRunner* I always get the same *NullPointerException*.

The reason I found for this error is that while reading the table, *if there are any null values it ignore them without setting the null value*.

```

        PCollection tableRows = p.apply("ReadFromBQ", 

        BigQueryIO

        ._readTableRows_()

 
      .from(project:dataset.table)

        );

```

 

The only thing it worked was using DirectRunner with `.readTableRowsWithSchema()` but using DataflowRunner I'm still facing the issue.

 

Please solve this bug and/or help me to find a solution.

Imported from Jira [BEAM-12835](https://issues.apache.org/jira/browse/BEAM-12835). Original Jira may contain additional context.
Reported by: Fra.G.

Contributor guide

Open the contributing guide

Research direction

Start by reproducing the failure with BigQueryIO.readTableRows() under DataflowRunner using a table containing null values, then compare it with DirectRunner and readTableRowsWithSchema(). Trace the BigQueryIO read path to identify where null values are handled; done means the DataflowRunner read completes without a NullPointerException and preserves null fields.

Written by the indexing model from the issue text.

Assessment

Tech stack
google-cloud, java
Domain
databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.