apache / apache/pinot

When an Exception happens when reading a JSON Record file, a NullPointerException is propagated instead

Open
#5,129 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Java
Stars
6.1k
Forks
1.5k
Avg merge
2d 55m
Merged PRs (30d)
182

Description

The underlying Exception that caused the JSON parsing to fail is obfuscating by this exception. We need to wrap an

Got exception to kick off standalone data ingestion job -
java.lang.NullPointerException: null
at org.apache.pinot.plugin.inputformat.json.JSONRecordReader.init(JSONRecordReader.java:56) ~[pinot-json-0.3.0-SNAPSHOT-shaded.jar:0.3.0-SNAPSHOT-565374bbabd1b359f8d4f8d85d62ffd0f3d27458]
at org.apache.pinot.plugin.inputformat.json.JSONRecordReader.init(JSONRecordReader.java:67) ~[pinot-json-0.3.0-SNAPSHOT-shaded.jar:0.3.0-SNAPSHOT-565374bbabd1b359f8d4f8d85d62ffd0f3d27458]
at org.apache.pinot.plugin.ingestion.batch.common.SegmentGenerationTaskRunner.run(SegmentGenerationTaskRunner.java:88) ~[pinot-batch-ingestion-hadoop-0.3.0-SNAPSHOT-shaded.jar:0.3.0-SNAPSHOT-565374bbabd1b359f8d4f8d85d62ffd0f3d27458]
at org.apache.pinot.plugin.ingestion.batch.standalone.SegmentGenerationJobRunner.run(SegmentGenerationJobRunner.java:190) ~[pinot-batch-ingestion-standalone-0.3.0-SNAPSHOT-shaded.jar:0.3.0-SNAPSHOT-565374bbabd1b359f8d4f8d85d62ffd0f3d27458]
at org.apache.pinot.spi.ingestion.batch.IngestionJobLauncher.kickoffIngestionJob(IngestionJobLauncher.java:96) ~[pinot-all-0.3.0-SNAPSHOT-jar-with-dependencies.jar:0.3.0-SNAPSHOT-565374bbabd1b359f8d4f8d85d62ffd0f3d27458]
at org.apache.pinot.spi.ingestion.batch.IngestionJobLauncher.runIngestionJob(IngestionJobLauncher.java:77) ~[pinot-all-0.3.0-SNAPSHOT-jar-with-dependencies.jar:0.3.0-SNAPSHOT-565374bbabd1b359f8d4f8d85d62ffd0f3d27458]
at org.apache.pinot.spi.ingestion.batch.IngestionJobLauncher.main(IngestionJobLauncher.java:55) ~[pinot-all-0.3.0-SNAPSHOT-jar-with-dependencies.jar:0.3.0-SNAPSHOT-565374bbabd1b359f8d4f8d85d62ffd0f3d27458]
at org.apache.pinot.tools.admin.command.LaunchDataIngestionJobCommand.execute(LaunchDataIngestionJobCommand.java:50) [pinot-all-0.3.0-SNAPSHOT-jar-with-dependencies.jar:0.3.0-SNAPSHOT-565374bbabd1b359f8d4f8d85d62ffd0f3d27458]
at org.apache.pinot.tools.admin.PinotAdministrator.execute(PinotAdministrator.java:152) [pinot-all-0.3.0-SNAPSHOT-jar-with-dependencies.jar:0.3.0-SNAPSHOT-565374bbabd1b359f8d4f8d85d62ffd0f3d27458]
at org.apache.pinot.tools.admin.PinotAdministrator.main(PinotAdministrator.java:164) [pinot-all-0.3.0-SNAPSHOT-jar-with-dependencies.jar:0.3.0-SNAPSHOT-565374bbabd1b359f8d4f8d85d62ffd0f3d27458]
Exception caught:
java.lang.NullPointerException: null
at org.apache.pinot.plugin.inputformat.json.JSONRecordReader.init(JSONRecordReader.java:56) ~[pinot-json-0.3.0-SNAPSHOT-shaded.jar:0.3.0-SNAPSHOT-565374bbabd1b359f8d4f8d85d62ffd0f3d27458]
at org.apache.pinot.plugin.inputformat.json.JSONRecordReader.init(JSONRecordReader.java:67) ~[pinot-json-0.3.0-SNAPSHOT-shaded.jar:0.3.0-SNAPSHOT-565374bbabd1b359f8d4f8d85d62ffd0f3d27458]
at org.apache.pinot.plugin.ingestion.batch.common.SegmentGenerationTaskRunner.run(SegmentGenerationTaskRunner.java:88) ~[pinot-batch-ingestion-hadoop-0.3.0-SNAPSHOT-shaded.jar:0.3.0-SNAPSHOT-565374bbabd1b359f8d4f8d85d62ffd0f3d27458]
at org.apache.pinot.plugin.ingestion.batch.standalone.SegmentGenerationJobRunner.run(SegmentGenerationJobRunner.java:190) ~[pinot-batch-ingestion-standalone-0.3.0-SNAPSHOT-shaded.jar:0.3.0-SNAPSHOT-565374bbabd1b359f8d4f8d85d62ffd0f3d27458]
at org.apache.pinot.spi.ingestion.batch.IngestionJobLauncher.kickoffIngestionJob(IngestionJobLauncher.java:96) ~[pinot-all-0.3.0-SNAPSHOT-jar-with-dependencies.jar:0.3.0-SNAPSHOT-565374bbabd1b359f8d4f8d85d62ffd0f3d27458]
at org.apache.pinot.spi.ingestion.batch.IngestionJobLauncher.runIngestionJob(IngestionJobLauncher.java:77) ~[pinot-all-0.3.0-SNAPSHOT-jar-with-dependencies.jar:0.3.0-SNAPSHOT-565374bbabd1b359f8d4f8d85d62ffd0f3d27458]
at org.apache.pinot.spi.ingestion.batch.IngestionJobLauncher.main(IngestionJobLauncher.java:55) ~[pinot-all-0.3.0-SNAPSHOT-jar-with-dependencies.jar:0.3.0-SNAPSHOT-565374bbabd1b359f8d4f8d85d62ffd0f3d27458]
at org.apache.pinot.tools.admin.command.LaunchDataIngestionJobCommand.execute(LaunchDataIngestionJobCommand.java:50) ~[pinot-all-0.3.0-SNAPSHOT-jar-with-dependencies.jar:0.3.0-SNAPSHOT-565374bbabd1b359f8d4f8d85d62ffd0f3d27458]
at org.apache.pinot.tools.admin.PinotAdministrator.execute(PinotAdministrator.java:152) [pinot-all-0.3.0-SNAPSHOT-jar-with-dependencies.jar:0.3.0-SNAPSHOT-565374bbabd1b359f8d4f8d85d62ffd0f3d27458]
at org.apache.pinot.tools.admin.PinotAdministrator.main(PinotAdministrator.java:164) [pinot-all-0.3.0-SNAPSHOT-jar-with-dependencies.jar:0.3.0-SNAPSHOT-565374bbabd1b359f8d4f8d85d62ffd0f3d27458]

Contributor guide

Open the contributing guide

Research direction

Start in JSONRecordReader.init at lines 56 and 67, then trace how the JSON parsing failure is handled during standalone ingestion. Reproduce the issue with malformed JSON and verify that the resulting error preserves the underlying parsing exception instead of propagating only a NullPointerException.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
data-engineering
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.