apache / apache/pinot

JSON Index - Parser fails for some normal JSONs

Open
#6,699 2 comments 0 reactions 0 assignees View on GitHub
Dominant language
Java
Stars
6.1k
Forks
1.5k
Avg merge
1d 21h
Merged PRs (30d)
189

Description

I have a JSON which looks like the one below. But JSON parser fails with errors

```
{
"message": "\"Executing query =: \"",
"messageobj": "{\"statement\":\"select sysdate from dual\",\"binds\":{},\"opts\":{\"outFormat\":4002,\"autoCommit\":true}}",
"executionContext": "{\"system\":\"test\",\"subsystem\":\"operations\",\"capability\":\"checksettings\",\"resource\":\"simplefireconsumer-hsh\",\"transactionid\":\"10ba4d0d-0a63-4a40-be70-92c2a7837e3c\",\"username\":\"--REDACTED--\",\"urlpath\":\"/operations/checksettings/v1/simplefireconsumer-hsh/healthcheck\",\"requestheaders\":{\"host\":\"145.72.134.21:3000\",\"user-agent\":\"kube-probe/1.16+\",\"accept-encoding\":\"gzip\",\"connection\":\"close\",\"messageauditid\":\"23cf3b61-acc7-42a7-8fac-2f22506f4652\"}}",
"transactionid": "10ba4d0d-0a63-4a40-be70-92c2a7837e3c",
"correlationid": "",
"sessionid": "",
"sender": "",
"system": "test",
"subsystem": "operations",
"capability": "checksettings",
"resource": "simplefireconsumer-hsh",
"urlpath": "/operations/checksettings/v1/simplefireconsumer-hsh/healthcheck",
"testloggerversion": "2.0",
"level": "info",
"timestamp": "2021-03-19T04:39:43.822Z"
}
```

JSON errors

server java.lang.IndexOutOfBoundsException: Index: 6279, Size: 1
at java.util.ArrayList.get(ArrayList.java:435) ~[?:1.8.0_282]

shaded.com.fasterxml.jackson.core.JsonParseException: Unexpected character ('/' (code 47)): maybe a (non-standard) comment? (not recognized as one since Feature 'ALLOW_COMMENTS' not enabled for parser)

Is it possible to have a dirty XML parser option as well. Where it parses whatever value it can and just skips the really bad ones?

Contributor guide

Open the contributing guide

Research direction

Start with Pinot's JSON Index parser and the Jackson JsonParseException shown in the report, reproducing the supplied JSON as the failing input. Trace the parser path around the reported IndexOutOfBoundsException, then verify that valid JSON is accepted and that the requested behavior for malformed values is clearly defined and tested.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.