JSON Index - Parser fails for some normal JSONs
- Dominant language
- Java
- Stars
- 6.1k
- Forks
- 1.5k
- Avg merge
- 1d 21h
- Merged PRs (30d)
- 189
Description
I have a JSON which looks like the one below. But JSON parser fails with errors
```
{
"message": "\"Executing query =: \"",
"messageobj": "{\"statement\":\"select sysdate from dual\",\"binds\":{},\"opts\":{\"outFormat\":4002,\"autoCommit\":true}}",
"executionContext": "{\"system\":\"test\",\"subsystem\":\"operations\",\"capability\":\"checksettings\",\"resource\":\"simplefireconsumer-hsh\",\"transactionid\":\"10ba4d0d-0a63-4a40-be70-92c2a7837e3c\",\"username\":\"--REDACTED--\",\"urlpath\":\"/operations/checksettings/v1/simplefireconsumer-hsh/healthcheck\",\"requestheaders\":{\"host\":\"145.72.134.21:3000\",\"user-agent\":\"kube-probe/1.16+\",\"accept-encoding\":\"gzip\",\"connection\":\"close\",\"messageauditid\":\"23cf3b61-acc7-42a7-8fac-2f22506f4652\"}}",
"transactionid": "10ba4d0d-0a63-4a40-be70-92c2a7837e3c",
"correlationid": "",
"sessionid": "",
"sender": "",
"system": "test",
"subsystem": "operations",
"capability": "checksettings",
"resource": "simplefireconsumer-hsh",
"urlpath": "/operations/checksettings/v1/simplefireconsumer-hsh/healthcheck",
"testloggerversion": "2.0",
"level": "info",
"timestamp": "2021-03-19T04:39:43.822Z"
}
```
JSON errors
server java.lang.IndexOutOfBoundsException: Index: 6279, Size: 1
at java.util.ArrayList.get(ArrayList.java:435) ~[?:1.8.0_282]
shaded.com.fasterxml.jackson.core.JsonParseException: Unexpected character ('/' (code 47)): maybe a (non-standard) comment? (not recognized as one since Feature 'ALLOW_COMMENTS' not enabled for parser)
Is it possible to have a dirty XML parser option as well. Where it parses whatever value it can and just skips the really bad ones?
Contributor guide
Research direction
Start with Pinot's JSON Index parser and the Jackson JsonParseException shown in the report, reproducing the supplied JSON as the failing input. Trace the parser path around the reported IndexOutOfBoundsException, then verify that valid JSON is accepted and that the requested behavior for malformed values is clearly defined and tested.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- databases
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100