apache / apache/pinot

Add JSON integration tests to LLCRealtimeClusterIntegrationTest

Open
#10,316 8 comments 0 reactions 0 assignees View on GitHub
Dominant language
Java
Stars
6.1k
Forks
1.5k
Avg merge
2d 55m
Merged PRs (30d)
182

Description

There seems to be some issues in the code path that ingest JSON data for REALTIME tables.

If data like the following gets ingested from Kafka:
"EvaluationResult" : {
"AssertionEvaluationResult" : {
"summary" : {
"passed" : 0,
"failed" : 0,
"warned" : 0,
"overallStatus" : "XXXX"
},
"evaluatedAssertions" : [ ]
}
}

The nested empty array gets turned into a NULL in pinot DB, expected behavior would be we get an empty array in Pinot:

![Screenshot 2023-02-21 at 9 40 17 AM](https://user-images.githubusercontent.com/109560870/220419676-aaf1bb24-95d7-4157-97df-b4c2007d2a8e.png)

Contributor guide

Open the contributing guide

Research direction

Start with LLCRealtimeClusterIntegrationTest and trace its realtime JSON ingestion setup for Kafka input. Add coverage for the nested empty evaluatedAssertions array, then verify the integration test preserves it as an empty array rather than NULL in Pinot.

Written by the indexing model from the issue text.

Assessment

Tech stack
java, kafka
Domain
databases, testing
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Clearly specified
Newbie friendliness
50/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.