apache / apache/pinot

when pinot failed to add real time table, the only error given was "null"

Open
#10,694 1 comment 1 reaction 0 assignees View on GitHub
Dominant language
Java
Stars
6.1k
Forks
1.5k
Avg merge
2d 55m
Merged PRs (30d)
182

Description

Simply trying to add a realtime table to pinot where the table config json includes this:

```
"ingestionConfig": {
"streamIngestionConfig": {
"streamConfigs": [
{
"realtime.segment.flush.threshold.rows": "0",
"stream.kafka.decoder.prop.format": "JSON",
"bootstrap.servers": ":9092",
"key.serializer": "org.apache.kafka.common.serialization.ByteArraySerializer",
"stream.kafka.decoder.class.name": "org.apache.pinot.plugin.inputformat.json.JSONMessageDecoder",
"streamType": "kafka",
"value.serializer": "org.apache.kafka.common.serialization.ByteArraySerializer",
"stream.kafka.consumer.type": "LOWLEVEL",
"realtime.segment.flush.threshold.segment.size": "200M",
"stream.kafka.broker.list": ":9092",
"realtime.segment.flush.threshold.time": "24h",
"stream.kafka.consumer.factory.class.name": "org.apache.pinot.plugin.stream.kafka20.KafkaConsumerFactory",
"stream.kafka.consumer.prop.auto.offset.reset": "smallest",
"stream.kafka.topic.name": "test_topic"
}
]
},
```

fails, with the following api response:
```
{
"code": 400,
"error": null
}
```

In the pinot controller logs, the only related log messages are these:
```
2023/04/27 17:05:37.368 INFO [PinotTableRestletResource] [grizzly-http-server-2] null exception: null
2023/04/27 17:05:37.368 INFO [ControllerResponseFilter] [grizzly-http-server-2] Handled request from POST https://:443/tables, content-type application/json status code 400 Bad Request
```

which basically provides no useful information.

**ISSUE: In the logs, there should be _some_ piece of information logged/provided that helps to identify the problem cause.**
As shown above, there is none.
Here, the problem cause was a misconfiguration, it should have been
```
"ingestionConfig": {
"streamIngestionConfig": {
"streamConfigMaps": [
...
```
instead of
```
"ingestionConfig": {
"streamIngestionConfig": {
"streamConfigs": [
...
```

Contributor guide

Open the contributing guide

Research direction

Start by tracing the POST /tables request through PinotTableRestletResource and ControllerResponseFilter, using the malformed streamConfigs example to reproduce the 400 response. Inspect how the exception and error response are populated. Done means the logs or API response identify the configuration cause instead of reporting null.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
api, backend-api-design
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.