when pinot failed to add real time table, the only error given was "null"
- Dominant language
- Java
- Stars
- 6.1k
- Forks
- 1.5k
- Avg merge
- 2d 55m
- Merged PRs (30d)
- 182
Description
Simply trying to add a realtime table to pinot where the table config json includes this:
```
"ingestionConfig": {
"streamIngestionConfig": {
"streamConfigs": [
{
"realtime.segment.flush.threshold.rows": "0",
"stream.kafka.decoder.prop.format": "JSON",
"bootstrap.servers": ":9092",
"key.serializer": "org.apache.kafka.common.serialization.ByteArraySerializer",
"stream.kafka.decoder.class.name": "org.apache.pinot.plugin.inputformat.json.JSONMessageDecoder",
"streamType": "kafka",
"value.serializer": "org.apache.kafka.common.serialization.ByteArraySerializer",
"stream.kafka.consumer.type": "LOWLEVEL",
"realtime.segment.flush.threshold.segment.size": "200M",
"stream.kafka.broker.list": ":9092",
"realtime.segment.flush.threshold.time": "24h",
"stream.kafka.consumer.factory.class.name": "org.apache.pinot.plugin.stream.kafka20.KafkaConsumerFactory",
"stream.kafka.consumer.prop.auto.offset.reset": "smallest",
"stream.kafka.topic.name": "test_topic"
}
]
},
```
fails, with the following api response:
```
{
"code": 400,
"error": null
}
```
In the pinot controller logs, the only related log messages are these:
```
2023/04/27 17:05:37.368 INFO [PinotTableRestletResource] [grizzly-http-server-2] null exception: null
2023/04/27 17:05:37.368 INFO [ControllerResponseFilter] [grizzly-http-server-2] Handled request from POST https://:443/tables, content-type application/json status code 400 Bad Request
```
which basically provides no useful information.
**ISSUE: In the logs, there should be _some_ piece of information logged/provided that helps to identify the problem cause.**
As shown above, there is none.
Here, the problem cause was a misconfiguration, it should have been
```
"ingestionConfig": {
"streamIngestionConfig": {
"streamConfigMaps": [
...
```
instead of
```
"ingestionConfig": {
"streamIngestionConfig": {
"streamConfigs": [
...
```
Contributor guide
Research direction
Start by tracing the POST /tables request through PinotTableRestletResource and ControllerResponseFilter, using the malformed streamConfigs example to reproduce the 400 response. Inspect how the exception and error response are populated. Done means the logs or API response identify the configuration cause instead of reporting null.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- api, backend-api-design
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100