UI - Reporting invalid configurations during table creation
- Dominant language
- Java
- Stars
- 6.1k
- Forks
- 1.5k
- Avg merge
- 1d 21h
- Merged PRs (30d)
- 189
Description
Bug found in pinot v0.7.0
When creating a table definition in the UI, it will not allow definitions with `upsertConfig` due to reporting that such configurations are invalid.
A workaround for this is to directly use the swagger API. It seems the UI is not using the same backend service for configuration validation.
An example configuration could be:
```
{
"REALTIME": {
"tableName": "Table_REALTIME",
"tableType": "REALTIME",
"segmentsConfig": {
"timeColumnName": "timestamp",
"replication": "1",
"replicasPerPartition": "1",
"schemaName": "Table"
},
"tenants": {
"broker": "DefaultTenant",
"server": "DefaultTenant",
"tagOverrideConfig": {}
},
"tableIndexConfig": {
"invertedIndexColumns": [],
"rangeIndexColumns": [],
"varLengthDictionaryColumns": [],
"enableDefaultStarTree": false,
"enableDynamicStarTreeCreation": false,
"aggregateMetrics": false,
"nullHandlingEnabled": false,
"autoGeneratedInvertedIndex": false,
"createInvertedIndexDuringSegmentGeneration": false,
"bloomFilterColumns": [],
"loadMode": "MMAP",
"streamConfigs": {
"streamType": "kafka",
"stream.kafka.topic.name": "data.Table",
"stream.kafka.broker.list": "kafka.dc-kafka.svc.cluster.local:9092",
"stream.kafka.consumer.type": "lowlevel",
"stream.kafka.consumer.prop.auto.offset.reset": "smallest",
"stream.kafka.consumer.factory.class.name": "org.apache.pinot.plugin.stream.kafka20.KafkaConsumerFactory",
"stream.kafka.decoder.class.name": "org.apache.pinot.plugin.stream.kafka.KafkaJSONMessageDecoder",
"realtime.segment.flush.threshold.rows": "0",
"realtime.segment.flush.threshold.time": "24h",
"realtime.segment.flush.segment.size": "100M"
},
"noDictionaryColumns": [],
"onHeapDictionaryColumns": [],
"sortedColumn": []
},
"metadata": {},
"quota": {},
"routing": {},
"query": {},
"upsertConfig": {
"mode": "FULL"
},
"ingestionConfig": {},
"isDimTable": false
}
}
```
With a schema definition:
```
{
"schemaName": "Table",
"dimensionFieldSpecs": [
{
"name": "executionId",
"dataType": "STRING"
},
{
"name": "jobId",
"dataType": "STRING"
},
{
"name": "jobMemberId",
"dataType": "STRING"
},
{
"name": "projectId",
"dataType": "STRING"
}
],
"dateTimeFieldSpecs": [
{
"name": "timestamp",
"dataType": "LONG",
"format": "1:MILLISECONDS:EPOCH",
"granularity": "1:MILLISECONDS"
}
],
"primaryKeyColumns": [
"executionId"
]
}
```
Contributor guide
Research direction
Reproduce the problem in the table-creation UI using the supplied table and schema definitions, then compare its validation behavior with the Swagger API workaround. The work is done when a table definition containing upsertConfig is accepted consistently by the UI and backend.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- frontend
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 42/100