Auto tune table config
- Dominant language
- Java
- Stars
- 6.1k
- Forks
- 1.5k
- Avg merge
- 1d 21h
- Merged PRs (30d)
- 189
Description
Make it easier for people to create a Pinot table which can execute queries in a performant way. Usually this means the Pinot user has to configure things such as inverted indexes. We can do this automatically.
For instance, users can specify something like:
```
"tableIndexConfig": {
“mode”: “auto”
"streamConfigs" : {
"streamType": "kafka",
...
}
}
```
which can be automatically replaced with the necessary config.
The implementation can be done in 2 parts:
1) Have a pluggable interface for defining such customizations
2) Iterate on different types of such auto-tuning implementations.
Contributor guide
Research direction
Start by locating where Pinot parses and applies tableIndexConfig, then review how table configuration and streamConfigs are represented. The issue is complete when a pluggable customization interface is defined and an initial auto-tuning implementation can replace an auto mode with the necessary configuration.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- databases
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100