apache / apache/pinot

Clean up table management CLI commands

Open
#10,974 3 comments 1 reaction 1 assignee Claimed by @eupraxis1981 View on GitHub
feature
Dominant language
Java
Stars
6.1k
Forks
1.5k
Avg merge
2d 55m
Merged PRs (30d)
182

Description

# What's the problem?
Cannot add schema and tables separately via CLI. Current AddTable CLI command assumes table(s) and associated schema are uploaded together.

* Prevents using AddSchema -- adding the schema would prevent using AddTable later as it will flag a duplicate schema is being uploaded.
* Sometimes you want to evolve your tables and/or schema and need extra flexibility on when you define schemas vs tables that rely upon them.

# What's the proposal?
Modify AddTable command signature so you can add many tables at once without needing to specify schema separately on command line.

Also update AddSchema to enable updates/overwrites vs just inserts of schemas.

```
Usage: AddTable -tableConfigFiles tableConfigFile1[,tableConfigFile2][,...] [OPTIONS]
Add tables defined in comma-separated list of tableConfig files.
NOTE: Each tableConfig file must specify the table schema and type.

Options:
-schemaFiles Comma-separated list of schemaConfig files to add or update (default null)
-controllerProtocol Protocol to use to connect to controller (default 'http')
-controllerPort Port of controller (default 9000)
-user Username (default null)
-password Password associated with user (default null)
```

Since user has to specify the schema in the tableConfig it is redundant to require it in the command line. However, I see the usefulness of allowing a user to add/update a schema at the same time as uploading tables as a convenience.

We should also expand `AddSchema` to actually be an UPSERT so you can manage schemas more fully from CLI as well.

# Risk
None -- should be backwards compatible with scripts using earlier syntax. Only small risk is allowing overwrites vs triggering warning. Maybe have an `--overwrite` flag to enable but keep False by default?

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.