apache / apache/superset

feat(db_engine_specs): 12 engine specs carry no metadata, so lint_metadata reports them incomplete

Open
#42,980 2 comments 0 reactions 0 assignees View on GitHub
enhancement:request
Dominant language
Python
Stars
74.8k
Forks
18.3k
Avg merge
2d 5h
Merged PRs (30d)
685

Description

### Bug description

`superset/db_engine_specs/lint_metadata.py` exists to verify per-spec metadata completeness. Running it against `master` today:

```
$ python superset/db_engine_specs/lint_metadata.py --json
{"summary": {"total": 80, "with_metadata": 68, "all_required": 68, "average_score": 73.7}}
```

Twelve specs report `has_metadata: false`:

| module | engine |
| --- | --- |
| `aurora` | Aurora MySQL |
| `aurora` | Aurora MySQL (Data API) |
| `aurora` | Aurora PostgreSQL |
| `aurora` | Aurora PostgreSQL (Data API) |
| `kusto` | Azure Data Explorer (KQL) |
| `clickhouse` | ClickHouse (sqlalchemy) |
| `databricks` | Databricks (legacy) |
| `databricks` | Databricks Interactive Cluster |
| `databricks` | Databricks SQL Endpoint |
| `ibmi` | IBM Db2 for i |
| `odps` | ODPS (MaxCompute) |
| `elasticsearch` | OpenSearch (OpenDistro) |

The metadata drives what users see when picking a database, so a spec with none is one a user gets less help connecting to.

### Why this is a tractable target

The verification tool already exists, so each change is objectively checkable — a reviewer runs the linter rather than adjudicating taste. It parallelises at one spec per change, each small enough to review quickly.

### Open question worth settling first

Several of these are abstract base classes (`ClickHouseBaseEngineSpec`, `DatabricksBaseEngineSpec`, `PrestoBaseEngineSpec`, `DatabendBaseEngineSpec`). It is not obvious whether `lint_metadata.py` should exempt base classes or whether they should carry metadata like any other spec. Worth deciding before filling anything in — and if exempting, encoding the exemption in the linter rather than in a comment.

### Acceptance

- `lint_metadata.py --strict` passes for each spec touched.
- Metadata reflects each engine's actual capabilities, verified against its driver documentation rather than copied from a neighbouring spec.

Verified against `master` at `3539c41dab`.

Contributor guide

Open the contributing guide

Research direction

Start with superset/db_engine_specs/lint_metadata.py and run `python superset/db_engine_specs/lint_metadata.py --json` to inspect the twelve incomplete specs listed in the issue. Decide whether abstract base classes should be exempted, then review each engine's driver documentation before adding or defining metadata. Done means `lint_metadata.py --strict` passes for every touched spec and the metadata reflects actual capabilities.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
55/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.