apache / apache/superset

superset-extensions bundle fails: CLI's unbounded apache-superset-core resolves to stale PyPI 0.1.0

Open
#42,568 1 comment 0 reactions 0 assignees View on GitHub
dependencies:python
Dominant language
Python
Stars
74.8k
Forks
18.3k
Avg merge
2d 5h
Merged PRs (30d)
685

Description

### Bug description

`superset-extensions bundle` fails for **any** extension when the CLI is installed from a source checkout with `pip`, producing no `.supx` artifact:

```
File ".../superset_extensions_cli/cli.py", line 172, in build_manifest
return Manifest(
pydantic_core._pydantic_core.ValidationError: 1 validation error for Manifest
id
Field required [type=missing, input_value={'publisher': '...', ...}]
```

**Root cause — a version skew between the CLI and `apache-superset-core`:**

- #39171 changed `Manifest.id` from a required input field to a `@computed_field` on `BaseExtension`, and correctly updated `build_manifest()` to stop passing `id`.
- `superset-extensions-cli/pyproject.toml` declares its dependency as bare `apache-superset-core`, with the comment *"no bounds for apache-superset-core until we have a stable version"*.
- PyPI still serves `apache-superset-core` **0.1.0**, published 2026-05-11 — two months before that change — where `id` is still a required input field.

So a `pip install ./superset-extensions-cli` gets **new CLI code against stale core**, and `build_manifest()` omits a field that the installed core still requires.

### How to reproduce

From a clean `apache/superset` checkout at master:

```bash
python3 -m venv /tmp/v
/tmp/v/bin/pip install ./superset-extensions-cli
/tmp/v/bin/python -c "from superset_core.extensions.types import Manifest; print('id' in Manifest.model_fields)"
```

Prints `True` — pip resolved 0.1.0 from PyPI, where `id` is a required input field. At master it is computed, so this prints `False`. Running `superset-extensions bundle` in that venv then fails as above.

### On the `[tool.uv.sources]` workaround

`pyproject.toml` already redirects both packages to local editable paths:

```toml
[tool.uv.sources]
apache-superset-core = { path = "./superset-core", editable = true }
apache-superset-extensions-cli = { path = "./superset-extensions-cli", editable = true }
```

That is the right mechanism, but it does not appear to apply to `uv pip`. On **uv 0.9.21**, from clean venvs, all three of these still resolved `apache-superset-core` 0.1.0 from PyPI rather than linking `./superset-core`:

| cwd | command | result |
|---|---|---|
| repo root | `uv pip install -e ./superset-extensions-cli` | PyPI 0.1.0 |
| repo root | `uv pip install ./superset-extensions-cli` | PyPI 0.1.0 |
| outside repo | `uv pip install /superset-extensions-cli` | PyPI 0.1.0 |

In the editable case, core landed in `site-packages` rather than as a link to `./superset-core`. This matches uv's documented split — `tool.uv.sources` is honoured by the project interface (`uv sync`, `uv run`, `uv add`), not by the pip-compatible `uv pip` interface. `uv sync` from the repo root very likely does work; if a `uv pip install -e` appeared to work, it may have been in a venv that had already been synced.

### Additional context

Two cases the workspace redirect cannot cover, which is why a bound or a release probably matters:

1. **Consumers outside this repo.** We build extensions in a separate repository that vendors Superset as a submodule and installs the CLI from `./superset-oss/superset-extensions-cli`. A `tool.uv.sources` entry in Superset's root `pyproject.toml` never applies to an install driven from another project, regardless of pip vs uv.
2. **Anyone installing the published CLI from PyPI**, who gets the same stale core with no checkout to redirect to.

We have worked around it downstream by installing `superset-core` and the CLI from the submodule together, which is version-manager-agnostic:

```bash
pip install ./superset-oss/superset-core ./superset-oss/superset-extensions-cli
```

Possible fixes upstream, in rough order of durability:

- Publish an `apache-superset-core` release matching current master, so the unbounded dependency resolves to something compatible.
- Add a lower bound (`apache-superset-core>=`) at release-cut time, so a stale core fails resolution rather than failing at runtime with a schema error.
- Optionally, have `build_manifest()` fail with a clearer message when the installed core's `Manifest` disagrees with what the CLI passes — the current pydantic error gives no hint that two packages are out of step.

Happy to send a PR for whichever shape you prefer.

### Superset version

master / latest-dev

### Python version

3.11

### Node version

I don't know

### Browser

Not applicable

Contributor guide

Open the contributing guide

Research direction

Inspect superset-extensions-cli/pyproject.toml and superset_extensions_cli/cli.py, especially build_manifest(). Reproduce the clean pip installation and run superset-extensions bundle, comparing the installed apache-superset-core version with the checkout. Done means the dependency resolution is compatible and bundling produces a .supx artifact without the Manifest validation error.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
build-system, cli
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.