apache / apache/airflow

Remove check_flit_worktree_compatibility once upstream flit fix ships

Open
#65,772 0 comments 0 reactions 1 assignee Claimed by @potiuk View on GitHub
area:dev-env
Dominant language
Python
Stars
46.9k
Forks
17.8k
Avg merge
2d 9h
Merged PRs (30d)
472

Description

## Background

PR #65771 adds `check_flit_worktree_compatibility` to `dev/breeze/src/airflow_breeze/prepare_providers/provider_distributions.py`. The check refuses to build provider sdists when the source tree is a git worktree, because two related things go wrong otherwise and no warning is emitted:

1. **flit worktree detection bug.** `flit.vcs.identify_vcs` only checks `.git.is_dir()`, missing the `.git`-as-file pointer that git uses for worktrees and submodules. flit silently falls back to a non-VCS sdist that omits tracked files (`docs/`, `tests/`, `provider.yaml`, anything declared in `[tool.flit.sdist] include`), and the resulting packages fail reproducibility checks against the released sdists on dist.apache.org.
2. **Breeze Docker-mount case.** Even once flit is fixed, when Breeze runs the build inside Docker only the worktree directory is bind-mounted — the absolute `gitdir:` path baked into the worktree's `.git` file references the main repo's `.git/worktrees/` directory on the host, which is not reachable from the container. `git ls-files` then fails or misbehaves.

Upstream flit tracking:

- Issue: https://github.com/pypa/flit/issues/798
- PR: https://github.com/pypa/flit/pull/799

## What to do when this issue is actioned

Once **both** of the following hold:

- [ ] The flit fix (pypa/flit#799) is merged and released.
- [ ] Airflow's constraints pin to a flit release that contains the fix, **and** the Breeze Docker image is rebuilt to include that version.

…and the Docker-mount case has been addressed (either by mounting the main repo's `.git/worktrees/` directory alongside the worktree, or by rewriting the `gitdir:` pointer when sources are staged into the container, or by deciding the Docker-mount path is not supported from worktrees), remove:

- `check_flit_worktree_compatibility` in `dev/breeze/src/airflow_breeze/prepare_providers/provider_distributions.py`
- Its call site in `dev/breeze/src/airflow_breeze/commands/release_management_commands.py`
- Its tests in `dev/breeze/tests/test_provider_distributions.py`

If only the flit fix lands but the Docker-mount case is still broken, the check must stay (or be narrowed to the Docker case only) — the silent-incomplete-sdist failure is severe enough that we should not let a release train run into it unguarded.

## Acceptance criteria

- [ ] Provider sdists built from a `git worktree add ...` directory via `breeze release-management prepare-provider-distributions --distribution-format sdist` match byte-for-byte with sdists built from a plain `git checkout` of the same commit.
- [ ] The same command run inside Breeze's Docker container from a worktree either succeeds with matching sdists or produces a clear, explicit error (not a silent incomplete tarball).
- [ ] `check_flit_worktree_compatibility` removed and PR #65771's test coverage deleted along with it.

## Related

- PR #65771 — where the workaround lives.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.