apache / apache/airflow

GitDagBundle: promoting a SHA-pinned tracking_ref can fail with 'reference is not a tree' if local storage isn't cleared

Open
#71,388 0 comments 0 reactions 0 assignees View on GitHub
area:dag-bundles area:providers kind:bug provider:git
Dominant language
Python
Stars
46.9k
Forks
17.8k
Avg merge
2d 7h
Merged PRs (30d)
484

Description

### Description

`GitDagBundle` supports pinning `tracking_ref` to a full commit SHA (see #69735). Rolling back to an older SHA works reliably after a config change + Dag processor restart, because the objects are already present in local storage. **Promoting** to a new SHA that was created/pushed after the bundle's local storage was first populated can fail.

### Root cause

In [`GitDagBundle._initialize`](https://github.com/apache/airflow/blob/6af6a6a6bfe216618b0af4171bd5f1b10c83d14e/providers/git/src/airflow/providers/git/bundles/git.py#L153-L239):

- The working clone lives at a stable path (`self.repo_path`, e.g. `.../tracking_repo` when no separate `version` is set), not a fresh path per `tracking_ref` value.
- `_clone_repo_if_required()` only clones if that path doesn't already exist. On a restart where the bundle's storage directory has survived (the default — `dag_bundle_storage_path` defaults to `/tmp/airflow/dag_bundles`), the existing working clone is reopened, not re-fetched.
- Only the **bare mirror** is fetched inside `_clone_bare_repo_if_required` → `_fetch_bare_repo()`. The working clone (`tracking_repo`) is only fetched inside `refresh()`, which runs *after* the `self.repo.git.checkout(self.tracking_ref)` call at line 215 for the no-`version` code path.
- If the new SHA (or a newly created tag) isn't yet an object in the stale working clone, `checkout` raises `GitCommandError: fatal: reference is not a tree: ` (or `pathspec '' did not match` for a new tag).

### Repro

1. Configure a `GitDagBundle` with `tracking_ref` pinned to commit A. Let the Dag processor initialize the bundle (populates `tracking_repo`).
2. Push a new commit B to the source repo, then update the bundle config's `tracking_ref` to commit B's SHA.
3. Restart the Dag processor without clearing the bundle's local storage path.
4. `initialize()` raises `GitCommandError: fatal: reference is not a tree: `.

Rolling back to commit A (already present locally) after this failure succeeds normally. A fresh storage path (new pod, or manually deleted bundle directory) also succeeds for promotion, since `_clone_repo_if_required()` then performs a real clone against the already-updated bare mirror.

### Suggested fix

In `_initialize`, fetch the working clone (or check that the target ref is present and fetch if not) before calling `self.repo.git.checkout(self.tracking_ref)`, mirroring what `refresh()` already does for the branch/tag tracking case.

### Related

- Uncovered during review of #69735, which documents `tracking_ref`'s SHA support and scopes its restart-behavior docs/tests to this limitation until it's fixed here.

Contributor guide

Open the contributing guide

Research direction

Start in providers/git/src/airflow/providers/git/bundles/git.py, focusing on GitDagBundle._initialize and the existing refresh() path. Reproduce promotion from an older pinned SHA to a newly pushed SHA with the bundle storage retained, then add regression coverage for initialization. Done means a new SHA or tag can be checked out after restart without clearing local storage.

Written by the indexing model from the issue text.

Assessment

Tech stack
git, python
Domain
backend, data-engineering
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Clearly specified
Newbie friendliness
76/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.