apache / apache/airflow

Error cloning repository

Open
#61,716 5 comments 0 reactions 0 assignees View on GitHub
area:core area:providers kind:bug needs-triage provider:git
Dominant language
Python
Stars
46.9k
Forks
17.8k
Avg merge
2d 9h
Merged PRs (30d)
472

Description

### Apache Airflow version

3.1.7

### If "Other Airflow 3 version" selected, which one?

3.1.6

### What happened?

We are observing a sporadic issue (**so I don't know how to reproduce**) with GitDagBundles when cloning DAG from _self-hosted GitLab_ (see log below), and this error obviously is not raising an alert by calling DAG callback, so it is "hidden" in some sense.
```
[2026-02-10 06:05:02] WARNING - Repository clone/open failed, cleaning up and retrying bundle_name=etl-dags-bundle version=09b6... bare_repo_path=PosixPath('/tmp/airflow/dag_bundles/.../bare') repo_path=PosixPath('/tmp/airflow/dag_bundles/.../versions/09b6...') versions_path=PosixPath('/tmp/airflow/dag_bundles/etl-dags-bundle/versions') git_conn_id=git_conn submodules= exc=InvalidGitRepositoryError('/tmp/airflow/dag_bundles/.../versions/09b6...') source=airflow.providers.git.bundles.git loc=git.py:180
[2026-02-10 06:05:02] INFO - Cloning repository bundle_name=... version=09b6... bare_repo_path=PosixPath('/tmp/airflow/dag_bundles/.../bare') repo_path=PosixPath('/tmp/airflow/dag_bundles/.../versions/09b6...') versions_path=PosixPath('/tmp/airflow/dag_bundles/.../versions') git_conn_id=git_conn submodules= source=airflow.providers.git.bundles.git loc=git.py:166
[2026-02-10 06:05:02] WARNING - Repository clone/open failed, cleaning up and retrying bundle_name=... version=09b6... bare_repo_path=PosixPath('/tmp/airflow/dag_bundles/.../bare') repo_path=PosixPath('/tmp/airflow/dag_bundles/.../versions/09b6...') versions_path=PosixPath('/tmp/airflow/dag_bundles/.../versions') git_conn_id=git_conn submodules= exc=GitCommandError(['git', 'clone', '-v', '--', '/tmp/airflow/dag_bundles/.../bare', '/tmp/airflow/dag_bundles/.../versions/09b6...'], 128, b"fatal: destination path '/tmp/airflow/dag_bundles/.../versions/09b6...' already exists and is not an empty directory.\n") source=airflow.providers.git.bundles.git loc=git.py:180
[2026-02-10 06:05:02] ERROR - Top level error source=task loc=task_runner.py:1562
RuntimeError: Error cloning repository
File "/home/airflow/.local/lib/python3.12/site-packages/airflow/sdk/execution_time/task_runner.py", line 1550 in main
File "/home/airflow/.local/lib/python3.12/site-packages/airflow/sdk/execution_time/task_runner.py", line 732 in startup
File "/home/airflow/.local/lib/python3.12/site-packages/airflow/sdk/execution_time/task_runner.py", line 618 in parse
File "/home/airflow/.local/lib/python3.12/site-packages/airflow/providers/git/bundles/git.py", line 155 in initialize
File "/home/airflow/.local/lib/python3.12/site-packages/airflow/providers/git/bundles/git.py", line 126 in _initialize
GitCommandError: Cmd('git') failed due to: exit code(128)
cmdline: git clone -v -- /tmp/airflow/dag_bundles/.../bare /tmp/airflow/dag_bundles/.../versions/09b6...
stderr: 'fatal: destination path '/tmp/airflow/dag_bundles/.../versions/09b6...' already exists and is not an empty directory.
'
File "/home/airflow/.local/lib/python3.12/site-packages/airflow/providers/git/bundles/git.py", line 124 in _initialize
File "/home/airflow/.local/lib/python3.12/site-packages/tenacity/__init__.py", line 338 in wrapped_f
File "/home/airflow/.local/lib/python3.12/site-packages/tenacity/__init__.py", line 477 in __call__
File "/home/airflow/.local/lib/python3.12/site-packages/tenacity/__init__.py", line 378 in iter
File "/home/airflow/.local/lib/python3.12/site-packages/tenacity/__init__.py", line 420 in exc_check
File "/home/airflow/.local/lib/python3.12/site-packages/tenacity/__init__.py", line 187 in reraise
File "/usr/python/lib/python3.12/concurrent/futures/_base.py", line 449 in result
File "/usr/python/lib/python3.12/concurrent/futures/_base.py", line 401 in __get_result
File "/home/airflow/.local/lib/python3.12/site-packages/tenacity/__init__.py", line 480 in __call__
File "/home/airflow/.local/lib/python3.12/site-packages/airflow/providers/git/bundles/git.py", line 169 in _clone_repo_if_required
File "/home/airflow/.local/lib/python3.12/site-packages/git/repo/base.py", line 1538 in clone_from
File "/home/airflow/.local/lib/python3.12/site-packages/git/repo/base.py", line 1409 in _clone
File "/home/airflow/.local/lib/python3.12/site-packages/git/util.py", line 512 in finalize_process
File "/home/airflow/.local/lib/python3.12/site-packages/git/cmd.py", line 419 in wait
[2026-02-10 06:05:02] WARNING - Process exited abnormally exit_code=1 source=task
```

### What you think should happen instead?

I think that in [this Git provider's file](https://github.com/apache/airflow/blob/main/providers/git/src/airflow/providers/git/bundles/git.py), the `@retry` decorator from `tenacity` should be supplemented with parameters such as `wait`, `wait_exponential`, or `wait_fixed` to introduce a small delay between retries. As you can see in the log above, the current retries occur less than 1 second apart, which causes an issue.

### How to reproduce

I do not know.

### Operating System

linux

### Versions of Apache Airflow Providers

git provider v 0.2.0

### Deployment

Official Apache Airflow Helm Chart

### Deployment details

official helm chart 1.18

### Anything else?

_No response_

### Are you willing to submit PR?

- [ ] Yes I am willing to submit a PR!

### Code of Conduct

- [x] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.