libgit2 / libgit2/pygit2

repo.walk in thread hangs the main thread

Open
#1,003 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

support
Dominant language
Python
Stars
1.7k
Forks
408
Avg merge
2d 57m
Merged PRs (30d)
7

Description

When calling repo.walk in a new thread, it blocks the main thread until it finished!!!

See the example code below, when run with a huge repo (qt5, chromium e.g.), the main thread won't print any message until the repo.walk ended. (Use iter also the same)

It seems that repo.diff also have the problem.

from pygit2 import Repository, GIT_SORT_TOPOLOGICAL
from threading import Thread

import sys
import time


def thread_func(repo_dir):
    repo = Repository(repo_dir)

    print(">>>>>>>> begin diff")
    commits = list(repo.walk(repo.head.target, GIT_SORT_TOPOLOGICAL))
    #for commit in repo.walk(repo.head.target, GIT_SORT_TOPOLOGICAL):
    #    continue
    print(">>>>>>>> end diff")


def test(repo_dir):
    t = Thread(target=thread_func, args=[repo_dir])
    t.start()

    while t.is_alive():
        print("main thread...")
        time.sleep(0.01)


if __name__ == "__main__":
    if len(sys.argv) != 2:
        print(">>>>>>>> Invalid argument")
        sys.exit(-1)

    test(sys.argv[1])

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the provided Python reproducer and inspect the implementations of repo.walk, its iterator, and repo.diff for threading behavior. Verify whether the main thread remains responsive while the worker processes a large repository, then add regression coverage demonstrating the expected behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
git, python
Domain
backend
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.