libgit2 / libgit2/libgit2sharp

repo.Commits.QueryBy(filename) slow on large repos

Open
#1,705 6 comments 2 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
C#
Stars
3.5k
Forks
925
PR merge metrics
No merged PRs in 30d

Description

Reproduction steps

1): Clone a large repo
2): run this function on that repo with some random file:

public IEnumerable<string> TestSlow(string filename)
{
    using (var repo = new Repository(repoRoot))
    {
        string path = filename.Substring(repoRoot.Length + 1).Replace("\\", "/");
        foreach (LogEntry entry in repo.Commits.QueryBy(path))
        {
            yield return entry.Commit.Author.ToString();
        }
    }
}
  1. run this command on the same file: "git log --follow --oneline -- "
Expected behavior

I expect similar time to be taken by TestSlow and the git log command above

Actual behavior

The git log command finishes in about 1.6 ms on my repo
The TestSlow command takes about 70 seconds.

Here is what I see in my profiler:
profiler view

Version of LibGit2Sharp (release number or SHA1)

0.27.0-preview-0017
0.26.1
0.24.1

Operating system(s) tested; .NET runtime tested

.NET Framework 4.7.2 on Windows 10

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing the comparison between TestSlow using repo.Commits.QueryBy(path) and git log --follow --oneline -- <file> on a large repository. Use the reported profiler view and timings to locate where QueryBy spends its time. Done means reducing the large-repository runtime toward the comparable git command while preserving the returned commit entries.

Written by the indexing model from the issue text.

Assessment

Tech stack
csharp, git
Domain
performance
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.