libgit2 / libgit2/libgit2sharp

Parallel calls to Repository.ObjectDatabase.CreateBlob with identical contents fails.

未關閉
#1,122 2 則留言 0 個 reaction 已指派 0 人 在 GitHub 檢視

還沒有人認領這個 Issue。

主要語言
C#
星號
3.5k
分支
925
PR 合併指標
30 天內沒有已合併 PR

描述

I would like to insert blobs from memory into an object database in parallel (based on the technique described at http://stackoverflow.com/a/16244234). This works fine most of the time, but is subject to race conditions if the contents are the same. There are two classes of errors I observe:
1. Exception due to FILE_EXISTS conflict when trying to create the directory with the first two characters of the OID under objects.
2. NULL blob returned from CreateBlob.

Here's some example code to reproduce the problem:

using System;
using System.Text;
using System.IO;
using System.Threading.Tasks;
using LibGit2Sharp;

namespace GitODBStress
{
    class Program
    {
        static void Main()
        {

            string repoPath = Path.Combine(Path.GetTempPath(), "Git_Parallel_Odb");
            int iterationCount = 100;

            if (!Directory.Exists(repoPath))
            {
                Directory.CreateDirectory(repoPath);
            }

            var path = Repository.Init(repoPath, isBare:true);
            Console.WriteLine("Made repo at {0}", path);

            var repo = new Repository(path);

            var tasks = new Task[2];
            for (int i = 0; i < iterationCount; ++i)
            {
                var iter = i;
                byte[] buf = Encoding.ASCII.GetBytes("foo" + iter.ToString());
                for (int taskNo = 0; taskNo < 2; ++taskNo)
                {
                    tasks[taskNo] = Task.Factory.StartNew(() =>
                        {
                            MemoryStream ms = new MemoryStream(buf);
                            for (; ; )
                            {
                                try
                                {
                                    var blob = repo.ObjectDatabase.CreateBlob(ms);
                                    if (blob == null)
                                    {
                                        Console.WriteLine("ERROR: NULL blob on iteration {0}", iter);
                                        continue;
                                    }

                                    break;
                                }
                                catch (LibGit2SharpException e)
                                {
                                    Console.WriteLine("ERROR: Caught LibGit2SharpException on iteration {0}: {1}", iter, e.Message);
                                    if (e.Message.Contains("Cannot create a file when that file already exists."))
                                    {
                                        continue;
                                    }
                                }
                            }
                        });
                }

                Task.WaitAll(tasks);
            }

            repo.Dispose();

            // clean up temp repo files
            var files = Directory.GetFiles(repoPath, "*.*", SearchOption.AllDirectories);
            foreach (var f in files)
            {
                var attrib = File.GetAttributes(f);
                if (attrib.HasFlag(FileAttributes.ReadOnly))
                {
                    File.SetAttributes(f, attrib & ~FileAttributes.ReadOnly);
                }
            }

            Directory.Delete(repoPath, true);
        }
    }
}

Thanks!

貢獻指南

開啟貢獻指南

從這裡開始

  1. 先讀完整個 Issue,再讀專案的貢獻指南。
  2. 在 Issue 下留言說明你要接手 —— 這能避免兩個人做同樣的事。
  3. Fork 儲存庫,在一個分支上完成修改。
  4. 送出 Pull Request,並在描述裡引用這個 Issue 編號。

研究方向

從 Repository.ObjectDatabase.CreateBlob 開始,針對 bare repository 執行 C# 重現。追蹤相同內容的平行路徑,以及回報的 FILE_EXISTS 和 null-blob 結果。並行呼叫在建立預期 blob 的同時完成,且不出現任一失敗,即表示完成。

由索引模型根據 Issue 內容生成。

評估

技術堆疊
csharp, git
領域
databases
Issue 類型
缺陷
難度
4/5
預估耗時
3-5 天
活躍度
停滯
描述清晰度
基本清楚
新手友好度
35/100

把新 issue 寄到你的電子郵件信箱

精選適合新手參與的 GitHub issue 摘要。