microsoft / microsoft/VFSForGit
PackfileMaintenanceStep should dynamically resize the batch size
Nobody has claimed this yet.
- Dominant language
- C#
- Stars
- 6.1k
- Forks
- 474
- Avg merge
- 2d 4h
- Merged PRs (30d)
- 8
Description
The PackfileMaintenanceStep expires prefetch packs where every object appears in a new pack. It then condenses many packs into one pack using git multi-pack-index repack --batch-size=<size> where "size" is currently "2gb".
We cannot make "2gb" much larger due to issues with unsigned long on Windows. This also limits the amount of work we would do in one step. With this batch size, we anticipate the users with ~100 GB of prefetch data will get to a steady state in 30-60 runs of the step.
Once in a steady state, we anticipate that it will take a long time before the repack command has enough data for a full batch of data.
We can predict if the batch size is too large by examining the pack directory and counting the number of files smaller than 2gb. We could even predict a size that would group all packs into a single pack by adding their sizes and subtracting 1. If this size is larger than 2 * 1024 * 1024 * 1024, then we can stick with the default.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by locating PackfileMaintenanceStep and its use of git multi-pack-index repack --batch-size=2gb. Read how the pack directory is examined and validate the proposed size calculation, including the Windows unsigned-long limit. Done when the step dynamically selects a batch size for smaller packs, retains the default when the calculated size exceeds 2 * 1024 * 1024 * 1024, and regression coverage verifies the behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- csharp, git
- Domain
- tooling
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100