pytorch / pytorch/pytorch.github.io

Repo is too big

Open
#825 0 comments 2 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
HTML
Stars
285
Forks
317
PR merge metrics
No merged PRs in 30d

Description

This repo has a super long checkout time since it has a ton of history. A regular checkout git clone https://github.com/pytorch/pytorch.github.io.git takes 15 minutes on a devbig. A shallow checkout with --depth=1 takes only 1 minute

We have some options for fixing:

  1. Delete the history. I don't use this repo that often so I'm not sure how useful it is. We can also delete it in a smarter way, such as only deleting history from commits 100-N, but keep commits 1-100, or keep every 10th commit or something. Deleting everything (rm -rf .git; git init; git add .; git c -m"test") shrinks the .git folder from 4 GB to 300 MB on my machine, so this is a viable fix if we don't care about the history at all. We could also move the history to a non-default branch (site-old or something) and start fresh. We'd have to do this regularly though since we're not actually fixing the root cause.
  2. Split out the big parts: the entire docs/ folder lives in pytorch/pytorch. We could move that to its own repo (and maybe its own subdomain, with pytorch.org/docs redirecting there). This would keep the fast moving auto generated parts away from the main repo and enable faster development on blog posts/etc since they don't have to deal with the docs folder.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by comparing the repository's regular clone with the reported --depth=1 checkout and inspect the size of .git. Review the two proposed approaches: rewriting or relocating history, and splitting the docs/ folder from pytorch/pytorch. The issue does not select an approach, so done is not defined until the desired repository structure and history policy are decided.

Written by the indexing model from the issue text.

Assessment

Tech stack
git
Domain
devops
Issue type
Refactor
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.