1jehuang / 1jehuang/jcode

Repository is 440 MB due to large binary media files and build artifacts in git history

Open
#857 1 comment 2 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

autonomous: no tech-debt triage: needs-decision
Dominant language
Rust
Stars
19.9k
Forks
2.3k
Avg merge
2d 7h
Merged PRs (30d)
30

Description

I noticed that just cloning this repo is about 400mb, but most of that is wasted space. Leaving the issue below here as a summary of why that is and some proposed remediation steps.


Summary

Cloning the jcode repository downloads roughly 300–440 MB, even though the actual working tree is quite small. The bloat lives almost entirely inside .git history: the pack file is 441 MB while the checked-out code is only a few MB.

The cause: Git's history contains large binary files (demo videos, GIFs, and accidentally-committed iOS/Swift build artifacts). Binary files can't be delta-compressed or deduplicated by Git, so every version of each file is stored in full, forever.

Breakdown

Of the 441 MB pack, 430 MB comes from just 42 blobs ≥ 1 MB:

Category Size
Demo videos / GIFs / screenshots / fonts (assets/, docs/screenshots, crates/) 357 MB
iOS build artifacts (.pcm, .dSYM, .xctest, .swiftinterface) 55 MB
Root-level .mp4s and misc 17 MB
The single biggest offenders (top 10 blobs = 346 MB)
Path in git history Size
assets/readme/jcode-performance-demonstration.gif (v1) 90.7 MB
assets/demo.mp4 87.0 MB
assets/readme/100-sessions-spawn-demo.gif 57.1 MB
assets/readme/jcode-performance-demonstration.gif (v2) 27.3 MB
assets/demos/jcode-claudeai-demo.mp4 25.4 MB
assets/demos/exports/memory_demo_1m40_spedup.mp4 19.2 MB
assets/demos/exports/memory_demo_1m40_spedup_v2.mp4 16.8 MB
jcode_demo.mp4 (repo root) 16.7 MB
assets/demos/jcode_wolf_demo_final.mp4 11.8 MB
assets/demos/jcode_wolf_demo_v2.mp4 11.1 MB
Why it's so large
  1. GIFs are the worst. jcode-performance-demonstration.gif exists as two different versions (90.7 MB + 27.3 MB ≈ 118
    MB). A 90 MB GIF is extremely wasteful and incompressible.

  2. Large uncompressed MP4s committed directly (87 MB demo.mp4, 25 MB jcode-claudeai-demo.mp4, etc.) rather than
    re-encoded to a web format.

  3. iOS/Swift build output was committed. ios/.build/... contains module-cache .pcm files (5–7 MB each), .dSYM, and
    .xctest binaries. These are no longer tracked in the current tree (0 files remain), but they're still baked into
    history (55 MB).

  4. Iterative demo files. Multiple near-duplicate videos were committed separately: memory_demo_1m40_spedup.mp4 + _v2,
    jcode_wolf_demo_v2 + _final, several jcode_mermaid_demo* versions, and a duplicate jcode_demo.mp4 at the repo root.

  5. Renames don't free space. Moves like assets/demo.mp4 → assets/readme/... leave the old binary in history via
    earlier commits.

Proposed remediation steps

Immediate (no history rewrite, going forward):

  1. Add the build artifacts to .gitignore and git rm --cached -r them so they stop being committed:
    ios/.build/
    *.dSYM
    *.xctest
    *.pcm
    *.swiftinterface
  2. Stop committing large binary media. Do not add new .mp4/.gif files (especially multi-MB ones) to the repo. Serve
    repo/demo media from a releaser/LFS/GitHub Release asset instead.
  3. Compress media before adding. Convert demo GIFs → compressed WebM/AVIF or downscaled GIF, and re-encode MP4s (e.g.
    ffmpeg to H.264 + smaller resolution).

To actually reclaim the 400 MB (rewrites history — coordinate with collaborators):

  1. Purge the big binary blobs from history using git filter-repo (or git filter-branch/BFG). Example:
    # remove specific paths from all history
    git filter-repo --invert-paths --path-glob 'assets/**/*.mp4' \
        --path-glob 'assets/**/*.gif' \
        --path-glob 'ios/.build/**' \
        --path 'demo.mp4'
    
  2. Re-add only the media you actually want, now compressed/optimized.
  3. Run garbage collection to shrink the pack:
    git reflog expire --expire=now --all
    git gc --prune=now --aggressive
  4. Force-push to the remote and ask all collaborators to re-clone or hard-reset (old SHAs will be invalidated).

Note: Steps 4–7 rewrite all commit SHAs, so they should only be done deliberately, ideally on a clean branch/announcement, especially if there are open PRs or other forks.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reviewing the repository history and the listed paths, including assets/, ios/.build/, root-level media, and the proposed .gitignore entries. Use the described git filter-repo and garbage-collection steps only after coordinating the history rewrite; done means build artifacts and oversized media are addressed without leaving the repository's history bloated.

Written by the indexing model from the issue text.

Assessment

Tech stack
git
Domain
tooling
Issue type
Refactor
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.