Audit: files too large for agents, tooling, and GitHub UI
- Dominant language
- C#
- Stars
- 18.3k
- Forks
- 5.6k
- PR merge metrics
- PR metrics pending
Description
## Summary
Three distinct problems — agent/LLM token cost, GitHub tool interaction, and GitHub UI/search — all stem from the same root cause: files that have grown without a size budget. This issue tracks candidates for splitting and flags files that break specific documented thresholds.
Audit methodology: `git ls-tree -r -l HEAD` for byte sizes, `git show | split` for per-file line and max-line-length, `git log --oneline --since=2.years` for 2-year commit counts. No large files were opened whole; the audit itself was non-destructive.
---
## Confirmed thresholds
| Limit | Value | Source |
|-------|-------|--------|
| `view` tool truncation | **20 KB** | Tool documentation |
| Agent token cost | morph.cpp (~585 KB) ≈ 137k tokens ≈ **200 AI credits** per full read | Measured in this repo (tiktoken/o200k_base) |
| Code search file exclusion | **> 350 KiB** | [docs.github.com](https://docs.github.com/en/search-github/github-code-search/about-github-code-search) |
| Code search line truncation | **Lines > 1,024 chars** | Same |
| Code search full exclusion | **Files with > 1 line over 4,096 bytes** | Same |
| Per-PR diff | > 20,000 lines **or** > 1 MB | [docs.github.com](https://docs.github.com/en/repositories/creating-and-managing-repositories/repository-limits) |
| Per-file diff | > 20,000 lines **or** > **500 KB** not shown; 400 lines / 20 KB auto-loaded | Same |
| Blob view syntax highlighting | **> ~1 MB** (unverified — no official docs page found) | Community reports |
**160 files** in the repo currently exceed the 350 KiB code search exclusion.
---
## Native (JIT core)
Files are hand-maintained, in active development, and exceed the code search exclusion and/or the per-file diff limit.
| Path | Size (KB) | Lines | 2y commits | Exceeds |
|------|-----------|-------|------------|---------|
| `src/coreclr/jit/gentree.cpp` | 1,210 | 36,981 | **289** | All limits; ~282k tokens, context-window overrun |
| `src/coreclr/jit/compiler.h` | 524 | 13,724 | **392** | Code search, diff; highest-churn file in repo |
| `src/coreclr/jit/emitxarch.cpp` | 637 | 20,974 | 82 | Code search, diff; at the 20k-line diff boundary |
| `src/coreclr/jit/emitarm64sve.cpp` | 811 | 20,030 | 23 | Code search, diff; 20k-line boundary |
| `src/coreclr/jit/morph.cpp` | 572 | 16,231 | 190 | Code search, diff |
| `src/coreclr/jit/valuenum.cpp` | 548 | 16,149 | 115 | Code search, diff |
| `src/coreclr/jit/lsra.cpp` | 557 | 14,024 | 46 | Code search, diff |
| `src/coreclr/jit/importer.cpp` | 563 | 14,788 | 129 | Code search, diff |
| `src/coreclr/jit/importercalls.cpp` | 502 | 12,725 | 158 | Code search, diff |
| `src/coreclr/jit/lower.cpp` | 479 | 13,249 | 177 | Code search |
| `src/coreclr/jit/compiler.cpp` | 362 | 10,851 | 169 | Code search |
| `src/coreclr/jit/codegenxarch.cpp` | 428 | 11,645 | 104 | Code search |
| `src/coreclr/jit/lowerxarch.cpp` | 412 | 10,919 | 103 | Code search |
| `src/coreclr/jit/instrsxarch.h` | 439 | 1,411 | 42 | Code search; 1,411 lines in 439 KB (avg 318 bytes/line) |
| `src/coreclr/jit/emit.cpp` | 359 | 10,922 | 65 | Code search |
| `src/coreclr/jit/gentree.h` | 306 | 10,700 | 122 | Code search |
## Native (VM / Debug / Mono)
| Path | Size (KB) | Lines | 2y commits | Exceeds |
|------|-----------|-------|------------|---------|
| `src/coreclr/vm/jitinterface.cpp` | 521 | 15,982 | **188** | Code search, diff |
| `src/coreclr/vm/methodtablebuilder.cpp` | 520 | 13,172 | 53 | Code search, diff |
| `src/coreclr/vm/excep.cpp` | 376 | 10,760 | 82 | Code search |
| `src/coreclr/vm/proftoeeinterfaceimpl.cpp` | 324 | 11,093 | 19 | Code search |
| `src/coreclr/debug/ee/debugger.cpp` | 529 | 16,018 | 65 | Code search, diff |
| `src/coreclr/debug/di/process.cpp` | 529 | 14,793 | 50 | Code search, diff |
| `src/coreclr/debug/di/rspriv.h` | 421 | 11,634 | 41 | Code search |
| `src/coreclr/tools/superpmi/superpmi-shared/methodcontext.cpp` | 320 | 7,837 | 59 | Code search |
| `src/mono/mono/mini/mini-llvm.c` | 510 | 15,532 | 16 | Code search, diff |
| `src/mono/mono/mini/aot-compiler.c` | 498 | 15,976 | 12 | Code search |
| `src/mono/mono/mini/method-to-ir.c` | 452 | 14,048 | 12 | Code search |
## Managed
| Path | Size (KB) | Lines | Max line (chars) | 2y commits | Exceeds |
|------|-----------|-------|-----------------|------------|---------|
| `src/libraries/System.Runtime.Intrinsics/ref/System.Runtime.Intrinsics.cs` | 2,059 | 11,987 | 781 | 84 | Code search; ~490k tokens; unverified blob-view limit |
| `src/libraries/System.Runtime/ref/System.Runtime.cs` | 1,571 | 18,273 | **2,652** | 111 | Code search; max line trips 1,024-char search truncation; ~374k tokens |
| `src/libraries/System.Private.CoreLib/src/System/Runtime/Intrinsics/Arm/AdvSimd.cs` | 975 | 16,318 | 355 | 4 | Code search; ~233k tokens |
| `src/libraries/System.Private.CoreLib/src/System/Threading/Tasks/Task.cs` | 387 | 7,759 | 233 | 30 | Code search |
| `src/libraries/System.Private.Xml/src/System/Xml/Core/XmlTextReaderImpl.cs` | 353 | 9,751 | 185 | 3 | Code search |
| `src/libraries/System.Text.RegularExpressions/gen/RegexGenerator.Emitter.cs` | 327 | 5,902 | 259 | 24 | Code search |
| `src/libraries/System.Linq.Parallel/src/System/Linq/ParallelEnumerable.cs` | 322 | 5,967 | 215 | 5 | Code search |
| `src/libraries/System.Private.CoreLib/src/System/MemoryExtensions.cs` | 338 | 6,473 | 262 | 19 | Code search |
## `.github` instruction / skill files
A separate class of problem: these files are loaded as context by LLM agents. The 20 KB `view` tool truncation applies to files an agent reads at runtime (build config, KB docs, referenced instruction files). **Empirical testing confirmed that SKILL.md files are injected in full by the skill loader — not via `view` — so their content is not silently truncated on invocation.** Large skill files still consume more context budget per use, and any secondary files those skills reference at runtime (e.g. `pr-assessment.md`, `.github/instructions/*.md`) remain subject to the 20 KB truncation.
| File | Size | Lines | Problem |
|------|------|-------|---------|
| `.github/workflows/ci-failure-scan-feedback.lock.yml` | 126 KB | — | Auto-generated `gh-aw` lockfile; looks like a real workflow to glob, contains only SHAs and hashes |
| `.github/workflows/holistic-review.lock.yml` | 121 KB | — | Same |
| `.github/workflows/ci-failure-fix.lock.yml` | 119 KB | — | Same |
| `.github/workflows/closed-issue-reference-check.lock.yml` | 118 KB | — | Same |
| `.github/workflows/ci-failure-scan.lock.yml` | 114 KB | — | Same |
| `.github/workflows/breaking-change-doc.lock.yml` | 112 KB | — | Same |
| `.github/workflows/holistic-review-orchestrator.yml` | 34.9 KB | 736 | Real workflow; 1.75x view truncation limit; truncated when read at runtime |
| `.github/workflows/ci-failure-scan.md` | 32.4 KB | 409 | Knowledge-base doc; truncated when read at runtime |
| `Directory.Build.props` | 32 KB | 405 | Read via `view` when investigating any build issue; always truncated |
| `.github/workflows/ci-failure-fix.md` | 29.1 KB | 415 | Truncated when read at runtime |
| `.github/skills/issue-triage/SKILL.md` | 28.2 KB | 487 | Injected in full on invocation (empirically verified — all 487 lines present); 28 KB context cost per use |
| `.github/agents/system-net-review.agent.md` | 26 KB | 427 | Agent definition; loading mechanism not empirically tested |
| `.github/workflows/ci-failure-scan-feedback.md` | 27.2 KB | 349 | Truncated when read at runtime |
| `.github/skills/code-review/SKILL.md` | 22.6 KB | 203 | Injected in full on invocation (empirically verified — all 203 lines present); 22 KB context cost per use |
| `.github/skills/api-proposal/SKILL.md` | 22.4 KB | 389 | Expected to inject in full (same mechanism); context cost per use |
| `.github/workflows/shared/create-kbe.instructions.md` | 22.1 KB | 522 | Truncated when read at runtime |
| `.github/agents/extensions-reviewer.agent.md` | 20.2 KB | 245 | Agent definition; loading mechanism not empirically tested |
The six `.lock.yml` files are the most acute: each is ~120 KB of binary-encoded metadata that an agent cannot reason about, and all six appear in any `*.yml` glob of `.github/workflows/`.
## Generated / by-design (not action items)
- `src/coreclr/pal/prebuilt/inc/corprof.h` / `cordebug.h` / `xclrdata.h` — MIDL-generated ("this ALWAYS GENERATED file")
- `src/coreclr/ilasm/prebuilt/asmparse.cpp` — Bison-generated
- `src/coreclr/debug/ee/amd64/amd64InstrDecode.h` — machine-generated, see `gen_amd64InstrDecode/README.md`
- `src/tools/ilasm/src/ILAssembler/gen/CILParser.cs` — ``
- `src/tests/JIT/jit64/opt/cse/hugeexpr1.cs` and siblings — intentionally enormous CSE stress tests
- `src/native/external/**` — vendored external code (brotli, zlib-ng, zstd, valgrind)
- `*.PlatformNotSupported.cs` intrinsics stubs — maintained alongside their real counterparts by design
---
## Candidate work items (priority order)
1. **Split `src/coreclr/jit/gentree.cpp`** — 1.2 MB / 37k lines / 289 commits/2y. One full read exceeds the 128k-token context of most models. Natural seams: node construction, printing, `fgMorphXxx` helpers, flag management, assertion logic.
2. **Decompose `src/coreclr/jit/compiler.h`** — 524 KB / 13.7k lines / **392 commits/2y** (highest churn in repo). Included by every JIT translation unit. Split by phase state, liveness helpers, assertion structs, opt flags.
3. **Split importer pipeline** — `importer.cpp` (563 KB / 129 commits) + `importercalls.cpp` (502 KB / 158 commits). Both exceed code search and diff limits. Further split by opcode category (arithmetic/compare, memory, control flow, calls).
4. **Address ref assembly size/line-length** — `System.Runtime.cs` (1.57 MB, max line 2,652 chars trips code search 1,024-char truncation). `System.Runtime.Intrinsics.cs` (2.05 MB, ~490k tokens). Whether split by logical area or otherwise is an api-review-process question.
5. **Split `src/coreclr/vm/jitinterface.cpp`** — 521 KB / 16k lines / 188 commits/2y. Second-highest-churn VM file; every meaningful change produces a partially-hidden diff. Natural split: helper implementations, EE callbacks for type/method queries, JIT code manager helpers.
6. **Split `src/coreclr/jit/morph.cpp` + `lower.cpp`** — `morph.cpp` (572 KB / 190 commits) is the calibration file at ~200 AI credits per read. `lower.cpp` (479 KB / 177 commits) already has a precedent split with `lowerxarch.cpp`. Both exceed code search and diff limits.
> [!NOTE]
> This issue was researched and drafted with GitHub Copilot (Claude Sonnet 4.6). The analysis methodology, threshold verification, and prioritization were performed autonomously from `git` metadata without opening large files. Thresholds were confirmed against official documentation where available; the blob-view rendering limit (~1 MB) is noted as unverified.
Contributor guide
Research direction
Start by reviewing the prioritized candidate, such as src/coreclr/jit/gentree.cpp, and use the listed natural seams plus git log --oneline --since=2.years to understand its structure and churn. The work is done when a selected oversized hand-maintained file is split along a defensible boundary without changing behavior, while reducing the documented size and diff/search problems.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- c, cpp, csharp
- Domain
- developer-experience, tooling
- Issue type
- Refactor
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100