Incremental builds: guarantee clean-build equivalence

Summary

Make incremental builds produce the same output file set and contents as a clean build, while retaining a useful edit-refresh speedup.

Gojekyll already has an incremental path: direct source changes can reload and write one document, while configuration, data, include, layout, and Sass changes force a full reload. The remaining work is therefore correctness and dependency tracking, not merely skipping Clean() or comparing mtimes.

Correctness contract

For every supported change, an incremental build followed by reading _site must be equivalent to a clean build from the same source tree:

• identical output paths
• identical file contents
• no stale output from deleted, renamed, unpublished, or re-permalinked documents

When the implementation cannot prove which outputs depend on a change, it must fall back to a full rebuild.

Known dependency classes

Source and output identity

• new, deleted, and renamed documents
• front matter that changes published, collection membership, date, categories, or permalink
• a document whose previous output path must be removed
• static-file additions, deletions, and renames

Cross-document template dependencies

A page can change even when its own source does not. Examples include templates that read:

• site.pages, site.posts, collections, tags, or categories
• excerpts or metadata from another document
• site.static_files
• computed plugin values derived from the complete site

Until these reads are tracked, content changes that can affect aggregate values need conservative invalidation.

Shared sources

• layouts and nested layouts
• includes, including dynamically selected includes
• data and configuration files
• theme layouts, includes, assets, and configuration
• Sass partials and other generated assets

Implementation direction

1. Add a differential test harness that applies a filesystem change, runs an incremental rebuild, runs a clean build in a separate destination, and compares the complete output trees.
2. Cover additions, edits, deletions, renames, publication/permalink changes, aggregate pages, shared templates, data, static files, and theme sources.
3. Introduce explicit dependency records for document inputs and cross-document/site-drop reads where practical.
4. Use conservative full rebuilds for untracked or dynamic dependencies.
5. Remove stale outputs as part of the same transaction that updates the dependency state.
6. Benchmark both the common single-document case and conservative fallbacks on the generated benchmark site.

Acceptance criteria

• the differential suite demonstrates clean-build equivalence for every change class above
• no known edit can leave a stale or missing output file
• unsupported/dynamic dependency cases explicitly fall back to a full rebuild
• benchmark results show the retained speedup and identify which cases intentionally rebuild broadly

Write-if-changed output and batched directory creation may be useful complementary optimizations, but they do not substitute for this correctness contract.

Start by tracing Gojekyll’s existing incremental path and Clean() handling. Build the differential harness described in the issue, applying filesystem changes and comparing incremental output with a separate clean destination. Done means equivalent output trees for the listed change classes, conservative full rebuilds for unsupported dependencies, stale-output removal, and benchmark results.