bazel-contrib / bazel-contrib/rules_python

sphinxdocs: implement content-based change detection plugin

Open
#2,879 1 comment 0 reactions 0 assignees View on GitHub
sphinxdocs type: feature request
Dominant language
Starlark
Stars
688
Forks
721
Avg merge
15h 7m
Merged PRs (30d)
76

Description

Sphinx has change detection to facilitate incremental rebuilding, but it's timestamp based. Bazel doesn't reliably preserve timestamps, nor are timestamps highly reliable, so this functionality isn't usable. This means sphinx has to rebuild everything, every time, which can get quite slow. Pigweed, for example, takes many minutes. Even in rules_python, it takes just under a minute (long enough where I think, "its just building docs, why is this taking so long?")

To fix this, I think we can implement a plugin that uses the `env-get-outdated` event; see this comment: https://github.com/sphinx-doc/sphinx/issues/11556#issuecomment-1667507177

api docs: https://www.sphinx-doc.org/en/master/extdev/event_callbacks.html#event-env-get-outdated

All it has to do is calculate a hash of the file and compare it to a previous hash.

Looking through those API docs, I wonder if some of those other events would be of interest, especially for a persistent worker.

Contributor guide

Open the contributing guide

Research direction

Start with Sphinx's env-get-outdated event documentation and the linked Sphinx issue comment. Investigate how a plugin can calculate and persist file hashes for comparison during incremental builds, and determine whether the other events or persistent-worker behavior matter. Done means unchanged files are skipped while changed files trigger rebuilding.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
build-system, documentation
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.