sphinx-doc / sphinx-doc/sphinx
"env-merge-info" for the "write" phase
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 8k
- Forks
- 2.6k
- PR merge metrics
- No merged PRs in 30d
Description
Is your feature request related to a problem? Please describe.
There are currently no built-in facilities for the parallel-write-safety of extensions which need to collect data during writing and make use of that information later (during generation of output file or build-finished.
The specific issue here is related to the sphinx-sitemap extension, which basically does all its work during the writing phase: it collects data during html-page-context and writes out its production during built-finished. As a result it currently does not work at all in parallel_write mode, and it's not entirely clear how to cleanly make it parallel_write_safe. Which is a shame as that means sphinx-sitemap incurs a large build time wallclock regression when enabled (and also a bright red warning from Sphinx).
Describe the solution you'd like
An event similar to env-merge-info but taking place after the parallel writing phase. It's also not entirely clear whether "15. Generate output files" is parallelised or not, and if so how. If the output file generation is parallelised, there may need to be two different events (one to gather information after parallel post-transforms and doctree-resolution, and an other after output generation).
Describe alternatives you've considered
That Sphinx provides communication pipelines between the workers (and the controller) instead, creating those pipelines by hand is pretty error-prone and heavy especially as the specific parallelisation model is undefined (for good reasons), and will (hopefully) change in the future to better accomodate Windows and macOS.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by tracing the env-merge-info, html-page-context, and build-finished events through the parallel writing phase and the output-file generation stage. Clarify which stages are parallelized and define the event behavior needed for extensions such as sphinx-sitemap to collect data safely. Done means the event design and parallel-write safety semantics are agreed and documented.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- build-system, documentation
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 30/100