sphinx-doc / sphinx-doc/sphinx

Sphinx breaks transforms that depend on `document.transformer.components`, like the standard `Filter` transform

Open
#9,632 3 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

api type:question
Dominant language
Python
Stars
8k
Forks
2.6k
PR merge metrics
No merged PRs in 30d

Description

Describe the bug

Sphinx does not give transforms (nor post-transforms) a way to determine which writer will be used. This is because it reads the document with a DummyWriter to generate a doctree, and then it calls post_transforms with an empty list of self.document.transformer.components.

This breaks transforms that depend on components, including the standard docutils.transforms.components.Filter.

How to Reproduce

The standard Filter transform does this:

    def apply(self):
        pending = self.startnode
        component_type = pending.details['component'] # 'reader' or 'writer'
        format = pending.details['format']
        component = self.document.transformer.components[component_type]
        if component.supports(format):
            pending.replace_self(pending.details['nodes'])
        else:
            pending.parent.remove(pending)

This breaks with Sphinx: if added as a regular transform by note_pending, components[component_type] will be Sphinx' DummyWriter if component_type is "writer", and the call to supports will return the wrong results. If added as a post-transform it throws an exception because components is empty.

Here is another example simplified from a separate project; it works with plain Docutils, but not with Sphinx:

class MyTransform(Transform):
    def apply(self):
        formats = set(self.document.transformer.components['writer'].supported)
        for node in self.document.traverse(some_pending_node_type):
            if "html" in formats:
                node.replace_self(nodes.raw("<em>Hello!</em>", format="html"))
            if {'latex', 'xelatex', 'lualatex'} & formats:
                node.replace_self(nodes.raw(r"\emph{Hello!}", format="latex"))

If added as a regular transform, components['writer'] will be Sphinx' DummyWriter and supported will be only {'html'}. If added as a post_transform the code will throw an exception because components won't have a 'writer' key.

Expected behavior

Ideally, the Filter transform (and other similar transforms) should just work, which might require running transforms after the caching stage, with the correct set of components (I imagine DummyWriter is for caching purposes?).

If that's not possible, then maybe it's possible for post_transforms? At the moment post_transforms do not see any components at all.

If that's not possible, then it would be nice to have some (Sphinx-specific, unfortunately) way to determine the list of formats supported by the writer from a post-transform.

Python version

Python 3.8.10

Sphinx version

sphinx-build 3.5.4

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Reproduce the issue using a transform based on docutils.transforms.components.Filter and inspect how Sphinx invokes regular transforms and post_transforms around DummyWriter and document.transformer.components. Compare the available components with plain Docutils behavior. Done means transforms can determine the actual writer formats without incorrect results or missing component keys.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
documentation, tooling
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.