Exception in env-merge-info not printed correctly

Open
#13,339 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
3/5
Estimated time
1-2 days
Newbie friendliness
48/100
Issue type
Bug
Clarity
Clearly specified
Activity status
Stale
Tech stack
python
Domain
tooling

Research direction

Reproduce the failure with a parallel build using the conf.py handler shown, then inspect sphinx/util/parallel.py around join(), terminate(), and _join_one(). Trace how the env-merge-info exception travels through sphinx/events.py and sphinx/environment/init.py. Done means the user's relevant exception is displayed instead of the secondary KeyError.

Written by the indexing model from the issue text.

Description

internals:parallel type:bug
Describe the bug

If an env-merge-info event handler raises an exception, instead of getting propagated to the user the exception is swallowed by parallel.py.

The "traceback" part of the output is:

Traceback
=========

      File "/scratch/proj/sphinx/sphinx/util/parallel.py", line 126, in terminate
        self._result_funcs.pop(tid)
    KeyError: 0

Note that the full traceback is available in the log file. However, for most other exceptions, care is taken to display the user's most relevant traceback. Here's the full traceback from the log file:

Traceback
=========

    Traceback (most recent call last):
      File "/scratch/proj/sphinx/sphinx/events.py", line 404, in emit
        results.append(listener.handler(self.app, *args))
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/scratch/proj/sphinx-dummy/source/conf.py", line 14, in oh_no
        raise Exception('oh no!')
    Exception: oh no!

    The above exception was the direct cause of the following exception:

    Traceback (most recent call last):
      File "/scratch/proj/sphinx/sphinx/util/parallel.py", line 117, in join
        if not self._join_one():
               ^^^^^^^^^^^^^^^^
      File "/scratch/proj/sphinx/sphinx/util/parallel.py", line 140, in _join_one
        self._result_funcs.pop(tid)(self._args.pop(tid), result)
      File "/scratch/proj/sphinx/sphinx/builders/__init__.py", line 604, in merge
        self.env.merge_info_from(docs, env, self.app)
      File "/scratch/proj/sphinx/sphinx/environment/__init__.py", line 402, in merge_info_from
        self.events.emit('env-merge-info', self, docnames, other)
      File "/scratch/proj/sphinx/sphinx/events.py", line 415, in emit
        raise ExtensionError(
    sphinx.errors.ExtensionError: Handler <function setup.<locals>.oh_no at 0x7f63abe7e5c0> for event 'env-merge-info' threw an exception (exception: oh no!)

    During handling of the above exception, another exception occurred:

    Traceback (most recent call last):
      File "/scratch/proj/sphinx/sphinx/cmd/build.py", line 432, in build_main
        app.build(args.force_all, args.filenames)
      File "/scratch/proj/sphinx/sphinx/application.py", line 426, in build
        self.builder.build_update()
      File "/scratch/proj/sphinx/sphinx/builders/__init__.py", line 370, in build_update
        self.build(
      File "/scratch/proj/sphinx/sphinx/builders/__init__.py", line 398, in build
        updated_docnames = set(self.read())
                               ^^^^^^^^^^^
      File "/scratch/proj/sphinx/sphinx/builders/__init__.py", line 512, in read
        self._read_parallel(docnames, nproc=self.app.parallel)
      File "/scratch/proj/sphinx/sphinx/builders/__init__.py", line 613, in _read_parallel
        tasks.join()
      File "/scratch/proj/sphinx/sphinx/util/parallel.py", line 121, in join
        self.terminate()
      File "/scratch/proj/sphinx/sphinx/util/parallel.py", line 126, in terminate
        self._result_funcs.pop(tid)
    KeyError: 0
How to Reproduce

Enable parallel builds (-j 2 or or more) and have more than one input doc. Then add to your conf.py:

def setup(app):
    def oh_no(app, env, docnames, other):
        raise Exception('oh no!')

    app.connect("env-merge-info", oh_no)
Environment Information
Platform:              linux; (Linux-4.18.0-526.el8.x86_64-x86_64-with-glibc2.34)
Python version:        3.12.5 (main, Dec  3 2024, 00:00:00) [GCC 11.5.0 20240719 (Red Hat 11.5.0-2)])
Python implementation: CPython
Sphinx version:        8.2.0+/c49d925b4
Docutils version:      0.21.2
Jinja2 version:        3.1.5
Pygments version:      2.19.1
Sphinx extensions

Additional context

No response

Dominant language
Python
Stars
8k
Forks
2.6k
PR merge metrics
No merged PRs in 30d

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from sphinx-doc/sphinx

All issues in sphinx-doc/sphinx

Similar issues

More Python issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.