Incorrect `objects.inv` produced for `singlehtml`

Describe the bug

I haven't been able to get Intersphinx links to objects in Hyrule to work. I think this is because Hyrule uses the singlehtml builder and the generated objects.inv in this case has spurious anchors.

How to Reproduce

#!/bin/sh
mkdir testp
cd testp
mkdir mylib
printf 'def foo():\n "my docstring"\n return 1\n' >mylib/__init__.py
printf '.. automodule:: mylib\n  :members:\n' >index.rst
echo 'extensions = ["sphinx.ext.autodoc"]' >conf.py
mkdir build
PYTHONPATH=. sphinx-build . _build -b singlehtml
python3 -msphinx.ext.intersphinx _build/objects.inv

Expected behavior

I expected index.html#module-mylib in the output, as with the html builder, instead of index.html#document-index#module-mylib.

Your project

See the shell script above.

Screenshots

No response

OS

Ubuntu 21.10

Python version

3.9.7

Sphinx version

4.3.2

Sphinx extensions

sphinx.ext.autodoc

Extra tools

No response

Additional context

No response

Start by running the shell reproduction with Sphinx's singlehtml builder and inspect the generated _build/objects.inv using sphinx.ext.intersphinx. Trace how the singlehtml output records the module anchor, then verify that the inventory uses index.html#module-mylib rather than index.html#document-index#module-mylib.