Intersphinx references work without a namespace/identifier but fail if used

Describe the bug

We are trying to cross-reference several sites built with Sphinx. We found that some references are properly resolved, but others fail. Precisely, if we reference labels in other sites without specifying the namespace/identifier, it does work. However, that is ambiguous, because multiple sites might have the same label. Unfortunately, when using the namespace/identifier, references are not resolved. To our surprise, that is not consistent. It works for some sites, but not for others.

I'm not sure about this being a bug or a misconfiguration of some of the sites.

How to Reproduce

$ git clone https://github.com/edaa-org/edaa-org.github.io
$ cd edaa-org.github.io
$ git clone -b v1.0.0-rc1 https://github.com/buildthedocs/sphinx.theme _theme
$ make html
$ # open _build/html/index and see "Conceptual Model > 3 | Language Model".

As seen, refs to python or OSVB namespaces/identifiers are resolved, but references to pyVHDLModel or pySystemVerilogModel do fail.

Expected behavior

Refs to pyVHDLModel or pySystemVerilogModel Glossary labels should work as python:comparisons or OSVB:API:Core.

Your project

https://github.com/edaa-org/edaa-org.github.io

Screenshots

OS

Linux and MSYS2

Python version

3.9

Sphinx version

4.2

Sphinx extensions

https://github.com/edaa-org/edaa-org.github.io/blob/19f14102835c2fc1224363ee46296058c06a1753/conf.py#L168-L172

Extra tools

No response

Additional context

Same issue in VHDL/pyVHDLModel#32.

/cc @Paebbels

Reproduce the failure with the edaa-org/edaa-org.github.io project and inspect conf.py around lines 168-172, including its intersphinx configuration. Compare the working python and OSVB namespace references with the failing pyVHDLModel and pySystemVerilogModel references under Sphinx 4.2. Done means the glossary labels resolve with their namespace or identifier.