DiamondLightSource / DiamondLightSource/httomo

Combining `|` and `xp.ndarray` in a type causes warning in sphinx API docs build

Open
#492 2 comments 0 reactions 0 assignees View on GitHub
bug documentation
Dominant language
Python
Stars
10
Forks
5
Avg merge
5d 17h
Merged PRs (30d)
1

Description

As part of the warnings seen in #468, with a bit of trial and error I have narrowed down the cause of the "unsupported operand `|`" errors being produced by sphinx, despite running with a python 3.12 interpreter (which should recognise the `|` operator as another way to do `Union`).

It seems like mixing `|` and types that come from the `xp` variable causes the issue. If `xp` is not used, then the `|` poses no problem for sphinx.

I found this by being able reproduce the warning sphinx generates, but in a dummy module I created locally in my httomo repo and generating API docs for that module only.

The dummy module (which I lovingly called `bloop.py`) is the following:
```python
from typing import TypeAlias, Union

import numpy as np

try:
import cupy as xp

try:
xp.cuda.Device(0).compute_capability
gpu_enabled = True # CuPy is installed and GPU is available
except xp.cuda.runtime.CUDARuntimeError:
import numpy as xp

except ImportError:
import numpy as xp

Blah: TypeAlias = Union[np.ndarray, xp.ndarray]
Thing: TypeAlias = int | float
```

where you can see that I have defined `xp` as it is defined in `utils.py`: https://github.com/DiamondLightSource/httomo/blob/f619ed123f2c5aad790ad75751d737c817a44196/httomo/utils.py#L11-L21

Having `Blah` defined as:
```python
Blah: TypeAlias = Union[np.ndarray, xp.ndarray]
```

causes no issues, but having it defined as:
```python
Blah: TypeAlias = np.ndarray | xp.ndarray
```

produces the warning:
```
(/dls/science/users/twi18192/conda-envs/docs-httomo-py310-updated) [twi18192@pc0074 httomo (sphinx-fixes)]$ source docs/sphinx-build.sh
***********************************************************************
Starting the sphinx script
***********************************************************************
Creating plugin API files and html pages ..
Running Sphinx v8.1.3
loading translations [en]... done
making output directory... done
Blah is int | float
[autosummary] generating autosummary for: autosummary_api.rst, backends/list.rst, backends/templates.rst, developers/how_to_contribute.rst, developers/memory_calculation.rst, explanation/faq.rst, explanation/process_list.rst, explanation/templates.rst, howto/how_to_run/at_diamond.rst, howto/how_to_run/outside_diamond.rst, ..., introduction/indepth/chunks.rst, introduction/indepth/detailed_about.rst, introduction/indepth/memory_estimators.rst, introduction/indepth/reslice.rst, introduction/indepth/sections.rst, introduction/indepth/wrappers.rst, pipelines/yaml.rst, reference/loaders.rst, reference/yaml.rst, utilities/yaml_checker.rst
WARNING: [autosummary] failed to import httomo.bloop.
Possible hints:
* TypeError: unsupported operand type(s) for |: 'type' and 'ndarray'
* ImportError:
* AttributeError: module 'httomo' has no attribute 'bloop'
Writing evaluated template result to /dls/science/users/twi18192/httomo/docs/build/_static/nbsphinx-code-cells.css
building [mo]: all of 0 po files
writing output...
building [html]: all source files
updating environment: [new config] 36 added, 0 changed, 0 removed
reading sources... [100%] utilities/yaml_checker
/dls/science/users/twi18192/httomo/docs/source/autosummary_api.rst:4: WARNING: autosummary: failed to import httomo.bloop.
Possible hints:
* TypeError: unsupported operand type(s) for |: 'type' and 'ndarray'
* ImportError:
* AttributeError: module 'httomo' has no attribute 'bloop'
looking for now-outdated files... none found
pickling environment... done
checking consistency... done
preparing documents... done
copying assets...
copying static files...
Writing evaluated template result to /dls/science/users/twi18192/httomo/docs/build/_static/basic.css
Writing evaluated template result to /dls/science/users/twi18192/httomo/docs/build/_static/documentation_options.js
Writing evaluated template result to /dls/science/users/twi18192/httomo/docs/build/_static/language_data.js
Writing evaluated template result to /dls/science/users/twi18192/httomo/docs/build/_static/copybutton.js
copying static files: done
copying extra files...
copying extra files: done
copying assets: done
writing output... [100%] utilities/yaml_checker
generating indices... genindex done
copying linked files...
copying notebooks ...
highlighting module code...
writing additional pages... search done
copying images... [100%] _static/wrappers/wrappers.png
dumping search index in English (code: en)... done
dumping object inventory... done
build succeeded, 2 warnings.

The HTML pages are in docs/build.
***********************************************************************
End of script
***********************************************************************
```

Furthermore, note that `Thing` uses `|` in its definition and that it does _not_ produce the warning in any case.

Therefore, I have concluded for now that "something" about using `xp` with `|` isn't quite right, but what the "something" is I don't know.

I don't fancy trying to dig into sphinx internals or python issues regarding `|` and variables whose values are modules, so I think a reasonable quick solution to this would be to use `Union` if a union type involving `xp` is needed, and to use `|` in any other union type.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.