dask / dask/dask

Document (or extend) which dask.array APIs respect array.backend for host literals

Open
#12,440 1 comment 0 reactions 0 assignees View on GitHub
needs triage
Dominant language
Python
Stars
13.9k
Forks
2k
PR merge metrics
No merged PRs in 30d

Description

When using the CuPy array backend (`dask.config.set({"array.backend": "cupy"})` or
equivalent), **factory-style** constructors such as `dask.array.zeros` produce
collections with CuPy chunk meta, but **host literals** passed through
`dask.array.atleast_1d` or `dask.array.from_array(numpy_buffer)` still yield
**NumPy-meta** Dask arrays.

I would like this behaviour to be **explicitly documented** in Dask array
backend docs. Optionally, I'm interested in whether scalar promotion could
ever respect the active `array.backend` when the input is a Python scalar (not
only when the user passes a CuPy buffer explicitly).

### Minimal reproducer
```python
import dask
import dask.array as da
import numpy as np
with dask.config.set({"array.backend": "cupy"}):
import cupy as cp # noqa: F401 — register backend
z = da.zeros((2, 2))
literal = da.atleast_1d(1.5e9)
from_np = da.from_array(np.array([1.0, 2.0], dtype=np.float32), chunks=(2,))
z_meta = type(z._meta).__module__
lit_meta = type(literal._meta).__module__
from_meta = type(from_np._meta).__module__
print("zeros meta:", z_meta) # cupy — follows array.backend
print("atleast_1d(float) meta:", lit_meta) # numpy — does not
print("from_array(np) meta:", from_meta) # numpy — follows input buffer
```

On a CuPy-enabled environment we observe:
```
zeros → cupy
atleast_1d(1.5e9) → numpy
from_array(np.ndarray(...)) → numpy
```

### Questions for maintainers
- Is this intended? If so, which operations are guaranteed to honour `array.backend` vs which always follow the dtype/module of the host buffer?

- Should `da.atleast_1d(scalar)` under array.backend=cupy ever produce CuPy-meta chunks, or should callers always use `da.from_array(cupy.asarray([...]))`?

- If the design is “input array type wins for from_array”, could the docs state that clearly next to the array.backend configuration guide?

Environment
```
dask==2024.10.0 (also please note if behaviour differs across versions)
Python 3.11+
cupy + CUDA (CuPy wheel matching local toolkit)
```
### What would help downstream

A short table in the array backend documentation: operation → respects array.backend / respects input buffer / notes.
If behaviour changes in a future release, mention it in release notes so libraries can drop workarounds.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.