pymc-devs / pymc-devs/pytensor

PERF: TensorType.filter is O(N) for foreign array types (mx.array, jax.Array, torch.Tensor)

Open Beginner friendly
#2,087 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

backend compatibility enhancement jax mlx
Dominant language
Python
Stars
644
Forks
208
Avg merge
2d 14h
Merged PRs (30d)
16

Description

Part of #2085.

Current implementation

pytensor/tensor/type.py::TensorType.filter only fast-paths np.ndarray and np.memmap. Anything else (including mlx.core.array, jax.Array, torch.Tensor) falls into the trailing else branch:

def filter(self, data, strict=False, allow_downcast=None) -> np.ndarray:
    ...
    elif isinstance(data, np.ndarray) and (data.dtype == self.numpy_dtype):
        ...
    elif strict:
        ...
    else:
        if allow_downcast:
            data = np.asarray(data).astype(self.dtype)
        else:
            if isinstance(data, np.ndarray):
                ...
            elif (allow_downcast is None and isinstance(data, float | np.floating) ...):
                ...
            else:
                converted_data = np.asarray(data, self.dtype)
                if TensorType.values_eq(
                    np.asarray(data), converted_data, force_same_dtype=False
                ):
                    data = converted_data

The last branch does two np.asarray conversions and an element-wise comparison (values_eq runs a == b then np.all). Cost is O(N) in array size.

Reproducer + metrics

import time, mlx.core as mx, numpy as np
from pytensor.tensor.type import TensorType

T = TensorType("float32", shape=(None, None))
rng = np.random.default_rng(0)

for N in (256, 1024, 2048):
    x_np = rng.standard_normal((N, N)).astype(np.float32)
    x_mx = mx.array(x_np); mx.eval(x_mx); mx.synchronize()

    for label, arr in [("np", x_np), ("mx", x_mx)]:
        for _ in range(5): T.filter(arr)
        t0 = time.perf_counter()
        for _ in range(200): T.filter(arr)
        print(f"{N}x{N} filter({label}): {(time.perf_counter()-t0)/200*1e6:8.1f} us")

Output:

 256x256 filter(np):     0.7 us
 256x256 filter(mx):    21.3 us
1024x1024 filter(np):   0.7 us
1024x1024 filter(mx): 280.7 us
2048x2048 filter(np):   0.8 us
2048x2048 filter(mx): 1354.7 us

cProfile confirms 95 % of the cost is values_eq, not the conversion itself (np.asarray(mx.array) is ~18 µs zero-copy via Apple unified memory / DLPack).

End-to-end impact on a single f_mlx(mx_array, mx_array, mx_array) call for sin(x)*exp(y)+z at 2048²:

Input type Median (us)
np.ndarray 1417
mx.array (unpatched) 5854
mx.array (with fix below) 1397
mx.array (trust_input=True) 459
Raw mx.compile lower bound 478

4.2× speedup on mx.array input from a one-line addition.

Why __dlpack__ and not __array__

mx.array does not implement __array__. It implements __dlpack__ and the buffer protocol, which is what np.asarray uses internally. JAX jax.Array and PyTorch torch.Tensor also expose __dlpack__. Detecting via hasattr(..., "__dlpack__") or hasattr(..., "__array_interface__") covers all three.

Proposed change

File: pytensor/tensor/type.py, top of TensorType.filter:

def filter(self, data, strict=False, allow_downcast=None) -> np.ndarray:
    if isinstance(data, Variable):
        raise TypeError(...)

    # Fast-path for foreign array types implementing the array protocols
    # (mlx.core.array, jax.Array, torch.Tensor, etc.).
    if not isinstance(data, np.ndarray) and (
        hasattr(data, "__dlpack__") or hasattr(data, "__array_interface__")
    ):
        data = np.asarray(data)

    # ... rest unchanged ...

The conversion is zero-copy on Apple unified memory (M-series) and effectively zero-copy for any device array on the same host. Down-stream the existing np.ndarray fast path takes over.

This benefits MLX, JAX, and PyTorch users equally with no MLX-specific imports.

Tests

Add to tests/tensor/test_type.py:

def test_filter_accepts_dlpack_arrays():
    mx = pytest.importorskip("mlx.core")
    T = TensorType("float32", shape=(None, None))
    x = mx.array(np.zeros((4, 4), dtype=np.float32))
    out = T.filter(x)
    assert isinstance(out, np.ndarray)
    assert out.dtype == np.float32

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with TensorType.filter in pytensor/tensor/type.py and review the existing conversion and dtype-validation branches. Add the foreign-array coverage in tests/tensor/test_type.py, then run test_filter_accepts_dlpack_arrays and confirm the filter returns a NumPy array without the size-dependent comparison overhead.

Written by the indexing model from the issue text.

Assessment

Tech stack
numpy, python
Domain
backend, performance
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Quiet
Clarity
Clearly specified
Newbie friendliness
78/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.