google / google/torchax

aten::searchsorted lowering drops every keyword argument (right, side, out_int32, sorter)

Open
#106 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
243
Forks
38
Avg merge
15h 32m
Merged PRs (30d)
2

Description

## Summary

`torch.searchsorted(sorted_sequence, values, right=True)` raises `TypeError` under torchax. The
op **is** registered, so it does not fail with `OperatorNotFound`; the registered function
accepts only two positional parameters, while the aten schema has four keyword arguments.

`torchax/ops/jaten.py:277-286` (v0.0.13, unchanged on `main`):

```python
@op(torch.ops.aten.searchsorted.Tensor)
def _aten_searchsorted(sorted_sequence, values):
mappings.t2j_dtype(torch.get_default_dtype())
res = jnp.searchsorted(sorted_sequence, values)
if sorted_sequence.dtype == np.dtype(np.int32) or sorted_sequence.dtype == np.dtype(
np.int32
):
# res = res.astype(new_dtype)
res = res.astype(np.dtype(np.int64))
return res # jnp.searchsorted(sorted_sequence, values)
```

The aten schema is:

```
aten::searchsorted.Tensor(Tensor sorted_sequence, Tensor self, *, bool out_int32=False,
bool right=False, str? side=None, Tensor? sorter=None) -> Tensor
```

`jnp.searchsorted` supports the important one directly — `side='left'|'right'` — so this is a
signature gap, not a capability gap.

Three separate problems in those ten lines:

1. **`right` / `side` are unsupported**, so a caller asking for right-open bucketing silently
cannot express it, and gets a `TypeError` rather than a clear "unsupported argument".
2. **`out_int32` and `sorter` are likewise dropped.**
3. **The result dtype is forced to `int64` when the *haystack* is int32** (`:281-285`), which is
the opposite of `out_int32`'s meaning and, with `jax_enable_x64=False` (the default — see
below), produces a dtype JAX will not materialise at full width anyway. Line `:279` is a
no-op whose return value is discarded, and `:284` is commented-out code — this lowering looks
unfinished rather than deliberate.

Consistent with that reading, torchax's own conformance suite **skips** the op:
`test/test_ops.py:52` lists `"searchsorted"` in the skiplist, so the mismatch is known and
excluded rather than fixed.

## How it manifests

`torchax/tensor.py` dispatch calls `op.func(*args, **kwargs)` with the caller's kwargs passed
straight through and no per-op signature filtering, so a registered-but-narrower op raises a
plain Python `TypeError: _aten_searchsorted() got an unexpected keyword argument 'right'`. That
is harder to triage than a missing registration, because `OperatorNotFound` names the problem
and a `TypeError` points at torchax internals.

## Why it matters downstream

Encountered while running a PyTorch model on TPU through `tpu-inference` (which pins
`torchax==0.0.13`): the model used `torch.searchsorted(..., right=True)` for bucket lookup and
had to be rewritten to avoid it. Any model doing bucketing, histogram, or quantile-style index
lookup hits this, and the rewrite is not always semantics-preserving at bucket boundaries —
`right=True` is precisely the boundary behaviour.

## Suggested fix

```python
@op(torch.ops.aten.searchsorted.Tensor)
def _aten_searchsorted(sorted_sequence, values, *, out_int32=False, right=False,
side=None, sorter=None):
if sorter is not None:
sorted_sequence = jnp.take_along_axis(sorted_sequence, sorter, axis=-1)
if side is None:
side = "right" if right else "left"
elif right and side != ("right" if right else "left"):
raise ValueError("torch.searchsorted: `side` and `right` disagree")
res = jnp.searchsorted(sorted_sequence, values, side=side)
return res.astype(jnp.int32 if out_int32 else jnp.int64)
```

…and remove `"searchsorted"` from the `test/test_ops.py` skiplist so the OpInfo conformance test
covers it. Happy to send this as a PR with the skiplist entry removed if the shape looks right.

## Separately: the int64 half of what we originally reported

We had recorded "torchax has no int64". That is **not accurate** and we are correcting it here
rather than filing it:

- `torchax/ops/mappings.py:101-102` maps `torch.int64 -> jnp.int64` and `torch.long -> jnp.int64`
one-to-one. torchax does not downcast.
- 64-bit width is governed by JAX's global `jax_enable_x64`, which torchax exposes but leaves
off by default: `torchax/__init__.py:145` (`enable_accuracy_mode()` sets it `True`) versus
`:151` (`enable_performance_mode()` sets it `False`).

So the truncation we observed (`Explicitly requested dtype int64 ... truncated to int32`) comes
from JAX's x64 default, and — per JAX's own documentation on TPU — 64-bit integer support on
XLA:TPU is limited independently of that flag. A torchax-level fix would not restore it. Nothing
to file against torchax; the honest statement for a downstream backend is "assume 32-bit
integers on TPU and carry 64-bit arithmetic in limbs", which is what we did.

Contributor guide

Open the contributing guide

Research direction

Start in torchax/ops/jaten.py:277-286 and compare the registered searchsorted lowering with the aten schema and JAX searchsorted arguments. Check torchax/tensor.py dispatch behavior, then update test/test_ops.py to remove searchsorted from the skiplist and run its conformance coverage. Done means right/side, out_int32, and sorter are handled consistently, result dtype follows out_int32, and the relevant tests pass.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, pytorch
Domain
backend-api-design, machine-learning, testing-qa
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
78/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.