CPU delegate silently miscompiles a bool bitwise_and chain cast to float — GPU lane is correct, no diagnostic emitted
- Dominant language
- Python
- Stars
- 152
- Forks
- 45
- Avg merge
- 1d 7m
- Merged PRs (30d)
- 12
Description
### Summary
A boolean `bitwise_and` chain that is cast to a float dtype produces **silently wrong results on
the CPU delegate**. The GPU delegate computes the same graph correctly. Nothing raises, and no
diagnostic is emitted — the model simply returns different numbers depending on which compute
unit it ran on.
Two algebraically identical formulations of the same mask are correct on both lanes, which
isolates the defect to the `(a & b).to(dtype)` pattern.
Found while decomposing `torchvision.ops.deform_conv2d` into supported ops: the out-of-bounds
validity mask for bilinear sampling is naturally written as a bool AND chain, and the CPU lane
silently returned garbage (PSNR 10.6 dB against the eager reference, where the GPU lane gave
146.8 dB).
macOS 27.0 (26A5421a), M5 Max, `coreai-core==1.0.0b2`, `coreai-torch==0.4.1`, torch 2.11.0.
### Reproduction
```python
import asyncio, numpy as np, torch, torch.nn as nn
from pathlib import Path
from coreai_torch import TorchConverter, get_decomp_table
from coreai.runtime import AIModel, NDArray, SpecializationOptions, ComputeUnitKind
class M(nn.Module):
def __init__(self, mode):
super().__init__(); self.mode = mode
def forward(self, x):
i = torch.floor(x * 8.0) # integer-valued, some out of [0,4)
if self.mode == "bool":
v = ((i >= 0) & (i < 4)).to(x.dtype) # <-- wrong on CPU
elif self.mode == "mul":
v = (i >= 0).to(x.dtype) * (i < 4).to(x.dtype)
else:
v = torch.clamp(i + 1.0, 0.0, 1.0) * torch.clamp(4.0 - i, 0.0, 1.0)
return x * v
async def main():
torch.manual_seed(0)
x = torch.rand(1, 1, 8, 8) * 2.0 - 0.5
for mode in ("bool", "mul", "clamp"):
m = M(mode).eval()
with torch.no_grad():
ref = m(x).numpy().astype(np.float64)
p = Path(f"/tmp/min_{mode}.aimodel")
ep = torch.export.export(m, args=(x,)).run_decompositions(dict(get_decomp_table()))
TorchConverter().add_exported_program(
ep, input_names=["x"], output_names=["out"]).to_coreai().save_asset(p)
row = f"{mode:6s}"
for lane, mk in (("cpu", ComputeUnitKind.cpu), ("gpu", ComputeUnitKind.gpu)):
opts = SpecializationOptions.from_preferred_compute_unit_kind(mk())
mdl = await AIModel.load(str(p), specialization_options=opts)
fn = mdl.load_function(next(iter(mdl.function_names)))
r = await fn({"x": NDArray(x.numpy())})
got = np.asarray(r[next(iter(r))].numpy()).astype(np.float64)
row += f" {lane}: max|delta|={np.abs(got - ref).max():.3e}"
print(row)
asyncio.run(main())
```
### Observed
```
bool cpu: max|delta|=4.639e-01 gpu: max|delta|=0.000e+00
mul cpu: max|delta|=0.000e+00 gpu: max|delta|=0.000e+00
clamp cpu: max|delta|=0.000e+00 gpu: max|delta|=0.000e+00
```
### Expected
All three formulations are algebraically identical and agree exactly in eager PyTorch. All three
should agree on every compute unit. In particular the CPU lane should not disagree with the GPU
lane by 0.46 on data in roughly `[-0.5, 1.5]`.
### Why this one is worth prioritising
It is in the **silent-wrongness** class rather than the crash class:
- No exception, no warning, no ANE/CPU validation message.
- Correct on the GPU, wrong on the CPU — so it will pass a review that only exercises one lane,
and it makes the CPU lane unusable as a reference for validating the others.
- The affected pattern is idiomatic. Bounds masks written as `(i >= lo) & (i < hi)` appear in any
hand-written gather/sampling kernel — deformable convolution, grid sampling, custom
interpolation, padding emulation.
### Related
`(a & b)` on booleans is also rejected by ANE validation with
`ANE I/O op can only do F16 MemRef <-> F32 Tensor cast` (the failing nodes are named
`bitwise_and_*`), so the same pattern blocks Neural Engine residency. The `clamp` formulation
above avoids both problems and is exact — which may be a useful hint about where the CPU lowering
diverges.
Possibly the same underlying area as #11 (int64-comparison bool mask chain, deformable-attention
sampler pattern), though the symptom there is a clobbered tensor rather than a wrong mask.
Contributor guide
Assessment
This issue has not been assessed yet.