rms_norm translation inflates eps by max(|x|)^2 — percent-level errors on spiky activations even at FLOAT32 / CPU_ONLY

## 🐞 Describing the bug

The torch frontend's `rms_norm` handler (`converters/mil/frontend/torch/ops.py`) rescales the input by its max absolute value to prevent fp16 overflow on ANE:

```
rms = sqrt(mean((x / m)^2) + eps) * m where m = max(|x|)
```

Algebraically this equals `sqrt(mean(x^2) + eps * m^2)` — **the epsilon is inflated by `max(|x|)^2`**, because `eps` is added *after* the rescale instead of being rescaled with the input.

For activations where `max(|x|) >> rms(x)` (spiky distributions — common in transformer residual streams, register tokens, and learned embeddings), this changes the output by orders of magnitude more than the comment's "< 0.1% in practice" estimate. Two consequences:

- The error appears **even at `compute_precision=FLOAT32` with `CPU_ONLY`**, where no overflow protection is needed at all. The in-code note "For applications requiring exact PyTorch parity, consider using CPU/GPU compute units" does not hold — the deviation is in the emitted MIL math, not in execution precision.
- In a real model (a 380M-parameter bf16-trained EEG transformer with QK-RMSNorm and register tokens) we measured **4–6% relative L2** on encoder outputs at fp32/CPU_ONLY, entirely attributable to this op (verified by bisection and by patching the module to the explicit formula, which drops end-to-end error to ~1e-6).

## To Reproduce

Self-contained script — plain `torch.nn.RMSNorm`, fp32, CPU_ONLY:

```python
import coremltools as ct
import numpy as np
import torch

class Model(torch.nn.Module):
def __init__(self, dim: int = 64) -> None:
super().__init__()
self.norm = torch.nn.RMSNorm(dim, eps=1e-5)

def forward(self, x):
return self.norm(x)

torch.manual_seed(0)
dim = 64
model = Model(dim).eval()

# Spiky input: mostly small values, one large component per row.
x = 0.01 * torch.randn(1, 8, dim)
x[..., 0] = 25.0

with torch.no_grad():
ref = model(x).numpy()

traced = torch.jit.trace(model, x)
mlmodel = ct.convert(
traced,
inputs=[ct.TensorType(name="x", shape=x.shape, dtype=np.float32)],
convert_to="mlprogram",
compute_precision=ct.precision.FLOAT32,
compute_units=ct.ComputeUnit.CPU_ONLY,
)
got = np.array(list(mlmodel.predict({"x": x.numpy()}).values())[0], dtype=np.float32)

rel = np.linalg.norm(ref - got) / np.linalg.norm(ref)
print(f"rel_l2 (fp32, CPU_ONLY): {rel:.6f}")
```

Output on coremltools 9.0:

```
rel_l2 (fp32, CPU_ONLY): 0.000319
```

Expected: ~1e-7 (fp32 rounding). Observed: 3.2e-4 on this toy — and percent-level on real models, where the spike ratio `m^2 / mean(x^2)` is much larger. Effective eps here: `1e-5 * 25^2 ≈ 6.3e-3`, i.e. the epsilon is inflated ~625×.

## Suggested fixes

Either (or both):

1. **Rescale eps along with the input** so the math is exact regardless of scale: `sqrt(mean((x/m)^2) + eps/m^2) * m`, guarding `m = 0` (e.g. `m = max(m, 1)` or a small floor — this also fixes the current `0/0` on an all-zero row, which the max-rescale itself introduces).
2. **Gate the rescale on fp16** — only emit the overflow-protection form when the op will actually run at fp16 (`compute_precision=FLOAT16`); emit the plain `x * rsqrt(mean(x^2) + eps)` form at fp32.

## Workaround (for anyone hitting this)

Replace `nn.RMSNorm` / `F.rms_norm` in the module with the explicit formula before tracing:

```python
def forward(self, x):
rms = torch.rsqrt((x * x).mean(-1, keepdim=True) + self.eps)
return x * rms * self.weight
```

This decomposes into exact MIL ops and restores parity (~1e-6 end-to-end in our 380M model).

## System environment

- coremltools version: 9.0
- OS: macOS 27.0 (26A5406e beta) — also reproduced on macOS 15
- Python: 3.11.14
- torch: 2.11.0 (also reproduced with 2.10 and 2.13 nightly)
- How you install python: uv venv

Start in converters/mil/frontend/torch/ops.py at the torch frontend’s rms_norm handler, then run the self-contained reproduction with FLOAT32 and CPU_ONLY. The work is done when spiky and all-zero inputs avoid epsilon inflation and the converted output matches the PyTorch reference within expected fp32 rounding.

Written by the indexing model from the issue text.