Video models are unusable on Apple Silicon: quantized pipelines hit an unimplemented MPS operator (aten::_int_mm)

Problem

Video models in this library don't work out of the box on Apple Silicon. The int8 quantization path this library uses (optimum-quanto with qint8) depends on aten::_int_mm, and upstream PyTorch has no MPS implementation for that operator — so on a Mac the generation fails with:

NotImplementedError: The operator 'aten::_int_mm' is not currently implemented for the MPS device.

This is tracked upstream in the MPS operator coverage issue: https://github.com/pytorch/pytorch/issues/141287

Desired outcome: an Apple Silicon user can install this library and run a video model successfully without hand-building PyTorch or discovering on their own which quantization settings are unusable on MPS. Whether that means a different quantization backend on MPS, a clear guard with an actionable message, a supported custom PyTorch build, or waiting on upstream is exactly the thing to work out.

Origin

Reported by Joshua Bretag. Surfaces when running video-model pipelines on Apple Silicon hardware; not specific to a single node.

Impact

Apple Silicon is a primary desktop target for Griptape Nodes users, and video models are currently unusable there in the quantized configuration. Today the failure is a raw NotImplementedError from the torch dispatcher, so a user has no path forward from the error alone — related to the un-actionable error surface noted in issue 17.

Repro / evidence

Minimal reproduction of the missing operator on an official macOS arm64 PyTorch build (from the upstream report):

import torch

a = torch.randint(-128, 128, (16, 16), dtype=torch.int8).to("mps")
b = torch.randint(-128, 128, (16, 16), dtype=torch.int8).to("mps")
torch._int_mm(a, b)

Relevant code in this repo:

• modular_diffusion_nodes_library/utils/pipeline_utils.py — _quantize_diffusion_pipeline() maps int8 to optimum.quanto.qint8 with no device-type branching, so MPS takes the same path as CUDA.
• _automatic_optimize_diffusion_pipeline() has an MPS branch for memory/offload behavior but nothing that accounts for unsupported quantization ops.

Possible direction

Not a decision, just the starting point that prompted this issue: a community fork adds MPS dispatch for _int_mm and _int_mm.out with a tiled Metal kernel — https://github.com/froggy-hyun/pytorch/tree/mps-int-mm. It is an unmerged fork of PyTorch main, whereas this library pins torch==2.7.0, so adopting it directly would mean shipping or asking users to build a patched PyTorch. Worth weighing against alternatives such as selecting a different quantization mode (or none) on MPS, or gating with a clear message until upstream lands support.