'Illegal instruction' when initializing SAM-3D Inference

Hi, and thanks for releasing this project. I’m trying to run SAM-3D Objects on a rented GPU machine and consistently hit an `Illegal instruction` crash during inference initialization. On a Vast.ai instance with an NVIDIA RTX A6000, SAM loads fine and CUDA is visible, but as soon as the SAM-3D pipeline initializes, the process exits with:

```text
Skipping custom attention backend; using default PyTorch attention.
Illegal instruction
```

Before patching, the last line was:

```text
2025-11-26 ... INFO sam3d_objects.pipeline.inference_pipeline:set_attention_backend:15 - GPU name is NVIDIA RTX A6000
Illegal instruction
```

So the crash appears to be inside or immediately after the attention backend setup in `sam3d_objects.pipeline.inference_pipeline`, or in one of the downstream CUDA/C++ extensions (MoGe/utils3d/etc.).

### Environment

* **Provider:** Vast.ai

* **GPU:** NVIDIA RTX A6000

* **OS:** Ubuntu 24.04.3 LTS

* **Python:** 3.11.x (in a virtual env/conda env called `sam3d-objects`)

* **PyTorch:** `torch.__version__` = `???` (installed from the official cu121 wheel index)

* **CUDA:** cu121 (PyTorch wheels)

* **Repo:** `facebookresearch/sam-3d-objects` (cloned at current `main` as of 2025-11-26)

* **Install:** from repo root

```bash
export PIP_EXTRA_INDEX_URL="https://pypi.ngc.nvidia.com https://download.pytorch.org/whl/cu121"
export PIP_FIND_LINKS="https://nvidia-kaolin.s3.us-east-2.amazonaws.com/torch-2.5.1_cu121.html"

pip install -e ".[inference]"
```

* **Additional steps:**

* Downloaded HF pipeline:

```python
from huggingface_hub import snapshot_download
snapshot_download(
repo_id="facebook/sam-3d-objects",
repo_type="model",
local_dir="checkpoints/hf",
local_dir_use_symlinks=False,
)
```

* Downloaded SAM ViT-H checkpoint into repo root:

```bash
wget -c https://dl.fbaipublicfiles.com/segment_anything/sam_vit_h_4b8939.pth
```

### How I call the model

From `/root/3D-reasoning/sam-3d-objects` I run a script that follows the same pattern as `demo.py`:

* Load SAM ViT-H and generate a mask for a single MS COCO image.
* Import `Inference` from `notebook/inference.py`.
* Initialize `Inference(config_path, compile=False)` with `config_path="checkpoints/hf/pipeline.yaml"`.
* Call `inference(image_rgb, mask, seed=42)` and save `output["gs"].save_ply(...)`.

The crash happens **before** any actual per-image inference loop, right after the SAM-3D pipeline logs the GPU name.

Minimal repro (conceptually):

```bash
cd /root/3D-reasoning/sam-3d-objects
python run_mscoco_batch.py # or an equivalent minimal script calling Inference(...)
```

Log:

```text
[INFO] Device: cuda
[INFO] Setting up SAM 2D...
[INFO] Using existing SAM weights at /root/3D-reasoning/sam-3d-objects/sam_vit_h_4b8939.pth
... FutureWarning from torch.load ...
2025-11-26 ... | INFO sam3d_objects.pipeline.inference_pipeline:set_attention_backend:15 - GPU name is NVIDIA RTX A6000
Illegal instruction
```

After I patched `set_attention_backend` to a no-op that just prints a message, I get:

```text
[INFO] Device: cuda
[INFO] Setting up SAM 2D...
[INFO] Using existing SAM weights at /root/3D-reasoning/sam-3d-objects/sam_vit_h_4b8939.pth
... FutureWarning from torch.load ...
Skipping custom attention backend; using default PyTorch attention.
Illegal instruction
```

So even with attention backend selection disabled, something in the SAM-3D initialization still triggers `Illegal instruction`.

### Things I already tried

* Installed `MoGe` from the commit referenced in your issues:

```bash
pip uninstall -y moge utils3d
pip install --no-binary :all: \
"MoGe @ git+https://github.com/microsoft/MoGe.git@a8c37341bc0325ca99b9d57981cc3bb2bd3e255b"
```

* Confirmed `moge` and `utils3d` import.

* Built `pytorch3d` from source against the current PyTorch/CUDA.

* Stubbed all Kaolin visualization usage (I’m not using the interactive visualizer).

* Patched `set_attention_backend` in `sam3d_objects/pipeline/inference_pipeline.py` to a no-op, as shown above.

Despite this, the process still dies with `Illegal instruction` immediately after that stage.

### Questions

1. Are there any **CPU instruction set assumptions** (e.g. AVX2, AVX512) in MoGe/utils3d/SAM-3D that could cause `Illegal instruction` on certain host CPUs, even with a supported GPU?
2. Is there a documented way to **force a simpler backend** (e.g. disable all custom CUDA kernels / fused attention) and run purely on standard PyTorch ops for debugging?
3. Is there a known **good environment matrix** (Python, torch, CUDA, CPU type) that you recommend for RTX A6000 where the current `main` + `.[inference]` is confirmed to work?
4. **How do I circumvent past this problem ?** If you’d like, I can try a fresh environment with specific version pins you suggest and share full logs, but right now my main blocker is this non-Python `Illegal instruction` as soon as the SAM-3D pipeline is initialized.