[Model]: TR-HASH MoE with stacked 3D expert weights

### Model name

TR-HASH MoE

### Model variant / size

TR-HASH MoE 200M · approximately 201.2M parameters · Full-SFT checkpoint · approximately 162B source-token exposure

### Model link

https://huggingface.co/AETHORIA-AI/TR-HASH-MoE-200M-160B-SFT

### What do you want AutoRound to support?

Quantization / rounding workflow support for this model

### Motivation & use case

We want to publish and evaluate a correctly quantized version of our open TR-HASH MoE model through the Intel Low Bit Open LLM Leaderboard.

TR-HASH uses deterministic token-ID routing instead of a learned router. It is designed for auditable and reproducible sparse computation.

Our goals are:

- W4A16 quantization of the complete model
- lower-memory CPU and CUDA inference
- PIQA and generation regression testing
- comparison with the original full-precision checkpoint

Quantizing only the standard Linear projections would be misleading because the stacked routed-expert tensors represent a substantial part of the model.

### Environment (optional but helpful)

Submission path: Intel Low Bit Open LLM Leaderboard

Model:
https://huggingface.co/AETHORIA-AI/TR-HASH-MoE-200M-160B-SFT

Framework:
https://github.com/Complexity-ML/complexity-framework

Core engine:
https://github.com/Complexity-ML/complexity-framework/blob/main/complexity/tr_hash/engine.py

Target schemes:
- W4A16 tuned AutoRound
- W4A16 model-free RTN

Target hardware:
- CUDA
- CPU

### Current blockers / errors (if any)

Intel Low Bit Open LLM Leaderboard submission failed before quantization started.

Selected configuration:
- Scheme: MXFP4
- Method: Tuning (AutoRound, recommended)

Displayed error:
"Quantization submission upload failed. Please try again later."

The interface did not expose a detailed backend traceback.

### Definition of Done (acceptance criteria)

Support is complete when:

1. AutoRound can load the public TR-HASH checkpoint.
2. It recognizes the stacked expert gate/up/down tensors.
3. Each expert slice is quantized as an independent 2D matrix.
4. The outer expert dimension and deterministic routing tables are preserved.
5. W4A16 tuned quantization completes successfully.
6. Model-free W4A16 RTN supports the same layout, or fails explicitly if unsupported.
7. The exported checkpoint loads successfully for inference.
8. The quantized model produces finite logits and passes generation and PIQA regression tests.
9. AutoRound clearly reports which weights were quantized or retained in full precision.

### Additional context

TR-HASH has 201.2M parameters, 16 transformer layers, GQA, four stored experts with top-2 activation, an always-active shared SwiGLU path, and tied embeddings.

Routing is deterministic and based on token IDs. There is no learned router to quantize. Integer routing tables must remain unchanged.

Each slice along the first dimension of an expert tensor is conceptually an ordinary 2D Linear projection, so per-expert quantization should be possible without changing routing behavior.

Relevant implementation:

- Engine:
https://github.com/Complexity-ML/complexity-framework/blob/main/complexity/tr_hash/engine.py

- MLP adapter:
https://github.com/Complexity-ML/complexity-framework/blob/main/complexity/core/mlp/tr_hash_engine.py

Related AutoRound request:
https://github.com/intel/auto-round/issues/1711

That request concerns Nemotron-H and does not cover TR-HASH’s custom stacked 3D expert parameters.

Read complexity/tr_hash/engine.py and complexity/core/mlp/tr_hash_engine.py, then trace AutoRound's model-loading and weight-discovery path using the public TR-HASH checkpoint. Verify how stacked expert tensors and deterministic routing are represented before assessing the quantization path. Done means the checkpoint loads, expert slices are handled without changing routing, export and inference work, and the stated regression checks pass.

Written by the indexing model from the issue text.