Clarification on fast-weight clipping and early training instability

Hi, thank you for releasing the In-Place TTT code.

I am studying a small-scale continual-training reproduction and found a
paper–code discrepancy regarding fast-weight clipping.

Appendix C.2 states that all reported Qwen3-4B evaluations clip each
inference update by Frobenius norm with tau=1e-5. However, in the current
public code:

- training directly applies `d_down_proj * ttt_lr` followed by `cumsum`;
- inference directly applies `current_w = current_w + dw`;

and I could not find the inference clipping described in the paper.
I also could not find any training-time fast-update clipping or
TTT-specific optimizer parameter group.

In a Qwen3-0.6B diagnostic run (32K sequence, chunk size 1024,
TTT layers [0,4,8,12,16,20,24]), after the first optimizer step:

- the first nonzero fast state increased future NLL from 1.86 to 10.81;
- max `||F_prefix z|| / ||W z||` reached 62.5;
- clipping each local update to max Frobenius norm 1 still gave a maximum
prefix output ratio of 15.2, because the updates accumulated across chunks.

I understand that this is not an exact reproduction of your large-scale
recipe. Could you please clarify:

1. Was the tau=1e-5 inference clipping used for the reported results, and
is it missing from the public inference code?
2. Is the clipping applied to each local chunk update or to the accumulated
prefix update, and before or after multiplying by `ttt_lr`?
3. Was any training-time stabilization used, such as fast-update clipping,
separate learning rates, parameter groups, warmup, or normalization?
4. What update/output scale is expected during the first few training steps?
5. Is there a minimal known-good public setup or checkpoint that exhibits
stable, nontrivial fast-weight behavior?

I can provide the full telemetry and reproduction code if useful.
Thank you very much.

Start by comparing the training and inference fast-weight update paths in the public code with the clipping described in Appendix C.2. Reproduce the Qwen3-0.6B diagnostic setup using the stated sequence length, chunk size, TTT layers, and telemetry. Done requires an authoritative clarification of clipping behavior, training stabilization, expected early-step scales, and a stable public setup or checkpoint.

Written by the indexing model from the issue text.