Clarifying in-place-TTT targets

Thank you for releasing the training and inference code for this wonderful work! However, I noticed a small discrepancy between Figure 1 in the final version of the paper and the actual codebase implementation regarding the Test-Time Training (TTT) target.

Figure.1 in your final version, the TTT target derived from **hidden states** or **input embeddings** **before** self-attention (sliding attention block).
The training code, encapsulated in [class Qwen3MLP(nn.Module):](https://github.com/ByteDance-Seed/In-Place-TTT/blob/be2324829b0e91c8fd10a74d4b43714fde6676e1/hf_models/hf_qwen3/modeling_qwen3.py#L72), operates Conv1D upon hidden states computed **after** self-attention block, which is the hidden states.

Could you please clarify whether the TTT objective should ideally align the hidden states before or after the attention block?

Conceptually, aligning the states before attention or after attention both seem to have distinct implications. Could you please clarify whether the TTT objective should ideally align the hidden states before or after the attention block?

Thank you again for your time and contribution.

Start by comparing Figure 1 in the final paper with the Qwen3MLP class in hf_models/hf_qwen3/modeling_qwen3.py, especially the Conv1D input and its position relative to self-attention. Trace which hidden states are used in the training code. Done means documenting whether the TTT target should be taken before or after attention and explaining the discrepancy.

Written by the indexing model from the issue text.