Retained sampled suffix can carry logprobs under a different causal prefix

Status reconciliation — September 11, 2026 (Schulman)

Default sampled-lineage attribution remains open; PR #885 is explicitly HELD by Brad. Do not merge it, add a blanket manual-history rejection rule or treat private experiment constraints as the shared API contract.

The independent prompt-aware capture-deduplication fix #886 is already merged at 66f644db05954ae950c13554ce0cc9f4df020af8. Earlier comments calling it unmerged are historical. #886 does not include #885 and does not resolve this separate default-attribution question.

Planned lane: Schulman and subagents, parked pending Brad's scope decision; no new implementation or launch is initiated by this status update. Manually assigned/edited histories remain supported.

Historical report (preserved):

Native multi-history tokenization can mark a retained response suffix
SAMPLED|EXACT and select it for training after earlier sampled tokens have been
removed from its conditioning context. Its stored logprob still belongs to the
original longer prefix. This is independent of the terminal renderer-tail work
in #882 and does not require the experimental raw-capture adapter.

Minimal synthetic capture:

The second request retains the first assistant's message text, but its prompt
tokens omit token 2. With multi_history=True and
reconcile_text_equivalent_tokenizations=False, ART emits a second history
[1, 3, 4, 5], marks token 3 as SAMPLED|EXACT|ASSISTANT|OUTPUT (23), and gives
it the first response's logprob -0.3. first_occurrence_masks(where=SAMPLED)
selects it: prefix [1, 3] differs from the actual sampled prefix [1, 2, 3].
The probability was observed for 3 | [1, 2], not 3 | [1].

A CPU regression using the repository's existing synthetic exchange fixture:

import art
from art.trajectories import TokenFlag
from tests.unit.trajectories.test_tokenize import _chat_exchange

def test_retained_suffix_requires_its_original_causal_prefix():
    trajectory = art.Trajectory(
        exchanges=art.trajectories.TrajectoryExchanges(
            chat_completions=[
                _chat_exchange([1], [2, 3]),
                _chat_exchange([1, 3, 4], [5], offset=1),
            ]
        )
    )
    tokenized = trajectory.tokenize(
        multi_history=True,
        reconcile_text_equivalent_tokenizations=False,
    )
    masks = tokenized.tensorize().first_occurrence_masks(where=TokenFlag.SAMPLED)
    captured_prefixes = {(1, 2), (1, 2, 3), (1, 3, 4, 5)}
    for history, mask in zip(tokenized.histories, masks, strict=True):
        for index, selected in enumerate(mask.tolist()):
            if selected:
                assert tuple(history.tokens[:index + 1]) in captured_prefixes

This reproduces on ART 1ce5d7213e2d9cea2dd8a04a608966c24f4629a0 and current
077abf6fa7a3a51f9ff255c37f8886c6c8256082 (both _tokenize.py SHA256
cf8745b44edf3fb22a377246ccb81fc476153f4d96d5680e9c953180c4cff134), and the
reviewed terminal-boundary overlay 7bcd560cc117e593bbc8fad17a856ffc8dcbf667
(9922a80bafeccc89d1439b4c04bc08f63daccf047b37084e01d819e6d2ec0dd2).
The existing first-occurrence trie behaves as designed: it keys model-visible
prefixes, so it cannot repair an earlier attribution error.

The source is _retained_output_suffix, which searches every output[start:]
against the later prompt. _tokenize_exact_projected_chat_history then attaches
the suffix's original logprobs and sampled/exact flags at len(prompt).
Its complete-output requirement only applies to earlier finish_reason=length
sources; a non-length source can therefore lose its original causal conditioning.

Independent offline analysis also found this shape in two real captured
rollouts: 321 and 896 positions respectively remained first-occurrence-selected
after two leading sampled tokens were omitted. Raw content is private; the
synthetic regression above contains no research payload and reproduces the
same native behavior. This establishes attribution risk, not a measured GPU
training-quality effect.

Expected behavior: every sampled position selected for importance weighting
must retain the complete conditioning under which its stored logprob was
captured. A changed-context suffix can remain ordinary conditioning, but must
not reuse the old probability as an exact sampled probability under the new
prefix. Preserve the full original sampled source in another history or refuse
when exact attribution cannot be established; avoid silently dropping observed
samples. The experiment-local adapter is being tightened separately while the
appropriate shared history/attribution behavior is reviewed.

Start with _retained_output_suffix and _tokenize_exact_projected_chat_history in the tokenization path, then run the synthetic regression in tests.unit.trajectories.test_tokenize using _chat_exchange. Review how retained suffixes receive original logprobs and flags, and confirm that changed-context suffixes are not selected as exact sampled positions unless their complete captured causal prefix is preserved; implementation remains pending the scope decision.