[Bug]: EAGLE3 dynamic-tree decoding emits tokens outside the requested grammar

System Info

• TensorRT-LLM 1.3.0rc26.dev202609050001, source 17803f6c0cbe8d9fed3ccf46a295b867e3493d3e.
• The same flat-mask path remains in main 3fed8e7103e3c549c56a5053f286bdcff9572706 and v1.3.0rc25; Reproduced on the development revision above; main and v1.3.0rc25 were not executed. Stable v1.2.1 lacks the reported one-model dynamic-tree workers.
• RTX 5070 Laptop, SM120, 8 GiB, driver 610.57.04; Linux x86_64.
• Python 3.12.3, PyTorch 2.12.0+cu130, FlashInfer 0.6.18, XGrammar 0.1.32, Transformers 5.5.4; CUDA 13.0.
• Environment and worker logs.

Information

• The official example scripts
• My own modified scripts

Tasks

• An officially supported task in the examples folder (such as GLUE/SQuAD, ...)
• My own task or dataset (give details below)

Reproduction

EAGLE3 dynamic-tree decoding can emit a prefix outside the requested grammar. Run this script against the TensorRT-LLM version above:

curl -fL https://raw.githubusercontent.com/1sgtpepper/TensorRT-LLM/3b46c569c7d60174d051abd7a3bb60296556d7ad/.github/guided_tree_worker.py -o guided_tree_worker.py
export TLLM_WORKER_USE_SINGLE_PROCESS=1
export TLLM_DEBUG_MODE=1
python guided_tree_worker.py --mode linear --output linear.json
python guided_tree_worker.py --mode tree --output tree.json

The public LLM API runs one-layer Llama/EAGLE3 models with dummy weights, seed 123 and a local 32-token character vocabulary; no weights are downloaded. Settings: grammar aa(bc|de)f, max_tokens=4, tree depth 2, maximum top-K 32, draft budget 64, TRTLLM attention and greedy sampling. Overlap and CUDA graphs are disabled.

Expected behavior

The four-token output is aabc or aade, the only valid prefixes. The feature matrix lists guided decoding with dynamic trees as supported.

actual behavior

The linear worker returns aabc. The tree worker returns aabb and fails with Real worker output violates the finite language.

additional notes

GuidedDecoder._build treats the flat draft list as one history and stops at the first rejected token. Each node instead needs its own ancestor history. With the same generated tree and committed prefix aa, correcting only the per-node masks changes aabb to aabc; restoring the original masks restores aabb.

The linked logs were recorded without TLLM_DEBUG_MODE=1; the reproduction commands enable it.

Before submitting a new issue...

• Make sure you already searched for relevant issues, and checked the documentation and examples for answers to frequently asked questions.

Start by downloading guided_tree_worker.py and running the two commands in the reproduction section with the stated environment. Trace GuidedDecoder._build, focusing on how flat draft tokens are turned into masks and histories. Done means the tree worker emits a valid prefix such as aabc or aade and no longer fails the finite-language check.