sgl-project / sgl-project/SpecForge
[Bug] Found an error when running the example.
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 1.2k
- Forks
- 347
- Avg merge
- 4d 1h
- Merged PRs (30d)
- 41
Description
Checklist
- 1. I have searched related issues but cannot get the expected help.
- 2. The bug has not been fixed in the latest version.
- 3. Please note that if the bug-related issue you submitted lacks corresponding environment info and a minimal reproducible demo, it will be challenging for us to reproduce and resolve the issue, reducing the likelihood of receiving feedback.
- 4. If the issue you raised is not a bug but a question, please raise a discussion at https://github.com/sgl-project/SpecForge/discussions/new/choose Otherwise, it will be closed.
- 5. Please use English, otherwise it will be closed.
Describe the bug
An error occurred when running the example according to the documentation.
Environment:
torch 2.8.0+cpu
torch_npu 2.8.0.post2
torchao 0.9.0
torchaudio 2.8.0
torchvision 0.23.0
sglang 0.5.9
sglang-router 0.3.2
datasets 4.8.5
transformers 4.57.1
Reproduction
python scripts/prepare_data.py --dataset sharegpt
bash ./examples/run_llama3.1_8b_eagle3_online.sh 4 2
Error message:
rank0]: Traceback (most recent call last):
[rank0]: File "/usr/local/python3.11.14/lib/python3.11/site-packages/multiprocess/pool.py", line 125, in worker
[rank0]: result = (True, func(*args, **kwds))
[rank0]: ^^^^^^^^^^^^^^^^^^^
[rank0]: File "/usr/local/python3.11.14/lib/python3.11/site-packages/datasets/utils/py_utils.py", line 585, in _write_generator_to_queue
[rank0]: for i, result in enumerate(func(**kwargs)):
[rank0]: File "/usr/local/python3.11.14/lib/python3.11/site-packages/datasets/arrow_dataset.py", line 3998, in _map_single
[rank0]: for i, batch in iter_outputs(shard_iterable):
[rank0]: File "/usr/local/python3.11.14/lib/python3.11/site-packages/datasets/arrow_dataset.py", line 3951, in iter_outputs
[rank0]: yield i, apply_function(example, i, offset=offset)
[rank0]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[rank0]: File "/usr/local/python3.11.14/lib/python3.11/site-packages/datasets/arrow_dataset.py", line 3872, in apply_function
[rank0]: processed_inputs = function(*fn_args, *additional_args, **fn_kwargs)
[rank0]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[rank0]: File "/workspace/SpecForge/specforge/data/preprocessing.py", line 412, in preprocess_function
[rank0]: processed = preprocess_conversations(
[rank0]: ^^^^^^^^^^^^^^^^^^^^^^^^^
[rank0]: File "/workspace/SpecForge/specforge/data/preprocessing.py", line 166, in preprocess_conversations
[rank0]: input_ids, loss_mask = parser.parse(
[rank0]: ^^^^^^^^^^^^^
[rank0]: File "/workspace/SpecForge/specforge/data/parse.py", line 197, in parse
[rank0]: conversation = self.apply_chat_template(messages, tool=tool, **kwargs)
[rank0]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[rank0]: File "/workspace/SpecForge/specforge/data/parse.py", line 123, in apply_chat_template
[rank0]: conversation = self.tokenizer.apply_chat_template(
[rank0]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[rank0]: File "/usr/local/python3.11.14/lib/python3.11/site-packages/transformers/tokenization_utils_base.py", line 1666, in apply_chat_template
[rank0]: rendered_chat, generation_indices = render_jinja_template(
[rank0]: ^^^^^^^^^^^^^^^^^^^^^^
[rank0]: File "/usr/local/python3.11.14/lib/python3.11/site-packages/transformers/utils/chat_template_utils.py", line 539, in render_jinja_template
[rank0]: rendered_chat = compiled_template.render(
[rank0]: ^^^^^^^^^^^^^^^^^^^^^^^^^
[rank0]: File "/usr/local/python3.11.14/lib/python3.11/site-packages/jinja2/environment.py", line 1295, in render
[rank0]: self.environment.handle_exception()
[rank0]: File "/usr/local/python3.11.14/lib/python3.11/site-packages/jinja2/environment.py", line 942, in handle_exception
[rank0]: raise rewrite_traceback_stack(source=source)
[rank0]: File "<template>", line 52, in top-level template code
[rank0]: File "/usr/local/python3.11.14/lib/python3.11/site-packages/jinja2/sandbox.py", line 401, in call
[rank0]: return __context.call(__obj, *args, **kwargs)
[rank0]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[rank0]: File "/usr/local/python3.11.14/lib/python3.11/site-packages/transformers/utils/chat_template_utils.py", line 447, in raise_exception
[rank0]: raise jinja2.exceptions.TemplateError(message)
[rank0]: jinja2.exceptions.TemplateError: Cannot put tools in the first user message when there's no first user message!
Environment
Using quay.io/ascend/sglang:v0.5.9-cann8.5.0-a3
Environment:
torch 2.8.0+cpu
torch_npu 2.8.0.post2
torchao 0.9.0
torchaudio 2.8.0
torchvision 0.23.0
sglang 0.5.9
sglang-router 0.3.2
datasets 4.8.5
transformers 4.57.1
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by running scripts/prepare_data.py and examples/run_llama3.1_8b_eagle3_online.sh with the reported environment, then trace the failure through specforge/data/preprocessing.py and specforge/data/parse.py. Determine why the chat template receives tools without a first user message, and verify that the example completes preprocessing without the reported TemplateError.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, pytorch
- Domain
- data, machine-learning
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 52/100