agentscope-ai / agentscope-ai/Trinity-RFT

[FEATURE] Prompt-length dataset pre-filtering safeguard or check `max_model_len` before forward

未關閉
#287 2 則留言 0 個 reaction 已指派 0 人 在 GitHub 檢視
enhancement
主要語言
Python
星號
701
分支
79
平均合併
8 小時 7 分鐘
30 天內合併 PR
1

描述

Otherwise, one can get an exception / training crash when some outlier prompt happens to be too long.

Part of the root cause:
- https://github.com/vllm-project/vllm/issues/16732

```
(WorkflowRunner pid=42768) ERROR 09-18 02:43:09 [workflow_runner.py:163] File "/tmp/ray/session_2025-09-18_02-35-21_268823_22052/runtime_resources/working_dir_files/_ray_pkg_83ec426c3cf029b3/.venv/lib/python3.12/site-packages/vllm/v1/engine/async_llm.py", line 237, in add_request
(WorkflowRunner pid=42768) ERROR 09-18 02:43:09 [workflow_runner.py:163] prompt_str, request = self.processor.process_inputs(
(WorkflowRunner pid=42768) ERROR 09-18 02:43:09 [workflow_runner.py:163] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(WorkflowRunner pid=42768) ERROR 09-18 02:43:09 [workflow_runner.py:163] File "/tmp/ray/session_2025-09-18_02-35-21_268823_22052/runtime_resources/working_dir_files/_ray_pkg_83ec426c3cf029b3/.venv/lib/python3.12/site-packages/vllm/v1/engine/processor.py", line 266, in process_inputs
(WorkflowRunner pid=42768) ERROR 09-18 02:43:09 [workflow_runner.py:163] self._validate_model_inputs(processed_inputs, lora_request)
(WorkflowRunner pid=42768) ERROR 09-18 02:43:09 [workflow_runner.py:163] File "/tmp/ray/session_2025-09-18_02-35-21_268823_22052/runtime_resources/working_dir_files/_ray_pkg_83ec426c3cf029b3/.venv/lib/python3.12/site-packages/vllm/v1/engine/processor.py", line 365, in _validate_model_inputs
(WorkflowRunner pid=42768) ERROR 09-18 02:43:09 [workflow_runner.py:163] self._validate_model_input(decoder_inputs,
(WorkflowRunner pid=42768) ERROR 09-18 02:43:09 [workflow_runner.py:163] File "/tmp/ray/session_2025-09-18_02-35-21_268823_22052/runtime_resources/working_dir_files/_ray_pkg_83ec426c3cf029b3/.venv/lib/python3.12/site-packages/vllm/v1/engine/processor.py", line 418, in _validate_model_input
(WorkflowRunner pid=42768) ERROR 09-18 02:43:09 [workflow_runner.py:163] raise ValueError(
(WorkflowRunner pid=42768) ERROR 09-18 02:43:09 [workflow_runner.py:163] ValueError: The decoder prompt (length 1282) is longer than the maximum model length of 1280. Make sure that `max_model_len` is no smaller than the number of text tokens.
(WorkflowRunner pid=42768) ERROR 09-18 02:43:09 [workflow_runner.py:163]
```

貢獻指南

開啟貢獻指南

評估

這個 Issue 還沒有評估資料。

把新 issue 寄到你的電子郵件信箱

精選適合新手參與的 GitHub issue 摘要。