agentscope-ai / agentscope-ai/Trinity-RFT

[FEATURE] Prompt-length dataset pre-filtering safeguard or check `max_model_len` before forward

Offen
#287 2 Kommentare 0 Reaktionen 0 zugewiesene Personen Auf GitHub ansehen
enhancement
Vorherrschende Sprache
Python
Sterne
701
Forks
79
Ø Merge
8 Std. 7 Min.
Gemergte PRs (30 T.)
1

Beschreibung

Otherwise, one can get an exception / training crash when some outlier prompt happens to be too long.

Part of the root cause:
- https://github.com/vllm-project/vllm/issues/16732

```
(WorkflowRunner pid=42768) ERROR 09-18 02:43:09 [workflow_runner.py:163] File "/tmp/ray/session_2025-09-18_02-35-21_268823_22052/runtime_resources/working_dir_files/_ray_pkg_83ec426c3cf029b3/.venv/lib/python3.12/site-packages/vllm/v1/engine/async_llm.py", line 237, in add_request
(WorkflowRunner pid=42768) ERROR 09-18 02:43:09 [workflow_runner.py:163] prompt_str, request = self.processor.process_inputs(
(WorkflowRunner pid=42768) ERROR 09-18 02:43:09 [workflow_runner.py:163] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(WorkflowRunner pid=42768) ERROR 09-18 02:43:09 [workflow_runner.py:163] File "/tmp/ray/session_2025-09-18_02-35-21_268823_22052/runtime_resources/working_dir_files/_ray_pkg_83ec426c3cf029b3/.venv/lib/python3.12/site-packages/vllm/v1/engine/processor.py", line 266, in process_inputs
(WorkflowRunner pid=42768) ERROR 09-18 02:43:09 [workflow_runner.py:163] self._validate_model_inputs(processed_inputs, lora_request)
(WorkflowRunner pid=42768) ERROR 09-18 02:43:09 [workflow_runner.py:163] File "/tmp/ray/session_2025-09-18_02-35-21_268823_22052/runtime_resources/working_dir_files/_ray_pkg_83ec426c3cf029b3/.venv/lib/python3.12/site-packages/vllm/v1/engine/processor.py", line 365, in _validate_model_inputs
(WorkflowRunner pid=42768) ERROR 09-18 02:43:09 [workflow_runner.py:163] self._validate_model_input(decoder_inputs,
(WorkflowRunner pid=42768) ERROR 09-18 02:43:09 [workflow_runner.py:163] File "/tmp/ray/session_2025-09-18_02-35-21_268823_22052/runtime_resources/working_dir_files/_ray_pkg_83ec426c3cf029b3/.venv/lib/python3.12/site-packages/vllm/v1/engine/processor.py", line 418, in _validate_model_input
(WorkflowRunner pid=42768) ERROR 09-18 02:43:09 [workflow_runner.py:163] raise ValueError(
(WorkflowRunner pid=42768) ERROR 09-18 02:43:09 [workflow_runner.py:163] ValueError: The decoder prompt (length 1282) is longer than the maximum model length of 1280. Make sure that `max_model_len` is no smaller than the number of text tokens.
(WorkflowRunner pid=42768) ERROR 09-18 02:43:09 [workflow_runner.py:163]
```

Beitragsleitfaden

Beitragsleitfaden öffnen

Bewertung

Dieses Issue wurde noch nicht bewertet.

Neue Issues direkt in Ihr Postfach

Eine kurze Übersicht über anfängerfreundliche GitHub-Issues.