基于megatron训练qwen3-omni gspo，模型输出存在：截断、不遵循格式、重复等现象

Checklist / 检查清单

• I have searched existing issues, and this is a new bug report. / 我已经搜索过现有的 issues，确认这是一个新的 bug report。

Bug Description / Bug 描述

根据自动保存的completions.jsonl，可以查看模型的输出，输出存在以下异常

1. 无意义的输出："<tool_call>"
2. 重复：The officer's claim of a gas leak is the ræð²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²²"
3. 不遵循格式，我的系统提示词为examples/train/grpo/prompt.txt，按理说qwen3-omni应该具备一定指令遵循的能力，但是输出没有按照格式

How to Reproduce / 如何复现

我的运行脚本
MAX_PIXELS=1003520 \ NPROC_PER_NODE=8 \ ENABLE_AUDIO_OUTPUT=0 \ FPS_MAX_FRAMES=8 \ ENCODE_TIMEOUT=30 \ USE_AUDIO_IN_VIDEO=1 \ CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7 \ megatron rlhf \ --rlhf_type grpo \ --model Qwen3-Omni-30B-A3B-Instruct \ --tuner_type lora \ --dataset ms-swift-datasets/train_with_audio.jsonl \ --reward_funcs external_r1v_acc format \ --reward_weights 1 0.5 \ --external_plugins /examples/train/grpo/plugin/plugin.py \ \ --bf16 true \ --num_train_epochs 1 \ --micro_batch_size 1 \ --tensor_model_parallel_size 4 \ --expert_model_parallel_size 4 \ --pipeline_model_parallel_size 1 \ --context_parallel_size 1 \ --sequence_parallel true \ --attention_backend flash \ --padding_free false \ \ --recompute_granularity selective \ --offload_model true \ --offload_optimizer true \ --optimizer_cpu_offload true \ --use_precision_aware_optimizer \ --sleep_level 0 \ \ --moe_permute_fusion true \ --moe_grouped_gemm true \ --moe_shared_expert_overlap true \ --moe_aux_loss_coeff 1e-3 \ \ --freeze_vit true \ --freeze_aligner true \ --freeze_parameters talker code2wav \ --vit_gradient_checkpointing true \ \ --lr 1e-6 \ --min_lr 1e-7 \ --lr_warmup_fraction 0.05 \ --beta 0.00 \ --epsilon 3e-4 \ --epsilon_high 4e-4 \ --loss_type grpo \ --importance_sampling_level sequence \ \ --save_total_limit 20 \ --save_steps 50 \ --eval_steps 50 \ --logging_steps 1 \ --no_save_optim true \ --no_save_rng true \ --output_dir ms-swift/outputs \ \ --global_batch_size 16 \ --num_generations 4 \ --steps_per_generation 1 \ --max_length 10240 \ --max_completion_length 896 \ --max_pixels 300000 \ \ --dataset_num_proc 2 \ --dataloader_num_workers 0 \ --temperature 1.0 \ --top_p 0.99 \ --top_k 50 \ --system 'examples/train/grpo/prompt.txt' \ --log_completions true \ --dynamic_sample false \ --overlong_filter true \ --truncation_strategy delete \ --seed 42 \ \ --use_vllm true \ --vllm_mode colocate \ --vllm_tensor_parallel_size 4 \ --vllm_gpu_memory_utilization 0.25 \ --vllm_max_model_len 10240 \ --vllm_mm_processor_cache_gb 10 \ --vllm_enable_prefix_caching true

我的库版本

Package Version Editable project location

absl-py 2.4.0
accelerate 1.12.0
addict 2.4.0
aiofiles 24.1.0
aiohappyeyeballs 2.6.1
aiohttp 3.13.3
aiohttp-cors 0.8.1
aiosignal 1.4.0
aliyun-python-sdk-core 2.16.0
aliyun-python-sdk-kms 2.16.5
annotated-doc 0.0.4
annotated-types 0.7.0
anthropic 0.71.0
antlr4-python3-runtime 4.9.3
anyio 4.12.1
apache-tvm-ffi 0.1.6
astor 0.8.1
attrdict 2.0.1
attrs 25.4.0
audioread 3.1.0
av 16.1.0
binpacking 2.0.1
blake3 1.0.8
brotli 1.2.0
cachetools 7.0.1
cbor2 5.8.0
certifi 2026.1.4
cffi 2.0.0
charset-normalizer 3.4.4
click 8.2.1
cloudpickle 3.1.2
cmake 4.2.1
codetiming 1.4.0
colorful 0.5.8
compressed-tensors 0.12.2
contourpy 1.3.3
cpm-kernels 1.0.11
crcmod 1.7
cryptography 46.0.5
cuda-bindings 12.9.0
cuda-pathfinder 1.4.0
cuda-python 12.9.0
cupy-cuda12x 14.0.1
cycler 0.12.1
dacite 1.9.2
datasets 3.6.0
debugpy 1.8.20
decorator 5.2.1
decord 0.6.0
deepspeed 0.18.7
Deprecated 1.3.1
depyf 0.20.0
dill 0.3.8
diskcache 5.6.3
distlib 0.4.0
distro 1.9.0
dnspython 2.8.0
docopt 0.6.2
docstring_parser 0.17.0
einops 0.8.2
email-validator 2.3.0
Farama-Notifications 0.0.4
fastapi 0.129.0
fastapi-cli 0.0.24
fastapi-cloud-cli 0.14.0
fastar 0.8.0
ffmpeg-python 0.2.0
ffmpy 1.0.0
filelock 3.24.3
flash_attn 2.8.3
flashinfer-python 0.5.3
fonttools 4.61.1
frozenlist 1.8.0
fsspec 2025.3.0
future 1.0.0
genson 1.3.0
gguf 0.17.1
gitdb 4.0.12
GitPython 3.1.46
google-api-core 2.30.0
google-auth 2.48.0
googleapis-common-protos 1.72.0
gradio 5.50.0
gradio_client 1.14.0
groovy 0.1.2
grouped_gemm 0.3.0
grpcio 1.78.1
gym 0.26.2
gym-notices 0.1.0
gym-sokoban 0.0.6
gymnasium 1.2.3
h11 0.16.0
hf-xet 1.4.2
hjson 3.1.0
hope 3.6.6
httpcore 1.0.9
httptools 0.7.1
httpx 0.28.1
huggingface_hub 0.36.2
hydra-core 1.3.2
idna 3.11
ImageIO 2.37.2
importlib_metadata 8.7.1
iniconfig 2.3.0
interegular 0.3.3
isort 7.0.0
jieba 0.42.1
Jinja2 3.1.6
jiter 0.13.0
jmespath 0.10.0
joblib 1.5.3
json_repair 0.58.4
jsonlines 4.0.0
jsonpath-ng 1.7.0
jsonschema 4.26.0
jsonschema-specifications 2025.9.1
kiwisolver 1.4.9
lark 1.2.2
latex2sympy2 1.5.4
latex2sympy2_extended 1.11.0
lazy_loader 0.4
librosa 0.11.0
linkify-it-py 2.1.0
llguidance 1.3.0
llvmlite 0.44.0
lm-format-enforcer 0.11.3
logger 1.4
loguru 0.7.3
loralib 0.1.2
Markdown 3.10.2
markdown-it-py 4.0.0
MarkupSafe 3.0.3
math-verify 0.9.0
matplotlib 3.10.8
mcore_adapter 0.8.0
mdit-py-plugins 0.5.0
mdurl 0.1.2
megatron-core 0.15.0
memray 1.19.1
mistral_common 1.9.1
ml_dtypes 0.5.4
model-hosting-container-standards 0.1.9
modelscope 1.34.0
more-itertools 10.8.0
mpmath 1.3.0
ms_swift 4.1.0.dev0
msgpack 1.1.2
msgspec 0.20.0
multidict 6.7.1
multiprocess 0.70.16
nest-asyncio 1.6.0
networkx 3.6.1
ninja 1.13.0
nltk 3.9.3
numba 0.61.2
numpy 1.26.4
nvdlfw_inspect 0.2.2
nvidia-cublas-cu12 12.9.1.4
nvidia-cuda-cupti-cu12 12.9.79
nvidia-cuda-nvrtc-cu12 12.9.86
nvidia-cuda-runtime-cu12 12.9.79
nvidia-cudnn-cu12 9.10.2.21
nvidia-cudnn-frontend 1.17.0
nvidia-cufft-cu12 11.4.1.4
nvidia-cufile-cu12 1.14.1.1
nvidia-curand-cu12 10.3.10.19
nvidia-cusolver-cu12 11.7.5.82
nvidia-cusparse-cu12 12.5.10.65
nvidia-cusparselt-cu12 0.7.1
nvidia-cutlass-dsl 4.3.4
nvidia-ml-py 13.590.48
nvidia-nccl-cu12 2.27.5
nvidia-nvjitlink-cu12 12.9.86
nvidia-nvshmem-cu12 3.3.20
nvidia-nvtx-cu12 12.9.79
nvitop 1.6.2
omegaconf 2.3.0
onnx 1.20.1
onnx-ir 0.2.0
onnxscript 0.6.2
openai 2.21.0
openai-harmony 0.0.8
openai-whisper 20250625
opencensus 0.11.4
opencensus-context 0.1.3
opencv-python-headless 4.13.0.92
opentelemetry-api 1.39.1
opentelemetry-exporter-prometheus 0.60b1
opentelemetry-proto 1.39.1
opentelemetry-sdk 1.39.1
opentelemetry-semantic-conventions 0.60b1
orjson 3.11.7
oss2 2.19.1
outlines 0.0.46
outlines_core 0.2.11
packaging 26.0
pandas 2.3.3
partial-json-parser 0.2.1.1.post7
peft 0.18.1
pexpect 4.9.0
pillow 11.3.0
pip 24.0
platformdirs 4.8.0
pluggy 1.6.0
ply 3.11
pooch 1.9.0
prettytable 3.17.0
prometheus_client 0.24.1
prometheus-fastapi-instrumentator 7.1.0
propcache 0.4.1
proto-plus 1.27.1
protobuf 6.33.5
psutil 7.2.2
ptyprocess 0.7.0
py-cpuinfo 9.0.0
py-spy 0.4.1
pyairports 0.0.1
pyarrow 23.0.0
pyasn1 0.6.2
pyasn1_modules 0.4.2
pybase64 1.4.3
pybind11 3.0.2
pycountry 24.6.1
pycparser 3.0
pycryptodome 3.23.0
pydantic 2.12.3
pydantic_core 2.41.4
pydantic-extra-types 2.11.0
pydantic-settings 2.13.1
pydub 0.25.1
pygame 2.6.1
Pygments 2.19.2
pyparsing 3.3.2
pytest 9.0.2
python-dateutil 2.9.0.post0
python-discovery 1.1.0
python-dotenv 1.2.1
python-json-logger 4.0.0
python-multipart 0.0.22
pytz 2025.2
PyYAML 6.0.3
pyzmq 27.1.0
qwen-omni-utils 0.0.9
qwen-vl-utils 0.0.14
ray 2.48.0
referencing 0.37.0
regex 2026.1.15
requests 2.32.5
rich 14.3.3
rich-toolkit 0.19.7
rignore 0.7.6
rouge 1.0.1
rpds-py 0.30.0
rsa 4.9.1
ruff 0.15.5
safehttpx 0.1.7
safetensors 0.7.0
scikit-learn 1.8.0
scipy 1.17.0
semantic-version 2.10.0
sentencepiece 0.2.1
sentry-sdk 2.52.0
setproctitle 1.3.7
setuptools 80.0.0
setuptools-scm 9.2.2
shellingham 1.5.4
simplejson 3.20.2
six 1.17.0
smart_open 7.5.1
smmap 5.0.2
sniffio 1.3.1
sortedcontainers 2.4.0
soundfile 0.13.1
soxr 1.0.0
starlette 0.52.1
sty 1.0.6
supervisor 4.2.5
sympy 1.14.0
tabulate 0.9.0
tensorboard 2.19.0
tensorboard-data-server 0.7.2
tensordict 0.9.0
textual 8.0.2
threadpoolctl 3.6.0
thrift 0.21.0
tiktoken 0.12.0
tokenizers 0.22.2
tomlkit 0.13.3
torch 2.9.0+cu129
torchaudio 2.9.0+cu129
torchcodec 0.9.0
torchvision 0.24.0+cu129
tqdm 4.67.3
transformer_engine 2.12.0
transformer_engine_cu12 2.12.0
transformer_engine_torch 2.12.0
transformers 4.57.6
transformers-stream-generator 0.0.5
triton 3.5.0
trl 0.28.0
typeguard 4.4.2
typer 0.24.1
typer-slim 0.15.3
typing_extensions 4.15.0
typing-inspection 0.4.2
tyro 1.0.6
tzdata 2025.3
uc-micro-py 2.0.0
urllib3 2.6.3
uvicorn 0.40.0
uvloop 0.22.1
validators 0.35.0
virtualenv 21.1.0
vllm 0.12.0
wandb 0.25.0
watchfiles 1.1.1
wcwidth 0.6.0
websockets 15.0.1
Werkzeug 3.1.6
wheel 0.43.0
whisper 1.1.10
wrapt 2.1.1
xgrammar 0.1.27
xxhash 3.6.0
yarl 1.22.0
zipp 3.23.0
zstandard 0.25.0

Additional Information / 补充信息

No response

Start with the reported completions.jsonl outputs and the reproduction command, then compare the behavior against examples/train/grpo/prompt.txt and examples/train/grpo/plugin/plugin.py. Check how the Megatron GRPO run handles the Qwen3-Omni prompt, rewards, truncation, and generation settings. Done means the same setup no longer produces truncated, repeated, or incorrectly formatted completions.