modelscope / modelscope/ms-swift
Chord微调报错：Fatal Python error: none_dealloc: deallocating None: bug likely caused by a refcount error in a C extension
Open
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 15.7k
- Forks
- 1.7k
- Avg merge
- 1d 16h
- Merged PRs (30d)
- 136
Description
Describe the bug
What the bug is, and how to reproduce, better with screenshots(描述bug以及复现过程，最好有截图)
使用Chord进行微调时，在 282 steps 时，出现 Fatal Python error: none_dealloc: deallocating None: bug likely caused by a refcount error in a C extension 错误。此报错在之前ms-swift和vllm的issue里都有出现并已解决，但是仍然出现。
报错信息
Fatal Python error: none_dealloc: deallocating None: bug likely caused by a refcount error in a C extension
Python runtime state: initialized
Thread 0x00007f6f10ff9700 (most recent call first):
Thread 0x00007f6f117fa700 (most recent call first):
Thread 0x00007f6f11ffb700 (most recent call first):
File "/home/ubuntu/anaconda3/envs/swift_env_0927/lib/python3.11/threading.py", line 320 in wait
File "/home/ubuntu/anaconda3/envs/swift_env_0927/lib/python3.11/multiprocessing/queues.py", line 231 in _feed
File "/home/ubuntu/anaconda3/envs/swift_env_0927/lib/python3.11/threading.py", line 975 in run
File "/home/ubuntu/anaconda3/envs/swift_env_0927/lib/python3.11/threading.py", line 1038 in _bootstrap_inner
File "/home/ubuntu/anaconda3/envs/swift_env_0927/lib/python3.11/threading.py", line 995 in _bootstrap
Thread 0x00007f6f127fc700 (most recent call first):
File "/home/ubuntu/anaconda3/envs/swift_env_0927/lib/python3.11/threading.py", line 320 in wait
File "/home/ubuntu/anaconda3/envs/swift_env_0927/lib/python3.11/multiprocessing/queues.py", line 231 in _feed
File "/home/ubuntu/anaconda3/envs/swift_env_0927/lib/python3.11/threading.py", line 975 in run
File "/home/ubuntu/anaconda3/envs/swift_env_0927/lib/python3.11/threading.py", line 1038 in _bootstrap_inner
File "/home/ubuntu/anaconda3/envs/swift_env_0927/lib/python3.11/threading.py", line 995 in _bootstrap
Thread 0x00007f6f12ffd700 (most recent call first):
File "/home/ubuntu/anaconda3/envs/swift_env_0927/lib/python3.11/threading.py", line 320 in wait
File "/home/ubuntu/anaconda3/envs/swift_env_0927/lib/python3.11/multiprocessing/queues.py", line 231 in _feed
File "/home/ubuntu/anaconda3/envs/swift_env_0927/lib/python3.11/threading.py", line 975 in run
File "/home/ubuntu/anaconda3/envs/swift_env_0927/lib/python3.11/threading.py", line 1038 in _bootstrap_inner
File "/home/ubuntu/anaconda3/envs/swift_env_0927/lib/python3.11/threading.py", line 995 in _bootstrap
运行脚本
export CHORD_SYSTEM_PROMPT="You are a helpful assistant that solves MATH problems.
You should first think about the reasoning process in mind and then provide the user with the answer.
You should present your reasoning process using the format: <think>\n...your reasoning process here... </think>\n"
CUDA_VISIBLE_DEVICES=0,1 \
NPROC_PER_NODE=2 \
swift rlhf \
--rlhf_type grpo \
--model Qwen/Qwen3-1.7B \
--dataset AI-MO/NuminaMath-TIR \
--load_from_cache_file true \
--torch_dtype bfloat16 \
--beta 0.0 \
--steps_per_generation 1 \
--num_train_epochs 1 \
--per_device_train_batch_size 2 \
--gradient_accumulation_steps 16 \
--chord_sft_per_device_train_batch_size 1 \
--chord_sft_dataset AI-MO/NuminaMath-TIR \
--chord_enable_phi_function false \
--chord_mu_warmup_steps 0 \
--chord_mu_decay_steps 200 \
--chord_mu_peak 0.9 \
--chord_mu_valley 0.05 \
--num_generations 2 \
--train_type lora \
--reward_funcs accuracy \
--system "$CHORD_SYSTEM_PROMPT" \
--use_vllm true \
--vllm_mode colocate \
--vllm_gpu_memory_utilization 0.7 \
--vllm_max_model_len 4096 \
--max_completion_length 2048 \
--overlong_filter true \
--offload_optimizer true \
--offload_model true \
--sleep_level 1 \
--save_steps 1000 \
--learning_rate 1e-6 \
--save_total_limit 2 \
--logging_steps 1 \
--warmup_ratio 0.05 \
--dataloader_num_workers 4 \
--deepspeed zero3 \
--log_completions true \
--report_to wandb
Your hardware and system info
Write your system info like CUDA version/system/GPU/torch version here(在这里给出硬件信息和系统信息，如CUDA版本，系统，GPU型号和torch版本等)
系统：Ubuntu 20.04.6 LTS
GPU：4090*2 驱动：570.190
CUDA：10.1
pip list
Package Version Editable project location
--------------------------------- ------------- -------------------------------------------
absl-py 2.3.1
accelerate 1.10.1
addict 2.4.0
aiofiles 24.1.0
aiohappyeyeballs 2.6.1
aiohttp 3.12.15
aiosignal 1.4.0
aliyun-python-sdk-core 2.16.0
aliyun-python-sdk-kms 2.16.5
annotated-types 0.7.0
antlr4-python3-runtime 4.13.2
anyio 4.11.0
astor 0.8.1
attrdict 2.0.1
attrs 25.3.0
binpacking 1.5.2
blake3 1.0.6
Brotli 1.1.0
cachetools 6.2.0
cbor2 5.7.0
certifi 2025.8.3
cffi 2.0.0
charset-normalizer 3.4.3
click 8.3.0
cloudpickle 3.1.1
compressed-tensors 0.11.0
contourpy 1.3.3
cpm-kernels 1.0.11
crcmod 1.7
cryptography 46.0.1
cupy-cuda12x 13.6.0
cycler 0.12.1
dacite 1.9.2
datasets 3.6.0
deepspeed 0.17.5
depyf 0.19.0
dill 0.3.8
diskcache 5.6.3
distro 1.9.0
dnspython 2.8.0
einops 0.8.1
email-validator 2.3.0
fastapi 0.117.1
fastapi-cli 0.0.13
fastapi-cloud-cli 0.2.1
fastrlock 0.8.3
ffmpy 0.6.1
filelock 3.19.1
fonttools 4.60.0
frozendict 2.4.6
frozenlist 1.7.0
fsspec 2025.3.0
future 1.0.0
gguf 0.17.1
gitdb 4.0.12
GitPython 3.1.45
gradio 5.47.2
gradio_client 1.13.3
groovy 0.1.2
grpcio 1.75.1
h11 0.16.0
hf-xet 1.1.10
hjson 3.1.0
httpcore 1.0.9
httptools 0.6.4
httpx 0.28.1
huggingface-hub 0.35.1
idna 3.10
importlib_metadata 8.7.0
interegular 0.3.3
jieba 0.42.1
Jinja2 3.1.6
jiter 0.11.0
jmespath 0.10.0
joblib 1.5.2
json_repair 0.51.0
jsonschema 4.25.1
jsonschema-specifications 2025.9.1
kiwisolver 1.4.9
lark 1.2.2
latex2sympy2_extended 1.0.6
llguidance 0.7.30
llvmlite 0.44.0
lm-format-enforcer 0.11.3
Markdown 3.9
markdown-it-py 4.0.0
MarkupSafe 3.0.2
math-verify 0.5.2
matplotlib 3.10.6
mdurl 0.1.2
mistral_common 1.8.5
modelscope 1.30.0
mpmath 1.3.0
ms_swift 3.9.0.dev0
msgpack 1.1.1
msgspec 0.19.0
multidict 6.6.4
multiprocess 0.70.16
networkx 3.5
ninja 1.13.0
nltk 3.9.1
numba 0.61.2
numpy 2.2.6
nvidia-cublas-cu12 12.8.4.1
nvidia-cuda-cupti-cu12 12.8.90
nvidia-cuda-nvrtc-cu12 12.8.93
nvidia-cuda-runtime-cu12 12.8.90
nvidia-cudnn-cu12 9.10.2.21
nvidia-cufft-cu12 11.3.3.83
nvidia-cufile-cu12 1.13.1.3
nvidia-curand-cu12 10.3.9.90
nvidia-cusolver-cu12 11.7.3.90
nvidia-cusparse-cu12 12.5.8.93
nvidia-cusparselt-cu12 0.7.1
nvidia-ml-py 13.580.82
nvidia-nccl-cu12 2.27.3
nvidia-nvjitlink-cu12 12.8.93
nvidia-nvtx-cu12 12.8.90
openai 1.109.1
openai-harmony 0.0.4
opencv-python-headless 4.12.0.88
orjson 3.11.3
oss2 2.19.1
outlines_core 0.2.11
packaging 25.0
pandas 2.3.2
partial-json-parser 0.2.1.1.post6
peft 0.17.1
pillow 11.3.0
pip 25.2
platformdirs 4.4.0
prometheus_client 0.23.1
prometheus-fastapi-instrumentator 7.1.0
propcache 0.3.2
protobuf 6.32.1
psutil 7.1.0
py-cpuinfo 9.0.0
pyarrow 21.0.0
pybase64 1.4.2
pycountry 24.6.1
pycparser 2.23
pycryptodome 3.23.0
pydantic 2.11.9
pydantic_core 2.33.2
pydantic-extra-types 2.10.5
pydub 0.25.1
Pygments 2.19.2
pyparsing 3.2.5
python-dateutil 2.9.0.post0
python-dotenv 1.1.1
python-json-logger 3.3.0
python-multipart 0.0.20
pytz 2025.2
PyYAML 6.0.3
pyzmq 27.1.0
ray 2.49.2
referencing 0.36.2
regex 2025.9.18
requests 2.32.5
rich 14.1.0
rich-toolkit 0.15.1
rignore 0.6.4
rouge 1.0.1
rpds-py 0.27.1
ruff 0.13.2
safehttpx 0.1.6
safetensors 0.6.2
scipy 1.16.2
semantic-version 2.10.0
sentencepiece 0.2.1
sentry-sdk 2.39.0
setproctitle 1.3.7
setuptools 78.1.1
shellingham 1.5.4
simplejson 3.20.2
six 1.17.0
smmap 5.0.2
sniffio 1.3.1
sortedcontainers 2.4.0
soundfile 0.13.1
soxr 1.0.0
starlette 0.48.0
sympy 1.14.0
tensorboard 2.20.0
tensorboard-data-server 0.7.2
tiktoken 0.11.0
tokenizers 0.22.1
tomlkit 0.13.3
torch 2.8.0
torchaudio 2.8.0
torchvision 0.23.0
tqdm 4.67.1
transformers 4.56.2
transformers-stream-generator 0.0.5
triton 3.4.0
trl 0.20.0
typer 0.19.2
typing_extensions 4.15.0
typing-inspection 0.4.1
tzdata 2025.2
urllib3 2.5.0
uvicorn 0.37.0
uvloop 0.21.0
vllm 0.10.2
wandb 0.22.0
watchfiles 1.1.0
websockets 15.0.1
Werkzeug 3.1.3
wheel 0.45.1
xformers 0.0.32.post1
xgrammar 0.1.23
xxhash 3.5.0
yarl 1.20.1
zipp 3.23.0
zstandard 0.25.0
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reproducing the reported swift rlhf Chord fine-tuning command and note whether the crash still occurs at 282 steps. Read the Python 3.11 threading.py and multiprocessing/queues.py stack locations, then trace the involved training and multiprocessing components; done means identifying and reliably preventing the none_dealloc crash.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, pytorch
- Domain
- distributed-systems, machine-learning
- Issue type
- Bug
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 22/100