modelscope / modelscope/ms-swift
命令行swift infer与python脚本推理结果不一致
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 15.7k
- Forks
- 1.7k
- Avg merge
- 1d 16h
- Merged PRs (30d)
- 136
Description
命令行swift infer与python脚本推理结果不一致, 微调的qwen 2.5 vl模型, 采用seq_cls任务
命令行如下
swift infer \ --model /checkpoint-7814/ \ --val_dataset seq_15cls.json \ --infer_backend pt \ --logprobs true \ --use_chat_template false \ --metric acc \ --max_new_tokens 2 \ --max_batch_size 1 \ --result_path seq_15cls_result.json
python脚本如下
model_args = InferArguments(model=MODEL_PATH, infer_backend='pt', task_type='seq_cls', num_labels=15)
model, template = prepare_model_template(model_args)
engine = PtEngine.from_model_template(model, template, max_batch_size=1)
req_cfg = RequestConfig(max_tokens=2, temperature=0)
out = engine.infer([InferRequest(messages=msgs, images=imgs)], req_cfg)[0]
命令行推理结果
"logprobs":{"content":[{"index":0,"logprobs":-0.484375,"top_logprobs":[{"index":0,"logprob":-0.484375},{"index":3,"logprob":-1.125},{"index":12,"logprob":-3.578125},{"index":5,"logprob":-4.28125},{"index":8,"logprob":-4.6875},{"index":14,"logprob":-5.53125},{"index":11,"logprob":-6.53125},{"index":7,"logprob":-6.53125},{"index":1,"logprob":-7.3125},{"index":13,"logprob":-7.34375},{"index":4,"logprob":-7.625},{"index":2,"logprob":-8.9375},{"index":9,"logprob":-23.125},{"index":6,"logprob":-23.375},{"index":10,"logprob":-23.75}]}]}
Python脚本推理结果
"logprobs":{"content":[{"top_logprobs":[{"index":0,"logprob":-0.52734375},{"index":3,"logprob":-1.0546875},{"index":12,"logprob":-3.515625},{"index":5,"logprob":-4.21875},{"index":8,"logprob":-4.8125},{"index":14,"logprob":-5.46875},{"index":11,"logprob":-6.375},{"index":7,"logprob":-6.625},{"index":13,"logprob":-6.96875},{"index":4,"logprob":-7.1875},{"index":1,"logprob":-7.3125},{"index":2,"logprob":-8.9375},{"index":9,"logprob":-22.75},{"index":10,"logprob":-22.75},{"index":6,"logprob":-22.875}]}]}
请问该如何解决, 是python脚本问题吗
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reproducing the CLI swift infer command and the Python path using InferArguments, prepare_model_template, PtEngine.from_model_template, and engine.infer with the reported Qwen 2.5 VL sequence-classification settings. Compare how logprobs, max_new_tokens/max_tokens, and sampling configuration are passed; done means identifying the differing path or documenting why the reported probabilities differ, with a regression test if an existing test location is found.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100