modelscope / modelscope/ms-swift

命令行swift infer与python脚本推理结果不一致

Open
#5,881 11 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

stale
Dominant language
Python
Stars
15.7k
Forks
1.7k
Avg merge
1d 16h
Merged PRs (30d)
136

Description

命令行swift infer与python脚本推理结果不一致, 微调的qwen 2.5 vl模型, 采用seq_cls任务
命令行如下
swift infer \ --model /checkpoint-7814/ \ --val_dataset seq_15cls.json \ --infer_backend pt \ --logprobs true \ --use_chat_template false \ --metric acc \ --max_new_tokens 2 \ --max_batch_size 1 \ --result_path seq_15cls_result.json
python脚本如下
model_args = InferArguments(model=MODEL_PATH, infer_backend='pt', task_type='seq_cls', num_labels=15)
model, template = prepare_model_template(model_args)
engine = PtEngine.from_model_template(model, template, max_batch_size=1)
req_cfg = RequestConfig(max_tokens=2, temperature=0)
out = engine.infer([InferRequest(messages=msgs, images=imgs)], req_cfg)[0]

命令行推理结果

"logprobs":{"content":[{"index":0,"logprobs":-0.484375,"top_logprobs":[{"index":0,"logprob":-0.484375},{"index":3,"logprob":-1.125},{"index":12,"logprob":-3.578125},{"index":5,"logprob":-4.28125},{"index":8,"logprob":-4.6875},{"index":14,"logprob":-5.53125},{"index":11,"logprob":-6.53125},{"index":7,"logprob":-6.53125},{"index":1,"logprob":-7.3125},{"index":13,"logprob":-7.34375},{"index":4,"logprob":-7.625},{"index":2,"logprob":-8.9375},{"index":9,"logprob":-23.125},{"index":6,"logprob":-23.375},{"index":10,"logprob":-23.75}]}]}

Python脚本推理结果

"logprobs":{"content":[{"top_logprobs":[{"index":0,"logprob":-0.52734375},{"index":3,"logprob":-1.0546875},{"index":12,"logprob":-3.515625},{"index":5,"logprob":-4.21875},{"index":8,"logprob":-4.8125},{"index":14,"logprob":-5.46875},{"index":11,"logprob":-6.375},{"index":7,"logprob":-6.625},{"index":13,"logprob":-6.96875},{"index":4,"logprob":-7.1875},{"index":1,"logprob":-7.3125},{"index":2,"logprob":-8.9375},{"index":9,"logprob":-22.75},{"index":10,"logprob":-22.75},{"index":6,"logprob":-22.875}]}]}

请问该如何解决, 是python脚本问题吗

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing the CLI swift infer command and the Python path using InferArguments, prepare_model_template, PtEngine.from_model_template, and engine.infer with the reported Qwen 2.5 VL sequence-classification settings. Compare how logprobs, max_new_tokens/max_tokens, and sampling configuration are passed; done means identifying the differing path or documenting why the reported probabilities differ, with a regression test if an existing test location is found.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.