modelscope / modelscope/ms-swift
paddleocr vl在进行swift rollout 模型启动的时候报错
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 15.7k
- Forks
- 1.7k
- Avg merge
- 1d 16h
- Merged PRs (30d)
- 136
Description
Describe the bug
What the bug is, and how to reproduce, better with screenshots(描述bug以及复现过程，最好有截图)
packages/torch/_dynamo/eval_frame.py", line 736, in compile_wrapper
(EngineCore_DP0 pid=141602) return fn(*args, **kwargs)
(EngineCore_DP0 pid=141602) File "??", line 0, in PyObject_Call
(EngineCore_DP0 pid=141602) File "", line 0, in method_vectorcall
(EngineCore_DP0 pid=141602) File "", line 0, in _PyObject_VectorcallTstate.lto_priv.4.cold
(EngineCore_DP0 pid=141602) File "", line 0, in dynamo__custom_eval_frame
(EngineCore_DP0 pid=141602) File "", line 0, in _PyObject_Call.localalias
(EngineCore_DP0 pid=141602) File "", line 0, in slot_tp_call
(EngineCore_DP0 pid=141602) File "??", line 0, in _PyObject_Call_Prepend
(EngineCore_DP0 pid=141602) File "", line 0, in _PyObject_FastCallDictTstate.localalias
(EngineCore_DP0 pid=141602) File "", line 0, in _PyFunction_Vectorcall.cold
(EngineCore_DP0 pid=141602) File "??", line 0, in _PyEval_EvalFrameDefault
(EngineCore_DP0 pid=141602) File "/home/search/houchangjian/cephfs-houchangjian/envs/conda/envs/mineru/lib/python3.11/site-packages/torch/_dynamo/convert_frame.py", line 1495, in call
(EngineCore_DP0 pid=141602) return self._torchdynamo_orig_callable(
(EngineCore_DP0 pid=141602) File "", line 0, in _PyObject_MakeTpCall.localalias
(EngineCore_DP0 pid=141602) File "", line 0, in slot_tp_call
(EngineCore_DP0 pid=141602) File "??", line 0, in _PyObject_Call_Prepend
(EngineCore_DP0 pid=141602) File "", line 0, in _PyObject_FastCallDictTstate.localalias
(EngineCore_DP0 pid=141602) File "", line 0, in _PyFunction_Vectorcall.cold
(EngineCore_DP0 pid=141602) File "", line 0, in _PyEval_EvalFrameDefault.cold
(EngineCore_DP0 pid=141602) File "/home/search/houchangjian/cephfs-houchangjian/envs/conda/envs/mineru/lib/python3.11/site-packages/torch/_dynamo/convert_frame.py", line 629, in call
(EngineCore_DP0 pid=141602) return _compile(
(EngineCore_DP0 pid=141602) File "??", line 0, in PyObject_Vectorcall
(EngineCore_DP0 pid=141602) File "", line 0, in _PyFunction_Vectorcall.cold
(EngineCore_DP0 pid=141602) File "", line 0, in _PyEval_EvalFrameDefault.cold
(EngineCore_DP0 pid=141602) File "/home/search/houchangjian/cephfs-houchangjian/envs/conda/envs/mineru/lib/python3.11/site-packages/torch/_dynamo/convert_frame.py", line 1111, in _compile
(EngineCore_DP0 pid=141602) guarded_code = compile_inner(code, one_graph, hooks, transform)
(EngineCore_DP0 pid=141602) File "??", line 0, in PyObject_Vectorcall
(EngineCore_DP0 pid=141602) File "", line 0, in _PyFunction_Vectorcall.cold
(EngineCore_DP0 pid=141602) File "??", line 0, in _PyEval_EvalFrameDefault
(EngineCore_DP0 pid=141602) File "/home/search/houchangjian/cephfs-houchangjian/envs/conda/envs/mineru/lib/python3.11/site-packages/torch/_utils_internal.py", line 97, in wrapper_function
(EngineCore_DP0 pid=141602) return function(*args, **kwargs)
(EngineCore_DP0 pid=141602) File "", line 0, in _PyFunction_Vectorcall.cold
(EngineCore_DP0 pid=141602) File "", line 0, in _PyEval_EvalFrameDefault.cold
(EngineCore_DP0 pid=141602) File "/home/search/houchangjian/cephfs-houchangjian/envs/conda/envs/mineru/lib/python3.11/site-packages/torch/_dynamo/convert_frame.py", line 793, in compile_inner
(EngineCore_DP0 pid=141602) return _compile_inner(code, one_graph, hooks, transform)
(EngineCore_DP0 pid=141602) File "??", line 0, in PyObject_Vectorcall
(EngineCore_DP0 pid=141602) File "", line 0, in _PyFunction_Vectorcall.cold
(EngineCore_DP0 pid=141602) File "", line 0, in _PyEval_EvalFrameDefault.cold
(EngineCore_DP0 pid=141602) File "/home/search/houchangjian/cephfs-houchangjian/envs/conda/envs/mineru/lib/python3.11/site-packages/torch/_dynamo/convert_frame.py", line 832, in _compile_inner
(EngineCore_DP0 pid=141602) out_code = transform_code_object(code, transform)
(EngineCore_DP0 pid=141602) File "??", line 0, in PyObject_Vectorcall
(EngineCore_DP0 pid=141602) File "", line 0, in _PyFunction_Vectorcall.cold
(EngineCore_DP0 pid=141602) File "", line 0, in _PyEval_EvalFrameDefault.cold
(EngineCore_DP0 pid=141602) File "/home/search/houchangjian/cephfs-houchangjian/envs/conda/envs/mineru/lib/python3.11/site-packages/torch/_dynamo/bytecode_transformation.py", line 1424, in transform_code_object
(EngineCore_DP0 pid=141602) transformations(instructions, code_options)
(EngineCore_DP0 pid=141602) File "??", line 0, in PyObject_Vectorcall
(EngineCore_DP0 pid=141602) File "", line 0, in _PyFunction_Vectorcall.cold
(EngineCore_DP0 pid=141602) File "??", line 0, in _PyEval_EvalFrameDefault
(EngineCore_DP0 pid=141602) File "/home/search/houchangjian/cephfs-houchangjian/envs/conda/envs/mineru/lib/python3.11/site-packages/torch/_dynamo/convert_frame.py", line 267, in _fn
(EngineCore_DP0 pid=141602) return fn(*args, **kwargs)
(EngineCore_DP0 pid=141602) File "", line 0, in _PyFunction_Vectorcall.cold
(EngineCore_DP0 pid=141602) File "", line 0, in _PyEval_EvalFrameDefault.cold
(EngineCore_DP0 pid=141602) File "/home/search/houchangjian/cephfs-houchangjian/envs/conda/envs/mineru/lib/python3.11/site-packages/torch/_dynamo/convert_frame.py", line 753, in transform
(EngineCore_DP0 pid=141602) tracer.run()
(EngineCore_DP0 pid=141602) File "??", line 0, in PyObject_Vectorcall
(EngineCore_DP0 pid=141602) File "", line 0, in _PyFunction_Vectorcall.cold
(EngineCore_DP0 pid=141602) File "", line 0, in _PyEval_EvalFrameDefault.cold
(EngineCore_DP0 pid=141602) File "/home/search/houchangjian/cephfs-houchangjian/envs/conda/envs/mineru/lib/python3.11/site-packages/torch/_dynamo/symbolic_convert.py", line 3497, in run
(EngineCore_DP0 pid=141602) super().run()
(EngineCore_DP0 pid=141602) File "??", line 0, in PyObject_Vectorcall
(EngineCore_DP0 pid=141602) File "", line 0, in _PyFunction_Vectorcall.cold
(EngineCore_DP0 pid=141602) File "", line 0, in _PyEval_EvalFrameDefault.cold
(EngineCore_DP0 pid=141602) File "/home/search/houchangjian/cephfs-houchangjian/envs/conda/envs/mineru/lib/python3.11/site-packages/torch/_dynamo/symbolic_convert.py", line 1363, in run
(EngineCore_DP0 pid=141602) while self.step():
(EngineCore_DP0 pid=141602) File "??", line 0, in PyObject_Vectorcall
(EngineCore_DP0 pid=141602) File "", line 0, in _PyFunction_Vectorcall.cold
(EngineCore_DP0 pid=141602) File "", line 0, in _PyEval_EvalFrameDefault.cold
(EngineCore_DP0 pid=141602) File "/home/search/houchangjian/cephfs-houchangjian/envs/conda/envs/mineru/lib/python3.11/site-packages/torch/_dynamo/symbolic_convert.py", line 1267, in step
(EngineCore_DP0 pid=141602) self.dispatch_table[inst.opcode](self, inst)
(EngineCore_DP0 pid=141602) File "??", line 0, in PyObject_Vectorcall
(EngineCore_DP0 pid=141602) File "", line 0, in _PyFunction_Vectorcall.cold
(EngineCore_DP0 pid=141602) File "", line 0, in _PyEval_EvalFrameDefault.cold
(EngineCore_DP0 pid=141602) File "/home/search/houchangjian/cephfs-houchangjian/envs/conda/envs/mineru/lib/python3.11/site-packages/torch/_dynamo/symbolic_convert.py", line 834, in wrapper
(EngineCore_DP0 pid=141602) return inner_fn(self, inst)
(EngineCore_DP0 pid=141602) File "??", line 0, in PyObject_Vectorcall
(EngineCore_DP0 pid=141602) File "", line 0, in _PyFunction_Vectorcall.cold
(EngineCore_DP0 pid=141602) File "", line 0, in _PyEval_EvalFrameDefault.cold
(EngineCore_DP0 pid=141602) File "/home/search/houchangjian/cephfs-houchangjian/envs/conda/envs/mineru/lib/python3.11/site-packages/torch/_dynamo/symbolic_convert.py", line 2910, in CALL
(EngineCore_DP0 pid=141602) self._call(inst)
(EngineCore_DP0 pid=141602) File "??", line 0, in PyObject_Vectorcall
(EngineCore_DP0 pid=141602) File "", line 0, in _PyFunction_Vectorcall.cold
(EngineCore_DP0 pid=141602) File "", line 0, in _PyEval_EvalFrameDefault.cold
(EngineCore_DP0 pid=141602) File "/home/search/houchangjian/cephfs-houchangjian/envs/conda/envs/mineru/lib/python3.11/site-packages/torch/_dynamo/symbolic_convert.py", line 2904, in _call
(EngineCore_DP0 pid=141602) self.call_function(fn, args, kwargs)
(EngineCore_DP0 pid=141602) File "??", line 0, in PyObject_Vectorcall
(EngineCore_DP0 pid=141602) File "", line 0, in _PyFunction_Vectorcall.cold
(EngineCore_DP0 pid=141602) File "", line 0, in _PyEval_EvalFrameDefault.cold
(EngineCore_DP0 pid=141602) File "/home/search/houchangjian/cephfs-houchangjian/envs/conda/envs/mineru/lib/python3.11/site-packages/torch/_dynamo/symbolic_convert.py", line 1193, in call_function
(EngineCore_DP0 pid=141602) self.push(fn.call_function(self, args, kwargs)) # type: ignore[arg-type]
(EngineCore_DP0 pid=141602) File "??", line 0, in PyObject_Vectorcall
(EngineCore_DP0 pid=141602) File "", line 0, in _PyFunction_Vectorcall.cold
(EngineCore_DP0 pid=141602) File "??", line 0, in _PyEval_EvalFrameDefault
(EngineCore_DP0 pid=141602) File "/home/search/houchangjian/cephfs-houchangjian/envs/conda/envs/mineru/lib/python3.11/site-packages/torch/_dynamo/variables/lazy.py", line 201, in realize_and_forward
(EngineCore_DP0 pid=141602) return getattr(self.realize(), name)(*args, **kwargs)
(EngineCore_DP0 pid=141602) File "", line 0, in method_vectorcall.cold
(EngineCore_DP0 pid=141602) File "", line 0, in _PyEval_EvalFrameDefault.cold
(EngineCore_DP0 pid=141602) File "/home/search/houchangjian/cephfs-houchangjian/envs/conda/envs/mineru/lib/python3.11/site-packages/torch/_dynamo/variables/nn_module.py", line 1000, in call_function
(EngineCore_DP0 pid=141602) return variables.UserFunctionVariable(fn, source=source).call_function(
(EngineCore_DP0 pid=141602) File "??", line 0, in PyObject_Vectorcall
(EngineCore_DP0 pid=141602) File "", line 0, in _PyFunction_Vectorcall.cold
(EngineCore_DP0 pid=141602) File "", line 0, in _PyEval_EvalFrameDefault.cold
(EngineCore_DP0 pid=141602) File "/home/search/houchangjian/cephfs-houchangjian/envs/conda/envs/mineru/lib/python3.11/site-packages/torch/_dynamo/variables/functions.py", line 529, in call_function
(EngineCore_DP0 pid=141602) return super().call_function(tx, args, kwargs)
(EngineCore_DP0 pid=141602) File "??", line 0, in PyObject_Vectorcall
(EngineCore_DP0 pid=141602) File "", line 0, in _PyFunction_Vectorcall.cold
(EngineCore_DP0 pid=141602) File "", line 0, in _PyEval_EvalFrameDefault.cold
(EngineCore_DP0 pid=141602) File "/home/search/houchangjian/cephfs-houchangjian/envs/conda/envs/mineru/lib/python3.11/site-packages/torch/_dynamo/variables/functions.py", line 293, in call_function
(EngineCore_DP0 pid=141602) return tx.inline_user_function_return(self, [*self.self_args(), *args], kwargs)
(EngineCore_DP0 pid=141602) File "??", line 0, in PyObject_Vectorcall
(EngineCore_DP0 pid=141602) File "", line 0, in _PyFunction_Vectorcall.cold
(EngineCore_DP0 pid=141602) File "", line 0, in _PyEval_EvalFrameDefault.cold
(EngineCore_DP0 pid=141602) File "/home/search/houchangjian/cephfs-houchangjian/envs/conda/envs/mineru/lib/python3.11/site-packages/torch/_dynamo/symbolic_convert.py", line 1210, in inline_user_function_return
(EngineCore_DP0 pid=141602) return InliningInstructionTranslator.inline_call(self, fn, args, kwargs)
(EngineCore_DP0 pid=141602) File "??", line 0, in PyObject_Vectorcall
(EngineCore_DP0 pid=141602) File "", line 0, in _PyFunction_Vectorcall.cold
(EngineCore_DP0 pid=141602) File "", line 0, in _PyEval_EvalFrameDefault.cold
(EngineCore_DP0 pid=141602) File "/home/search/houchangjian/cephfs-houchangjian/envs/conda/envs/mineru/lib/python3.11/site-packages/vllm/compilation/decorators.py", line 287, in patched_inline_call
(EngineCore_DP0 pid=141602) return inline_call(parent, func, args, kwargs)
(EngineCore_DP0 pid=141602) File "??", line 0, in PyObject_Vectorcall
(EngineCore_DP0 pid=141602) File "", line 0, in _PyFunction_Vectorcall.cold
(EngineCore_DP0 pid=141602) File "", line 0, in _PyEval_EvalFrameDefault.cold
(EngineCore_DP0 pid=141602) File "/home/search/houchangjian/cephfs-houchangjian/envs/conda/envs/mineru/lib/python3.11/site-packages/torch/dynamo/symbolic_convert.py", line 3698, in inline_call
(EngineCore_DP0 pid=141602) return tracer.inline_call()
(EngineCore_DP0 pid=141602) File "??", line 0, in PyObject_Vectorcall
(EngineCore_DP0 pid=141602) File "", line 0, in _PyFunction_Vectorcall.cold
(EngineCore_DP0 pid=141602) File "", line 0, in _PyEval_EvalFrameDefault.cold
(EngineCore_DP0 pid=141602) File "/home/search/houchangjian/cephfs-houchangjian/envs/conda/envs/mineru/lib/python3.11/site-packages/torch/dynamo/symbolic_convert.py", line 3901, in inline_call
(EngineCore_DP0 pid=141602) self.run()
(EngineCore_DP0 pid=141602) File "??", line 0, in PyObject_Vectorcall
(EngineCore_DP0 pid=141602) File "", line 0, in _PyFunction_Vectorcall.cold
(EngineCore_DP0 pid=141602) File "", line 0, in _PyEval_EvalFrameDefault.cold
(EngineCore_DP0 pid=141602) File "/home/search/houchangjian/cephfs-houchangjian/envs/conda/envs/mineru/lib/python3.11/site-packages/torch/_dynamo/symbolic_convert.py", line 1363, in run
(EngineCore_DP0 pid=141602) while self.step():
(EngineCore_DP0 pid=141602) File "??", line 0, in PyObject_Vectorcall
(EngineCore_DP0 pid=141602) File "", line 0, in _PyFunction_Vectorcall.cold
(EngineCore_DP0 pid=141602) File "", line 0, in _PyEval_EvalFrameDefault.cold
(EngineCore_DP0 pid=141602) File "/home/search/houchangjian/cephfs-houchangjian/envs/conda/envs/mineru/lib/python3.11/site-packages/torch/_dynamo/symbolic_convert.py", line 1267, in step
(EngineCore_DP0 pid=141602) self.dispatch_table[inst.opcode](self, inst)
(EngineCore_DP0 pid=141602) File "??", line 0, in PyObject_Vectorcall
(EngineCore_DP0 pid=141602) File "", line 0, in _PyFunction_Vectorcall.cold
(EngineCore_DP0 pid=141602) File "", line 0, in _PyEval_EvalFrameDefault.cold
(EngineCore_DP0 pid=141602) File "/home/search/houchangjian/cephfs-houchangjian/envs/conda/envs/mineru/lib/python3.11/site-packages/torch/_dynamo/symbolic_convert.py", line 834, in wrapper
(EngineCore_DP0 pid=141602) return inner_fn(self, inst)
(EngineCore_DP0 pid=141602) File "??", line 0, in PyObject_Vectorcall
(EngineCore_DP0 pid=141602) File "", line 0, in _PyFunction_Vectorcall.cold
(EngineCore_DP0 pid=141602) File "", line 0, in _PyEval_EvalFrameDefault.cold
(EngineCore_DP0 pid=141602) File "/home/search/houchangjian/cephfs-houchangjian/envs/conda/envs/mineru/lib/python3.11/site-packages/torch/_dynamo/symbolic_convert.py", line 2228, in CALL_FUNCTION_EX
(EngineCore_DP0 pid=141602) self.call_function(fn, argsvars.items, kwargsvars)
(EngineCore_DP0 pid=141602) File "??", line 0, in PyObject_Vectorcall
(EngineCore_DP0 pid=141602) File "", line 0, in _PyFunction_Vectorcall.cold
(EngineCore_DP0 pid=141602) File "", line 0, in _PyEval_EvalFrameDefault.cold
(EngineCore_DP0 pid=141602) File "/home/search/houchangjian/cephfs-houchangjian/envs/conda/envs/mineru/lib/python3.11/site-packages/torch/_dynamo/symbolic_convert.py", line 1193, in call_function
(EngineCore_DP0 pid=141602) self.push(fn.call_function(self, args, kwargs)) # type: ignore[arg-type]
(EngineCore_DP0 pid=141602) File "??", line 0, in PyObject_Vectorcall
(EngineCore_DP0 pid=141602) File "", line 0, in _PyFunction_Vectorcall.cold
(EngineCore_DP0 pid=141602) File "??", line 0, in _PyEval_EvalFrameDefault
(EngineCore_DP0 pid=141602) File "/home/search/houchangjian/cephfs-houchangjian/envs/conda/envs/mineru/lib/python3.11/site-packages/torch/_dynamo/variables/lazy.py", line 201, in realize_and_forward
(EngineCore_DP0 pid=141602) return getattr(self.realize(), name)(*args, **kwargs)
(EngineCore_DP0 pid=141602) File "", line 0, in method_vectorcall.cold
(EngineCore_DP0 pid=141602) File "", line 0, in _PyEval_EvalFrameDefault.cold
(EngineCore_DP0 pid=141602) File "/home/search/houchangjian/cephfs-houchangjian/envs/conda/envs/mineru/lib/python3.11/site-packages/torch/_dynamo/variables/nn_module.py", line 1000, in call_function
(EngineCore_DP0 pid=141602) return variables.UserFunctionVariable(fn, source=source).call_function(
(EngineCore_DP0 pid=141602) File "??", line 0, in PyObject_Vectorcall
(EngineCore_DP0 pid=141602) File "", line 0, in _PyFunction_Vectorcall.cold
(EngineCore_DP0 pid=141602) File "", line 0, in _PyEval_EvalFrameDefault.cold
(EngineCore_DP0 pid=141602) File "/home/search/houchangjian/cephfs-houchangjian/envs/conda/envs/mineru/lib/python3.11/site-packages/torch/_dynamo/variables/functions.py", line 529, in call_function
(EngineCore_DP0 pid=141602) return super().call_function(tx, args, kwargs)
(EngineCore_DP0 pid=141602) File "??", line 0, in PyObject_Vectorcall
(EngineCore_DP0 pid=141602) File "", line 0, in _PyFunction_Vectorcall.cold
(EngineCore_DP0 pid=141602) File "", line 0, in _PyEval_EvalFrameDefault.cold
(EngineCore_DP0 pid=141602) File "/home/search/houchangjian/cephfs-houchangjian/envs/conda/envs/mineru/lib/python3.11/site-packages/torch/_dynamo/variables/functions.py", line 293, in call_function
(EngineCore_DP0 pid=141602) return tx.inline_user_function_return(self, [*self.self_args(), *args], kwargs)
(EngineCore_DP0 pid=141602) File "??", line 0, in PyObject_Vectorcall
(EngineCore_DP0 pid=141602) File "", line 0, in _PyFunction_Vectorcall.cold
(EngineCore_DP0 pid=141602) File "", line 0, in _PyEval_EvalFrameDefault.cold
(EngineCore_DP0 pid=141602) File "/home/search/houchangjian/cephfs-houchangjian/envs/conda/envs/mineru/lib/python3.11/site-packages/torch/_dynamo/symbolic_convert.py", line 1210, in inline_user_function_return
(EngineCore_DP0 pid=141602) return InliningInstructionTranslator.inline_call(self, fn, args, kwargs)
(EngineCore_DP0 pid=141602) File "??", line 0, in PyObject_Vectorcall
(EngineCore_DP0 pid=141602) File "", line 0, in _PyFunction_Vectorcall.cold
(EngineCore_DP0 pid=141602) File "", line 0, in _PyEval_EvalFrameDefault.cold
(EngineCore_DP0 pid=141602) File "/home/search/houchangjian/cephfs-houchangjian/envs/conda/envs/mineru/lib/python3.11/site-packages/vllm/compilation/decorators.py", line 287, in patched_inline_call
(EngineCore_DP0 pid=141602) return inline_call(parent, func, args, kwargs)
(EngineCore_DP0 pid=141602) File "??", line 0, in PyObject_Vectorcall
(EngineCore_DP0 pid=141602) File "", line 0, in _PyFunction_Vectorcall.cold
(EngineCore_DP0 pid=141602) File "", line 0, in _PyEval_EvalFrameDefault.cold
(EngineCore_DP0 pid=141602) File "/home/search/houchangjian/cephfs-houchangjian/envs/conda/envs/mineru/lib/python3.11/site-packages/torch/dynamo/symbolic_convert.py", line 3698, in inline_call
(EngineCore_DP0 pid=141602) return tracer.inline_call()
(EngineCore_DP0 pid=141602) File "??", line 0, in PyObject_Vectorcall
(EngineCore_DP0 pid=141602) File "", line 0, in _PyFunction_Vectorcall.cold
(EngineCore_DP0 pid=141602) File "", line 0, in _PyEval_EvalFrameDefault.cold
(EngineCore_DP0 pid=141602) File "/home/search/houchangjian/cephfs-houchangjian/envs/conda/envs/mineru/lib/python3.11/site-packages/torch/dynamo/symbolic_convert.py", line 3901, in inline_call
(EngineCore_DP0 pid=141602) self.run()
(EngineCore_DP0 pid=141602) File "??", line 0, in PyObject_Vectorcall
(EngineCore_DP0 pid=141602) File "", line 0, in _PyFunction_Vectorcall.cold
(EngineCore_DP0 pid=141602) File "", line 0, in _PyEval_EvalFrameDefault.cold
(EngineCore_DP0 pid=141602) File "/home/search/houchangjian/cephfs-houchangjian/envs/conda/envs/mineru/lib/python3.11/site-packages/torch/_dynamo/symbolic_convert.py", line 1363, in run
(EngineCore_DP0 pid=141602) while self.step():
(EngineCore_DP0 pid=141602) File "??", line 0, in PyObject_Vectorcall
(EngineCore_DP0 pid=141602) File "", line 0, in _PyFunction_Vectorcall.cold
(EngineCore_DP0 pid=141602) File "", line 0, in _PyEval_EvalFrameDefault.cold
(EngineCore_DP0 pid=141602) File "/home/search/houchangjian/cephfs-houchangjian/envs/conda/envs/mineru/lib/python3.11/site-packages/torch/_dynamo/symbolic_convert.py", line 1267, in step
(EngineCore_DP0 pid=141602) self.dispatch_table[inst.opcode](self, inst)
(EngineCore_DP0 pid=141602) File "??", line 0, in PyObject_Vectorcall
(EngineCore_DP0 pid=141602) File "", line 0, in _PyFunction_Vectorcall.cold
(EngineCore_DP0 pid=141602) File "", line 0, in _PyEval_EvalFrameDefault.cold
(EngineCore_DP0 pid=141602) File "/home/search/houchangjian/cephfs-houchangjian/envs/conda/envs/mineru/lib/python3.11/site-packages/torch/_dynamo/symbolic_convert.py", line 834, in wrapper
(EngineCore_DP0 pid=141602) return inner_fn(self, inst)
(EngineCore_DP0 pid=141602) File "??", line 0, in PyObject_Vectorcall
(EngineCore_DP0 pid=141602) File "", line 0, in _PyFunction_Vectorcall.cold
(EngineCore_DP0 pid=141602) File "", line 0, in _PyEval_EvalFrameDefault.cold
(EngineCore_DP0 pid=141602) File "/home/search/houchangjian/cephfs-houchangjian/envs/conda/envs/mineru/lib/python3.11/site-packages/torch/_dynamo/symbolic_convert.py", line 2228, in CALL_FUNCTION_EX
(EngineCore_DP0 pid=141602) self.call_function(fn, argsvars.items, kwargsvars)
(EngineCore_DP0 pid=141602) File "??", line 0, in PyObject_Vectorcall
(EngineCore_DP0 pid=141602) File "", line 0, in _PyFunction_Vectorcall.cold
(EngineCore_DP0 pid=141602) File "", line 0, in _PyEval_EvalFrameDefault.cold
(EngineCore_DP0 pid=141602) File "/home/search/houchangjian/cephfs-houchangjian/envs/conda/envs/mineru/lib/python3.11/site-packages/torch/_dynamo/symbolic_convert.py", line 1193, in call_function
(EngineCore_DP0 pid=141602) self.push(fn.call_function(self, args, kwargs)) # type: ignore[arg-type]
(EngineCore_DP0 pid=141602) File "??", line 0, in PyObject_Vectorcall
(EngineCore_DP0 pid=141602) File "", line 0, in _PyFunction_Vectorcall.cold
(EngineCore_DP0 pid=141602) File "??", line 0, in _PyEval_EvalFrameDefault
(EngineCore_DP0 pid=141602) File "/home/search/houchangjian/cephfs-houchangjian/envs/conda/envs/mineru/lib/python3.11/site-packages/torch/_dynamo/variables/lazy.py", line 201, in realize_and_forward
(EngineCore_DP0 pid=141602) return getattr(self.realize(), name)(*args, **kwargs)
(EngineCore_DP0 pid=141602) File "", line 0, in method_vectorcall.cold
(EngineCore_DP0 pid=141602) File "", line 0, in _PyEval_EvalFrameDefault.cold
(EngineCore_DP0 pid=141602) File "/home/search/houchangjian/cephfs-houchangjian/envs/conda/envs/mineru/lib/python3.11/site-packages/torch/_dynamo/variables/functions.py", line 529, in call_function
(EngineCore_DP0 pid=141602) return super().call_function(tx, args, kwargs)
(EngineCore_DP0 pid=141602) File "??", line 0, in PyObject_Vectorcall
(EngineCore_DP0 pid=141602) File "", line 0, in _PyFunction_Vectorcall.cold
(EngineCore_DP0 pid=141602) File "", line 0, in _PyEval_EvalFrameDefault.cold
(EngineCore_DP0 pid=141602) File "/home/search/houchangjian/cephfs-houchangjian/envs/conda/envs/mineru/lib/python3.11/site-packages/torch/_dynamo/variables/functions.py", line 293, in call_function
(EngineCore_DP0 pid=141602) return tx.inline_user_function_return(self, [self.self_args(), args], kwargs)
(EngineCore_DP0 pid=141602) File "??", line 0, in PyObject_Vectorcall
(EngineCore_DP0 pid=141602) File "", line 0, in _PyFunction_Vectorcall.cold
(EngineCore_DP0 pid=141602) File "", line 0, in _PyEval_EvalFrameDefault.cold
(EngineCore_DP0 pid=141602) File "/home/search/houchangjian/cephfs-houchangjian/envs/conda/envs/mineru/lib/python3.11/site-packages/torch/_dynamo/symbolic_convert.py", line 1210, in inline_user_function_return
(EngineCore_DP0 pid=141602) return InliningInstructionTranslator.inline_call(self, fn, args, kwargs)
(EngineCore_DP0 pid=141602) File "??", line 0, in PyObject_Vectorcall
(EngineCore_DP0 pid=141602) File "", line 0, in _PyFunction_Vectorcall.cold
(EngineCore_DP0 pid=141602) File "", line 0, in _PyEval_EvalFrameDefault.cold
(EngineCore_DP0 pid=141602) File "/home/search/houchangjian/cephfs-houchangjian/envs/conda/envs/mineru/lib/python3.11/site-packages/vllm/compilation/decorators.py", line 287, in patched_inline_call
(EngineCore_DP0 pid=141602) return inline_call(parent, func, args, kwargs)
(EngineCore_DP0 pid=141602) File "??", line 0, in PyObject_Vectorcall
(EngineCore_DP0 pid=141602) File "", line 0, in _PyFunction_Vectorcall.cold
(EngineCore_DP0 pid=141602) File "", line 0, in _PyEval_EvalFrameDefault.cold
(EngineCore_DP0 pid=141602) File "/home/search/houchangjian/cephfs-houchangjian/envs/conda/envs/mineru/lib/python3.11/site-packages/torch/dynamo/symbolic_convert.py", line 3698, in inline_call
(EngineCore_DP0 pid=141602) return tracer.inline_call()
(EngineCore_DP0 pid=141602) File "??", line 0, in PyObject_Vectorcall
(EngineCore_DP0 pid=141602) File "", line 0, in _PyFunction_Vectorcall.cold
(EngineCore_DP0 pid=141602) File "", line 0, in _PyEval_EvalFrameDefault.cold
(EngineCore_DP0 pid=141602) File "/home/search/houchangjian/cephfs-houchangjian/envs/conda/envs/mineru/lib/python3.11/site-packages/torch/dynamo/symbolic_convert.py", line 3901, in inline_call
(EngineCore_DP0 pid=141602) self.run()
(EngineCore_DP0 pid=141602) File "??", line 0, in PyObject_Vectorcall
(EngineCore_DP0 pid=141602) File "", line 0, in _PyFunction_Vectorcall.cold
(EngineCore_DP0 pid=141602) File "", line 0, in _PyEval_EvalFrameDefault.cold
(EngineCore_DP0 pid=141602) File "/home/search/houchangjian/cephfs-houchangjian/envs/conda/envs/mineru/lib/python3.11/site-packages/torch/_dynamo/symbolic_convert.py", line 1363, in run
(EngineCore_DP0 pid=141602) while self.step():
(EngineCore_DP0 pid=141602) File "??", line 0, in PyObject_Vectorcall
(EngineCore_DP0 pid=141602) File "", line 0, in _PyFunction_Vectorcall.cold
(EngineCore_DP0 pid=141602) File "", line 0, in _PyEval_EvalFrameDefault.cold
(EngineCore_DP0 pid=141602) File "/home/search/houchangjian/cephfs-houchangjian/envs/conda/envs/mineru/lib/python3.11/site-packages/torch/_dynamo/symbolic_convert.py", line 1267, in step
(EngineCore_DP0 pid=141602) self.dispatch_table[inst.opcode](self, inst)
(EngineCore_DP0 pid=141602) File "??", line 0, in PyObject_Vectorcall
(EngineCore_DP0 pid=141602) File "", line 0, in _PyFunction_Vectorcall.cold
(EngineCore_DP0 pid=141602) File "", line 0, in _PyEval_EvalFrameDefault.cold
(EngineCore_DP0 pid=141602) File "/home/search/houchangjian/cephfs-houchangjian/envs/conda/envs/mineru/lib/python3.11/site-packages/torch/_dynamo/symbolic_convert.py", line 834, in wrapper
(EngineCore_DP0 pid=141602) return inner_fn(self, inst)
(EngineCore_DP0 pid=141602) File "??", line 0, in PyObject_Vectorcall
(EngineCore_DP0 pid=141602) File "", line 0, in _PyFunction_Vectorcall.cold
(EngineCore_DP0 pid=141602) File "", line 0, in _PyEval_EvalFrameDefault.cold
(EngineCore_DP0 pid=141602) File "/home/search/houchangjian/cephfs-houchangjian/envs/conda/envs/mineru/lib/python3.11/site-packages/torch/_dynamo/symbolic_convert.py", line 2910, in CALL
(EngineCore_DP0 pid=141602) self._call(inst)
(EngineCore_DP0 pid=141602) File "??", line 0, in PyObject_Vectorcall
(EngineCore_DP0 pid=141602) File "", line 0, in _PyFunction_Vectorcall.cold
(EngineCore_DP0 pid=141602) File "", line 0, in _PyEval_EvalFrameDefault.cold
(EngineCore_DP0 pid=141602) File "/home/search/houchangjian/cephfs-houchangjian/envs/conda/envs/mineru/lib/python3.11/site-packages/torch/_dynamo/symbolic_convert.py", line 2904, in _call
(EngineCore_DP0 pid=141602) self.call_function(fn, args, kwargs)
(EngineCore_DP0 pid=141602) File "??", line 0, in PyObject_Vectorcall
(EngineCore_DP0 pid=141602) File "", line 0, in _PyFunction_Vectorcall.cold
(EngineCore_DP0 pid=141602) File "", line 0, in _PyEval_EvalFrameDefault.cold
(EngineCore_DP0 pid=141602) File "/home/search/houchangjian/cephfs-houchangjian/envs/conda/envs/mineru/lib/python3.11/site-packages/torch/_dynamo/symbolic_convert.py", line 1193, in call_function
(EngineCore_DP0 pid=141602) self.push(fn.call_function(self, args, kwargs)) # type: ignore[arg-type]
(EngineCore_DP0 pid=141602) File "??", line 0, in PyObject_Vectorcall
(EngineCore_DP0 pid=141602) File "", line 0, in _PyFunction_Vectorcall.cold
(EngineCore_DP0 pid=141602) File "??", line 0, in _PyEval_EvalFrameDefault
(EngineCore_DP0 pid=141602) File "/home/search/houchangjian/cephfs-houchangjian/envs/conda/envs/mineru/lib/python3.11/site-packages/torch/_dynamo/variables/lazy.py", line 201, in realize_and_forward
(EngineCore_DP0 pid=141602) return getattr(self.realize(), name)(args, **kwargs)
(EngineCore_DP0 pid=141602) File "", line 0, in method_vectorcall.cold
(EngineCore_DP0 pid=141602) File "", line 0, in _PyEval_EvalFrameDefault.cold
(EngineCore_DP0 pid=141602) File "/home/search/houchangjian/cephfs-houchangjian/envs/conda/envs/mineru/lib/python3.11/site-packages/torch/_dynamo/variables/nn_module.py", line 1000, in call_function
(EngineCore_DP0 pid=141602) return variables.UserFunctionVariable(fn, source=source).call_function(
(EngineCore_DP0 pid=141602) File "??", line 0, in PyObject_Vectorcall
(EngineCore_DP0 pid=141602) File "", line 0, in _PyFunction_Vectorcall.cold
(EngineCore_DP0 pid=141602) File "", line 0, in _PyEval_EvalFrameDefault.cold
(EngineCore_DP0 pid=141602) File "/home/search/houchangjian/cephfs-houchangjian/envs/conda/envs/mineru/lib/python3.11/site-packages/torch/_dynamo/variables/functions.py", line 529, in call_function
(EngineCore_DP0 pid=141602) return super().call_function(tx, args, kwargs)
(EngineCore_DP0 pid=141602) File "??", line 0, in PyObject_Vectorcall
(EngineCore_DP0 pid=141602) File "", line 0, in _PyFunction_Vectorcall.cold
(EngineCore_DP0 pid=141602) File "", line 0, in _PyEval_EvalFrameDefault.cold
(EngineCore_DP0 pid=141602) File "/home/search/houchangjian/cephfs-houchangjian/envs/conda/envs/mineru/lib/python3.11/site-packages/torch/_dynamo/variables/functions.py", line 293, in call_function
(EngineCore_DP0 pid=141602) return tx.inline_user_function_return(self, [self.self_args(), args], kwargs)
(EngineCore_DP0 pid=141602) File "??", line 0, in PyObject_Vectorcall
(EngineCore_DP0 pid=141602) File "", line 0, in _PyFunction_Vectorcall.cold
(EngineCore_DP0 pid=141602) File "", line 0, in _PyEval_EvalFrameDefault.cold
(EngineCore_DP0 pid=141602) File "/home/search/houchangjian/cephfs-houchangjian/envs/conda/envs/mineru/lib/python3.11/site-packages/torch/_dynamo/symbolic_convert.py", line 1210, in inline_user_function_return
(EngineCore_DP0 pid=141602) return InliningInstructionTranslator.inline_call(self, fn, args, kwargs)
(EngineCore_DP0 pid=141602) File "??", line 0, in PyObject_Vectorcall
(EngineCore_DP0 pid=141602) File "", line 0, in _PyFunction_Vectorcall.cold
(EngineCore_DP0 pid=141602) File "", line 0, in _PyEval_EvalFrameDefault.cold
(EngineCore_DP0 pid=141602) File "/home/search/houchangjian/cephfs-houchangjian/envs/conda/envs/mineru/lib/python3.11/site-packages/vllm/compilation/decorators.py", line 287, in patched_inline_call
(EngineCore_DP0 pid=141602) return inline_call(parent, func, args, kwargs)
(EngineCore_DP0 pid=141602) File "??", line 0, in PyObject_Vectorcall
(EngineCore_DP0 pid=141602) File "", line 0, in _PyFunction_Vectorcall.cold
(EngineCore_DP0 pid=141602) File "", line 0, in _PyEval_EvalFrameDefault.cold
(EngineCore_DP0 pid=141602) File "/home/search/houchangjian/cephfs-houchangjian/envs/conda/envs/mineru/lib/python3.11/site-packages/torch/dynamo/symbolic_convert.py", line 3698, in inline_call
(EngineCore_DP0 pid=141602) return tracer.inline_call()
(EngineCore_DP0 pid=141602) File "??", line 0, in PyObject_Vectorcall
(EngineCore_DP0 pid=141602) File "", line 0, in _PyFunction_Vectorcall.cold
(EngineCore_DP0 pid=141602) File "", line 0, in _PyEval_EvalFrameDefault.cold
(EngineCore_DP0 pid=141602) File "/home/search/houchangjian/cephfs-houchangjian/envs/conda/envs/mineru/lib/python3.11/site-packages/torch/dynamo/symbolic_convert.py", line 3901, in inline_call
(EngineCore_DP0 pid=141602) self.run()
(EngineCore_DP0 pid=141602) File "??", line 0, in PyObject_Vectorcall
(EngineCore_DP0 pid=141602) File "", line 0, in _PyFunction_Vectorcall.cold
(EngineCore_DP0 pid=141602) File "", line 0, in _PyEval_EvalFrameDefault.cold
(EngineCore_DP0 pid=141602) File "/home/search/houchangjian/cephfs-houchangjian/envs/conda/envs/mineru/lib/python3.11/site-packages/torch/_dynamo/symbolic_convert.py", line 1363, in run
(EngineCore_DP0 pid=141602) while self.step():
(EngineCore_DP0 pid=141602) File "??", line 0, in PyObject_Vectorcall
(EngineCore_DP0 pid=141602) File "", line 0, in _PyFunction_Vectorcall.cold
(EngineCore_DP0 pid=141602) File "", line 0, in _PyEval_EvalFrameDefault.cold
(EngineCore_DP0 pid=141602) File "/home/search/houchangjian/cephfs-houchangjian/envs/conda/envs/mineru/lib/python3.11/site-packages/torch/_dynamo/symbolic_convert.py", line 1267, in step
(EngineCore_DP0 pid=141602) self.dispatch_table[inst.opcode](self, inst)
(EngineCore_DP0 pid=141602) File "??", line 0, in PyObject_Vectorcall
(EngineCore_DP0 pid=141602) File "", line 0, in _PyFunction_Vectorcall.cold
(EngineCore_DP0 pid=141602) File "", line 0, in _PyEval_EvalFrameDefault.cold
(EngineCore_DP0 pid=141602) File "/home/search/houchangjian/cephfs-houchangjian/envs/conda/envs/mineru/lib/python3.11/site-packages/torch/_dynamo/symbolic_convert.py", line 834, in wrapper
(EngineCore_DP0 pid=141602) return inner_fn(self, inst)
(EngineCore_DP0 pid=141602) File "??", line 0, in PyObject_Vectorcall
(EngineCore_DP0 pid=141602) File "", line 0, in _PyFunction_Vectorcall.cold
(EngineCore_DP0 pid=141602) File "", line 0, in _PyEval_EvalFrameDefault.cold
(EngineCore_DP0 pid=141602) File "/home/search/houchangjian/cephfs-houchangjian/envs/conda/envs/mineru/lib/python3.11/site-packages/torch/_dynamo/symbolic_convert.py", line 2228, in CALL_FUNCTION_EX
(EngineCore_DP0 pid=141602) self.call_function(fn, argsvars.items, kwargsvars)
(EngineCore_DP0 pid=141602) File "??", line 0, in PyObject_Vectorcall
(EngineCore_DP0 pid=141602) File "", line 0, in _PyFunction_Vectorcall.cold
(EngineCore_DP0 pid=141602) File "", line 0, in _PyEval_EvalFrameDefault.cold
(EngineCore_DP0 pid=141602) File "/home/search/houchangjian/cephfs-houchangjian/envs/conda/envs/mineru/lib/python3.11/site-packages/torch/_dynamo/symbolic_convert.py", line 1193, in call_function
(EngineCore_DP0 pid=141602) self.push(fn.call_function(self, args, kwargs)) # type: ignore[arg-type]
(EngineCore_DP0 pid=141602) File "??", line 0, in PyObject_Vectorcall
(EngineCore_DP0 pid=141602) File "", line 0, in _PyFunction_Vectorcall.cold
(EngineCore_DP0 pid=141602) File "??", line 0, in _PyEval_EvalFrameDefault
(EngineCore_DP0 pid=141602) File "/home/search/houchangjian/cephfs-houchangjian/envs/conda/envs/mineru/lib/python3.11/site-packages/torch/_dynamo/variables/lazy.py", line 201, in realize_and_forward
(EngineCore_DP0 pid=141602) return getattr(self.realize(), name)(args, kwargs)
(EngineCore_DP0 pid=141602) File "", line 0, in method_vectorcall.cold
(EngineCore_DP0 pid=141602) File "", line 0, in _PyEval_EvalFrameDefault.cold
(EngineCore_DP0 pid=141602) File "/home/search/houchangjian/cephfs-houchangjian/envs/conda/envs/mineru/lib/python3.11/site-packages/torch/_dynamo/variables/functions.py", line 529, in call_function
(EngineCore_DP0 pid=141602) return super().call_function(tx, args, kwargs)
(EngineCore_DP0 pid=141602) File "??", line 0, in PyObject_Vectorcall
(EngineCore_DP0 pid=141602) File "", line 0, in _PyFunction_Vectorcall.cold
(EngineCore_DP0 pid=141602) File "", line 0, in _PyEval_EvalFrameDefault.cold
(EngineCore_DP0 pid=141602) File "/home/search/houchangjian/cephfs-houchangjian/envs/conda/envs/mineru/lib/python3.11/site-packages/torch/_dynamo/variables/functions.py", line 293, in call_function
(EngineCore_DP0 pid=141602) return tx.inline_user_function_return(self, [self.self_args(), args], kwargs)
(EngineCore_DP0 pid=141602) File "??", line 0, in PyObject_Vectorcall
(EngineCore_DP0 pid=141602) File "", line 0, in _PyFunction_Vectorcall.cold
(EngineCore_DP0 pid=141602) File "", line 0, in _PyEval_EvalFrameDefault.cold
(EngineCore_DP0 pid=141602) File "/home/search/houchangjian/cephfs-houchangjian/envs/conda/envs/mineru/lib/python3.11/site-packages/torch/_dynamo/symbolic_convert.py", line 1210, in inline_user_function_return
(EngineCore_DP0 pid=141602) return InliningInstructionTranslator.inline_call(self, fn, args, kwargs)
(EngineCore_DP0 pid=141602) File "??", line 0, in PyObject_Vectorcall
(EngineCore_DP0 pid=141602) File "", line 0, in _PyFunction_Vectorcall.cold
(EngineCore_DP0 pid=141602) File "", line 0, in _PyEval_EvalFrameDefault.cold
(EngineCore_DP0 pid=141602) File "/home/search/houchangjian/cephfs-houchangjian/envs/conda/envs/mineru/lib/python3.11/site-packages/vllm/compilation/decorators.py", line 287, in patched_inline_call
(EngineCore_DP0 pid=141602) return inline_call(parent, func, args, kwargs)
(EngineCore_DP0 pid=141602) File "??", line 0, in PyObject_Vectorcall
(EngineCore_DP0 pid=141602) File "", line 0, in _PyFunction_Vectorcall.cold
(EngineCore_DP0 pid=141602) File "", line 0, in _PyEval_EvalFrameDefault.cold
(EngineCore_DP0 pid=141602) File "/home/search/houchangjian/cephfs-houchangjian/envs/conda/envs/mineru/lib/python3.11/site-packages/torch/dynamo/symbolic_convert.py", line 3698, in inline_call
(EngineCore_DP0 pid=141602) return tracer.inline_call()
(EngineCore_DP0 pid=141602) File "??", line 0, in PyObject_Vectorcall
(EngineCore_DP0 pid=141602) File "", line 0, in _PyFunction_Vectorcall.cold
(EngineCore_DP0 pid=141602) File "", line 0, in _PyEval_EvalFrameDefault.cold
(EngineCore_DP0 pid=141602) File "/home/search/houchangjian/cephfs-houchangjian/envs/conda/envs/mineru/lib/python3.11/site-packages/torch/dynamo/symbolic_convert.py", line 3901, in inline_call
(EngineCore_DP0 pid=141602) self.run()
(EngineCore_DP0 pid=141602) File "??", line 0, in PyObject_Vectorcall
(EngineCore_DP0 pid=141602) File "", line 0, in _PyFunction_Vectorcall.cold
(EngineCore_DP0 pid=141602) File "", line 0, in _PyEval_EvalFrameDefault.cold
(EngineCore_DP0 pid=141602) File "/home/search/houchangjian/cephfs-houchangjian/envs/conda/envs/mineru/lib/python3.11/site-packages/torch/_dynamo/symbolic_convert.py", line 1363, in run
(EngineCore_DP0 pid=141602) while self.step():
(EngineCore_DP0 pid=141602) File "??", line 0, in PyObject_Vectorcall
(EngineCore_DP0 pid=141602) File "", line 0, in _PyFunction_Vectorcall.cold
(EngineCore_DP0 pid=141602) File "", line 0, in _PyEval_EvalFrameDefault.cold
(EngineCore_DP0 pid=141602) File "/home/search/houchangjian/cephfs-houchangjian/envs/conda/envs/mineru/lib/python3.11/site-packages/torch/_dynamo/symbolic_convert.py", line 1267, in step
(EngineCore_DP0 pid=141602) self.dispatch_table[inst.opcode](self, inst)
(EngineCore_DP0 pid=141602) File "??", line 0, in PyObject_Vectorcall
(EngineCore_DP0 pid=141602) File "", line 0, in _PyFunction_Vectorcall.cold
(EngineCore_DP0 pid=141602) File "", line 0, in _PyEval_EvalFrameDefault.cold
(EngineCore_DP0 pid=141602) File "/home/search/houchangjian/cephfs-houchangjian/envs/conda/envs/mineru/lib/python3.11/site-packages/torch/_dynamo/symbolic_convert.py", line 2851, in BINARY_OP
(EngineCore_DP0 pid=141602) return _binary_op_lookup[inst.arg](self, inst)
(EngineCore_DP0 pid=141602) File "??", line 0, in PyObject_Vectorcall
(EngineCore_DP0 pid=141602) File "", line 0, in _PyFunction_Vectorcall.cold
(EngineCore_DP0 pid=141602) File "", line 0, in _PyEval_EvalFrameDefault.cold
(EngineCore_DP0 pid=141602) File "/home/search/houchangjian/cephfs-houchangjian/envs/conda/envs/mineru/lib/python3.11/site-packages/torch/_dynamo/symbolic_convert.py", line 425, in impl
(EngineCore_DP0 pid=141602) self.push(fn_var.call_function(self, self.popn(nargs), {}))
(EngineCore_DP0 pid=141602) File "??", line 0, in PyObject_Vectorcall
(EngineCore_DP0 pid=141602) File "", line 0, in _PyFunction_Vectorcall.cold
(EngineCore_DP0 pid=141602) File "", line 0, in _PyEval_EvalFrameDefault.cold
(EngineCore_DP0 pid=141602) File "/home/search/houchangjian/cephfs-houchangjian/envs/conda/envs/mineru/lib/python3.11/site-packages/torch/_dynamo/variables/builtin.py", line 1189, in call_function
(EngineCore_DP0 pid=141602) return handler(tx, args, kwargs)
(EngineCore_DP0 pid=141602) File "??", line 0, in PyObject_Vectorcall
(EngineCore_DP0 pid=141602) File "", line 0, in _PyFunction_Vectorcall.cold
(EngineCore_DP0 pid=141602) File "", line 0, in _PyEval_EvalFrameDefault.cold
(EngineCore_DP0 pid=141602) File "/home/search/houchangjian/cephfs-houchangjian/envs/conda/envs/mineru/lib/python3.11/site-packages/torch/_dynamo/variables/builtin.py", line 1149, in _handle_insert_op_in_graph
(EngineCore_DP0 pid=141602) return wrap_fx_proxy(tx, proxy)
(EngineCore_DP0 pid=141602) File "??", line 0, in PyObject_Vectorcall
(EngineCore_DP0 pid=141602) File "", line 0, in _PyFunction_Vectorcall.cold
(EngineCore_DP0 pid=141602) File "??", line 0, in _PyEval_EvalFrameDefault
(EngineCore_DP0 pid=141602) File "/home/search/houchangjian/cephfs-houchangjian/envs/conda/envs/mineru/lib/python3.11/site-packages/torch/_dynamo/variables/builder.py", line 2559, in wrap_fx_proxy
(EngineCore_DP0 pid=141602) return wrap_fx_proxy_cls(target_cls=TensorVariable, kwargs)
(EngineCore_DP0 pid=141602) File "??", line 0, in PyObject_Call
(EngineCore_DP0 pid=141602) File "", line 0, in _PyFunction_Vectorcall.cold
(EngineCore_DP0 pid=141602) File "??", line 0, in _PyEval_EvalFrameDefault
(EngineCore_DP0 pid=141602) File "/home/search/houchangjian/cephfs-houchangjian/envs/conda/envs/mineru/lib/python3.11/site-packages/torch/_dynamo/variables/builder.py", line 2625, in wrap_fx_proxy_cls
(EngineCore_DP0 pid=141602) return _wrap_fx_proxy(
(EngineCore_DP0 pid=141602) File "", line 0, in _PyFunction_Vectorcall.cold
(EngineCore_DP0 pid=141602) File "", line 0, in _PyEval_EvalFrameDefault.cold
(EngineCore_DP0 pid=141602) File "/home/search/houchangjian/cephfs-houchangjian/envs/conda/envs/mineru/lib/python3.11/site-packages/torch/_dynamo/variables/builder.py", line 2723, in _wrap_fx_proxy
(EngineCore_DP0 pid=141602) example_value = get_fake_value(proxy.node, tx, allow_non_graph_fake=True)
(EngineCore_DP0 pid=141602) File "??", line 0, in PyObject_Vectorcall
(EngineCore_DP0 pid=141602) File "", line 0, in _PyFunction_Vectorcall.cold
(EngineCore_DP0 pid=141602) File "", line 0, in _PyEval_EvalFrameDefault.cold
(EngineCore_DP0 pid=141602) File "/home/search/houchangjian/cephfs-houchangjian/envs/conda/envs/mineru/lib/python3.11/site-packages/torch/_dynamo/utils.py", line 3253, in get_fake_value
(EngineCore_DP0 pid=141602) ret_val = wrap_fake_exception(
(EngineCore_DP0 pid=141602) File "??", line 0, in PyObject_Vectorcall
(EngineCore_DP0 pid=141602) File "", line 0, in _PyFunction_Vectorcall.cold
(EngineCore_DP0 pid=141602) File "", line 0, in _PyEval_EvalFrameDefault.cold
(EngineCore_DP0 pid=141602) File "/home/search/houchangjian/cephfs-houchangjian/envs/conda/envs/mineru/lib/python3.11/site-packages/torch/_dynamo/utils.py", line 2753, in wrap_fake_exception
(EngineCore_DP0 pid=141602) return fn()
(EngineCore_DP0 pid=141602) File "??", line 0, in PyObject_Vectorcall
(EngineCore_DP0 pid=141602) File "", line 0, in _PyFunction_Vectorcall.cold
(EngineCore_DP0 pid=141602) File "", line 0, in _PyEval_EvalFrameDefault.cold
(EngineCore_DP0 pid=141602) File "/home/search/houchangjian/cephfs-houchangjian/envs/conda/envs/mineru/lib/python3.11/site-packages/torch/_dynamo/utils.py", line 3254, in
(EngineCore_DP0 pid=141602) lambda: run_node(tx.output, node, args, kwargs, nnmodule)
(EngineCore_DP0 pid=141602) File "??", line 0, in PyObject_Vectorcall
(EngineCore_DP0 pid=141602) File "", line 0, in _PyFunction_Vectorcall.cold
(EngineCore_DP0 pid=141602) File "??", line 0, in _PyEval_EvalFrameDefault
(EngineCore_DP0 pid=141602) File "/home/search/houchangjian/cephfs-houchangjian/envs/conda/envs/mineru/lib/python3.11/site-packages/torch/dynamo/utils.py", line 3421, in run_node
(EngineCore_DP0 pid=141602) return node.target(*args, *kwargs)
(EngineCore_DP0 pid=141602) File "", line 0, in cfunction_vectorcall_FASTCALL
(EngineCore_DP0 pid=141602) File "", line 0, in binary_op
(EngineCore_DP0 pid=141602) File "", line 0, in slot_nb_matrix_multiply
(EngineCore_DP0 pid=141602) File "", line 0, in vectorcall_maybe.constprop.0
(EngineCore_DP0 pid=141602) File "", line 0, in method_vectorcall_VARARGS_KEYWORDS
(EngineCore_DP0 pid=141602) File "python_variable_methods.cpp", line 0, in _object torch::autograd::TypeError_to_NotImplemented<&torch::autograd::THPVariable_matmul>(_object, _object, _object)
(EngineCore_DP0 pid=141602) File "python_variable_methods.cpp", line 0, in torch::autograd::THPVariable_matmul(_object, _object, _object)
(EngineCore_DP0 pid=141602) File "??", line 0, in at::_ops::matmul::call(at::Tensor const&, at::Tensor const&)
(EngineCore_DP0 pid=141602) File "", line 0, in c10::impl::BoxedKernelWrapper<at::Tensor (at::Tensor const&, at::Tensor const&), void>::call(c10::BoxedKernel const&, c10::OperatorHandle const&, c10::DispatchKeySet, at::Tensor const&, at::Tensor const&)
(EngineCore_DP0 pid=141602) File "PyInterpreter.cpp", line 0, in torch::detail::(anonymous namespace)::ConcretePyInterpreterVTable::python_dispatcher(c10::OperatorHandle const&, c10::DispatchKeySet, std::vector<c10::IValue, std::allocatorc10::IValue >) const
(EngineCore_DP0 pid=141602) File "", line 0, in c10::OperatorHandle::callBoxedForDispatchKey(c10::DispatchKey, std::vector<c10::IValue, std::allocatorc10::IValue >&) const
(EngineCore_DP0 pid=141602) File "PythonFallbackKernel.cpp", line 0, in void c10::BoxedKernel::make_boxed_function<&(anonymous namespace)::pythonTLSSnapshotFallback>(c10::OperatorKernel, c10::OperatorHandle const&, c10::DispatchKeySet, std::vector<c10::IValue, std::allocatorc10::IValue >)
(EngineCore_DP0 pid=141602) File "PyInterpreter.cpp", line 0, in torch::detail::(anonymous namespace)::ConcretePyInterpreterVTable::python_dispatcher(c10::OperatorHandle const&, c10::DispatchKeySet, std::vector<c10::IValue, std::allocatorc10::IValue >) const
(EngineCore_DP0 pid=141602) File "", line 0, in _PyFunction_Vectorcall.cold
(EngineCore_DP0 pid=141602) File "??", line 0, in _PyEval_EvalFrameDefault
(EngineCore_DP0 pid=141602) File "/home/search/houchangjian/cephfs-houchangjian/envs/conda/envs/mineru/lib/python3.11/site-packages/torch/_prims_common/wrappers.py", line 307, in _fn
(EngineCore_DP0 pid=141602) result = fn(args, is_out=(out is not None), **kwargs) # type: ignore[arg-type]
(EngineCore_DP0 pid=141602) File "??", line 0, in PyObject_Call
(EngineCore_DP0 pid=141602) File "", line 0, in _PyFunction_Vectorcall.cold
(EngineCore_DP0 pid=141602) File "", line 0, in _PyEval_EvalFrameDefault.cold
(EngineCore_DP0 pid=141602) File "/home/search/houchangjian/cephfs-houchangjian/envs/conda/envs/mineru/lib/python3.11/site-packages/torch/_decomp/decompositions.py", line 4496, in matmul
(EngineCore_DP0 pid=141602) torch.broadcast_shapes(batch_tensor1, batch_tensor2)
(EngineCore_DP0 pid=141602) File "??", line 0, in PyObject_Vectorcall
(EngineCore_DP0 pid=141602) File "", line 0, in _PyFunction_Vectorcall.cold
(EngineCore_DP0 pid=141602) File "", line 0, in _PyEval_EvalFrameDefault.cold
(EngineCore_DP0 pid=141602) File "/home/search/houchangjian/cephfs-houchangjian/envs/conda/envs/mineru/lib/python3.11/site-packages/torch/functional.py", line 145, in broadcast_shapes
(EngineCore_DP0 pid=141602) if guard_size_oblivious(shape[i] == 1) or guard_size_oblivious(
(EngineCore_DP0 pid=141602) File "??", line 0, in PyObject_Vectorcall
(EngineCore_DP0 pid=141602) File "", line 0, in _PyFunction_Vectorcall.cold
(EngineCore_DP0 pid=141602) File "", line 0, in _PyEval_EvalFrameDefault.cold
(EngineCore_DP0 pid=141602) File "/home/search/houchangjian/cephfs-houchangjian/envs/conda/envs/mineru/lib/python3.11/site-packages/torch/fx/experimental/symbolic_shapes.py", line 473, in guard_size_oblivious
(EngineCore_DP0 pid=141602) return expr.node.guard_size_oblivious("", 0)
(EngineCore_DP0 pid=141602) File "??", line 0, in PyObject_Vectorcall
(EngineCore_DP0 pid=141602) File "", line 0, in _PyFunction_Vectorcall.cold
(EngineCore_DP0 pid=141602) File "", line 0, in _PyEval_EvalFrameDefault.cold
(EngineCore_DP0 pid=141602) File "/home/search/houchangjian/cephfs-houchangjian/envs/conda/envs/mineru/lib/python3.11/site-packages/torch/fx/experimental/sym_node.py", line 596, in guard_size_oblivious
(EngineCore_DP0 pid=141602) r = self.evaluate(size_oblivious=True)
(EngineCore_DP0 pid=141602) File "??", line 0, in PyObject_Vectorcall
(EngineCore_DP0 pid=141602) File "", line 0, in _PyFunction_Vectorcall.cold
(EngineCore_DP0 pid=141602) File "", line 0, in _PyEval_EvalFrameDefault.cold
(EngineCore_DP0 pid=141602) File "/home/search/houchangjian/cephfs-houchangjian/envs/conda/envs/mineru/lib/python3.11/site-packages/torch/fx/experimental/sym_node.py", line 512, in evaluate
(EngineCore_DP0 pid=141602) return self.shape_env.evaluate_sym_node(self, size_oblivious)
(EngineCore_DP0 pid=141602) File "??", line 0, in PyObject_Vectorcall
(EngineCore_DP0 pid=141602) File "", line 0, in _PyFunction_Vectorcall.cold
(EngineCore_DP0 pid=141602) File "", line 0, in _PyEval_EvalFrameDefault.cold
(EngineCore_DP0 pid=141602) File "/home/search/houchangjian/cephfs-houchangjian/envs/conda/envs/mineru/lib/python3.11/site-packages/torch/fx/experimental/symbolic_shapes.py", line 7223, in evaluate_sym_node
(EngineCore_DP0 pid=141602) return self.evaluate_expr(
(EngineCore_DP0 pid=141602) File "??", line 0, in PyObject_Vectorcall
(EngineCore_DP0 pid=141602) File "", line 0, in _PyFunction_Vectorcall.cold
(EngineCore_DP0 pid=141602) File "??", line 0, in _PyEval_EvalFrameDefault
(EngineCore_DP0 pid=141602) File "/home/search/houchangjian/cephfs-houchangjian/envs/conda/envs/mineru/lib/python3.11/site-packages/torch/fx/experimental/symbolic_shapes.py", line 7323, in evaluate_expr
(EngineCore_DP0 pid=141602) return self._inner_evaluate_expr(
(EngineCore_DP0 pid=141602) File "", line 0, in _PyObject_MakeTpCall.localalias
(EngineCore_DP0 pid=141602) File "", line 0, in bounded_lru_cache_wrapper
(EngineCore_DP0 pid=141602) File "", line 0, in _PyFunction_Vectorcall.cold
(EngineCore_DP0 pid=141602) File "??", line 0, in _PyEval_EvalFrameDefault
(EngineCore_DP0 pid=141602) File "/home/search/houchangjian/cephfs-houchangjian/envs/conda/envs/mineru/lib/python3.11/site-packages/torch/fx/experimental/recording.py", line 272, in wrapper
(EngineCore_DP0 pid=141602) return retlog(fn(args, kwargs))
(EngineCore_DP0 pid=141602) File "", line 0, in _PyFunction_Vectorcall.cold
(EngineCore_DP0 pid=141602) File "", line 0, in _PyEval_EvalFrameDefault.cold
(EngineCore_DP0 pid=141602) File "/home/search/houchangjian/cephfs-houchangjian/envs/conda/envs/mineru/lib/python3.11/site-packages/torch/fx/experimental/symbolic_shapes.py", line 7346, in _inner_evaluate_expr
(EngineCore_DP0 pid=141602) return self._evaluate_expr(
(EngineCore_DP0 pid=141602) File "??", line 0, in PyObject_Vectorcall
(EngineCore_DP0 pid=141602) File "", line 0, in _PyFunction_Vectorcall.cold
(EngineCore_DP0 pid=141602) File "??", line 0, in _PyEval_EvalFrameDefault
(EngineCore_DP0 pid=141602) File "/home/search/houchangjian/cephfs-houchangjian/envs/conda/envs/mineru/lib/python3.11/site-packages/torch/fx/experimental/symbolic_shapes.py", line 7615, in _evaluate_expr
(EngineCore_DP0 pid=141602) self._maybe_guard_rel(g)
(EngineCore_DP0 pid=141602) File "", line 0, in _PyObject_MakeTpCall.localalias
(EngineCore_DP0 pid=141602) File "", line 0, in bounded_lru_cache_wrapper
(EngineCore_DP0 pid=141602) File "", line 0, in _PyFunction_Vectorcall.cold
(EngineCore_DP0 pid=141602) File "", line 0, in _PyEval_EvalFrameDefault.cold
(EngineCore_DP0 pid=141602) File "/home/search/houchangjian/cephfs-houchangjian/envs/conda/envs/mineru/lib/python3.11/site-packages/torch/fx/experimental/symbolic_shapes.py", line 6868, in _maybe_guard_rel
(EngineCore_DP0 pid=141602) self._refine_ranges(expr)
(EngineCore_DP0 pid=141602) File "??", line 0, in PyObject_Vectorcall
(EngineCore_DP0 pid=141602) File "", line 0, in _PyFunction_Vectorcall.cold
(EngineCore_DP0 pid=141602) File "", line 0, in _PyEval_EvalFrameDefault.cold
(EngineCore_DP0 pid=141602) File "/home/search/houchangjian/cephfs-houchangjian/envs/conda/envs/mineru/lib/python3.11/site-packages/torch/fx/experimental/symbolic_shapes.py", line 7819, in _refine_ranges
(EngineCore_DP0 pid=141602) self._set_replacement(
(EngineCore_DP0 pid=141602) File "??", line 0, in PyObject_Vectorcall
(EngineCore_DP0 pid=141602) File "", line 0, in _PyFunction_Vectorcall.cold
(EngineCore_DP0 pid=141602) File "", line 0, in _PyEval_EvalFrameDefault.cold
(EngineCore_DP0 pid=141602) File "/home/search/houchangjian/cephfs-houchangjian/envs/conda/envs/mineru/lib/python3.11/site-packages/torch/fx/experimental/symbolic_shapes.py", line 6768, in _set_replacement
(EngineCore_DP0 pid=141602) CapturedTraceback.extract(cpp=True)
(EngineCore_DP0 pid=141602) File "??", line 0, in PyObject_Vectorcall
(EngineCore_DP0 pid=141602) File "", line 0, in _PyFunction_Vectorcall.cold
(EngineCore_DP0 pid=141602) File "??", line 0, in _PyEval_EvalFrameDefault
(EngineCore_DP0 pid=141602) File "/home/search/houchangjian/cephfs-houchangjian/envs/conda/envs/mineru/lib/python3.11/site-packages/torch/utils/_traceback.py", line 212, in extract
(EngineCore_DP0 pid=141602) torch._C._profiler.gather_traceback(python=True, script=script, cpp=cpp),
(EngineCore_DP0 pid=141602) File "", line 0, in _PyObject_MakeTpCall.localalias
(EngineCore_DP0 pid=141602) File "", line 0, in cfunction_call
(EngineCore_DP0 pid=141602) File "", line 0, in pybind11::cpp_function::dispatcher(_object, _object, _object)
(EngineCore_DP0 pid=141602) File "", line 0, in pybind11::cpp_function::initialize<std::shared_ptrtorch::CapturedTraceback (&)(bool, bool, bool), std::shared_ptrtorch::CapturedTraceback, bool, bool, bool, pybind11::name, pybind11::scope, pybind11::sibling, pybind11::arg_v, pybind11::arg_v, pybind11::arg_v>(std::shared_ptrtorch::CapturedTraceback (&)(bool, bool, bool), std::shared_ptrtorch::CapturedTraceback ()(bool, bool, bool), pybind11::name const&, pybind11::scope const&, pybind11::sibling const&, pybind11::arg_v const&, pybind11::arg_v const&, pybind11::arg_v const&)::{lambda(pybind11::detail::function_call&)#3}::operator()(pybind11::detail::function_call&) const
(EngineCore_DP0 pid=141602) File "??", line 0, in torch::CapturedTraceback::gather(bool, bool, bool)
(EngineCore_DP0 pid=141602) File "??", line 0, in torch::unwind::unwind()
(EngineCore_DP0 pid=141602)
(EngineCore_DP0 pid=141602)
(EngineCore_DP0 pid=141602) User stack:
(EngineCore_DP0 pid=141602) File "/home/search/houchangjian/cephfs-houchangjian/envs/conda/envs/mineru/lib/python3.11/site-packages/vllm/model_executor/models/transformers.py", line 680, in forward
(EngineCore_DP0 pid=141602) hidden_states = self.model(
(EngineCore_DP0 pid=141602) File "/home/search/.cache/huggingface/modules/transformers_modules/paddleocr_hyphen_vl/modeling_paddleocr_vl.py", line 2452, in forward
(EngineCore_DP0 pid=141602) outputs = self.model(
(EngineCore_DP0 pid=141602) File "/home/search/houchangjian/cephfs-houchangjian/envs/conda/envs/mineru/lib/python3.11/site-packages/transformers/utils/generic.py", line 1064, in wrapper
(EngineCore_DP0 pid=141602) outputs = func(self, *args, **kwargs)
(EngineCore_DP0 pid=141602) File "/home/search/.cache/huggingface/modules/transformers_modules/paddleocr_hyphen_vl/modeling_paddleocr_vl.py", line 616, in forward
(EngineCore_DP0 pid=141602) position_embeddings = self.rotary_emb(hidden_states, position_ids)
(EngineCore_DP0 pid=141602) File "/home/search/houchangjian/cephfs-houchangjian/envs/conda/envs/mineru/lib/python3.11/site-packages/torch/utils/_contextlib.py", line 120, in decorate_context
(EngineCore_DP0 pid=141602) return func(*args, **kwargs)
(EngineCore_DP0 pid=141602) File "/home/search/.cache/huggingface/modules/transformers_modules/paddleocr_hyphen_vl/modeling_paddleocr_vl.py", line 145, in forward
(EngineCore_DP0 pid=141602) inv_freq_expanded.float() @ position_ids_expanded.float()
(EngineCore_DP0 pid=141602)
(EngineCore_DP0 pid=141602)
[rank0]:[W1125 14:04:36.904513633 ProcessGroupNCCL.cpp:1538] Warning: WARNING: destroy_process_group() was not called before program exit, which can leak resources. For more info, please see https://pytorch.org/docs/stable/distributed.html#shutdown (function operator())
Process Process-1:
Traceback (most recent call last):
File "/home/search/houchangjian/cephfs-houchangjian/envs/conda/envs/mineru/lib/python3.11/multiprocessing/process.py", line 314, in _bootstrap
self.run()
File "/home/search/houchangjian/cephfs-houchangjian/envs/conda/envs/mineru/lib/python3.11/multiprocessing/process.py", line 108, in run
self._target(*self._args, **self._kwargs)
File "/da2/search/houchangjian/cephfs-houchangjian/ms-swift-main/swift/llm/infer/rollout.py", line 195, in llm_worker
engine = SwiftRolloutDeploy.get_infer_engine(args, template=args.get_template(None))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/da2/search/houchangjian/cephfs-houchangjian/ms-swift-main/swift/llm/infer/rollout.py", line 350, in get_infer_engine
return GRPOVllmEngine(**kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^
File "/da2/search/houchangjian/cephfs-houchangjian/ms-swift-main/swift/llm/infer/infer_engine/grpo_vllm_engine.py", line 62, in init
super().init(
File "/da2/search/houchangjian/cephfs-houchangjian/ms-swift-main/swift/llm/infer/infer_engine/vllm_engine.py", line 142, in init
self._prepare_engine()
File "/da2/search/houchangjian/cephfs-houchangjian/ms-swift-main/swift/llm/infer/infer_engine/vllm_engine.py", line 152, in _prepare_engine
engine = llm_engine_cls.from_engine_args(self.engine_args)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/search/houchangjian/cephfs-houchangjian/envs/conda/envs/mineru/lib/python3.11/site-packages/vllm/v1/engine/llm_engine.py", line 177, in from_engine_args
return cls(vllm_config=vllm_config,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/search/houchangjian/cephfs-houchangjian/envs/conda/envs/mineru/lib/python3.11/site-packages/vllm/v1/engine/llm_engine.py", line 114, in init
self.engine_core = EngineCoreClient.make_client(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/search/houchangjian/cephfs-houchangjian/envs/conda/envs/mineru/lib/python3.11/site-packages/vllm/v1/engine/core_client.py", line 80, in make_client
return SyncMPClient(vllm_config, executor_class, log_stats)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/search/houchangjian/cephfs-houchangjian/envs/conda/envs/mineru/lib/python3.11/site-packages/vllm/v1/engine/core_client.py", line 602, in init
super().init(
File "/home/search/houchangjian/cephfs-houchangjian/envs/conda/envs/mineru/lib/python3.11/site-packages/vllm/v1/engine/core_client.py", line 448, in init
with launch_core_engines(vllm_config, executor_class,
File "/home/search/houchangjian/cephfs-houchangjian/envs/conda/envs/mineru/lib/python3.11/contextlib.py", line 144, in exit
next(self.gen)
File "/home/search/houchangjian/cephfs-houchangjian/envs/conda/envs/mineru/lib/python3.11/site-packages/vllm/v1/engine/utils.py", line 732, in launch_core_engines
wait_for_engine_startup(
File "/home/search/houchangjian/cephfs-houchangjian/envs/conda/envs/mineru/lib/python3.11/site-packages/vllm/v1/engine/utils.py", line 785, in wait_for_engine_startup
raise RuntimeError("Engine core initialization failed. "
RuntimeError: Engine core initialization failed. See root cause above. Failed core proc(s): {}
ERROR: Traceback (most recent call last):
File "/home/search/houchangjian/cephfs-houchangjian/envs/conda/envs/mineru/lib/python3.11/site-packages/starlette/routing.py", line 693, in lifespan
async with self.lifespan_context(app) as maybe_state:
File "/home/search/houchangjian/cephfs-houchangjian/envs/conda/envs/mineru/lib/python3.11/contextlib.py", line 210, in aenter
return await anext(self.gen)
^^^^^^^^^^^^^^^^^^^^^
File "/da2/search/houchangjian/cephfs-houchangjian/ms-swift-main/swift/llm/infer/rollout.py", line 310, in lifespan
msg = connection.recv()
^^^^^^^^^^^^^^^^^
File "/home/search/houchangjian/cephfs-houchangjian/envs/conda/envs/mineru/lib/python3.11/multiprocessing/connection.py", line 250, in recv
buf = self._recv_bytes()
^^^^^^^^^^^^^^^^^^
File "/home/search/houchangjian/cephfs-houchangjian/envs/conda/envs/mineru/lib/python3.11/multiprocessing/connection.py", line 430, in _recv_bytes
buf = self._recv(4)
^^^^^^^^^^^^^
File "/home/search/houchangjian/cephfs-houchangjian/envs/conda/envs/mineru/lib/python3.11/multiprocessing/connection.py", line 399, in _recv
raise EOFError
EOFError
ERROR: Application startup failed. Exiting.
Your hardware and system info
Write your system info like CUDA version/system/GPU/torch version here(在这里给出硬件信息和系统信息，如CUDA版本，系统，GPU型号和torch版本等)
flash_attn 2.8.3+cu12torch28cxx11abitrue
torch 2.8.0
torchaudio 2.8.0
torchvision 0.23.0
A100
Additional context
Add any other context about the problem here(在这里补充其他信息)
在使用qwenvl 3进行启动的时候确没有问题，但是在paddle 却不行
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reproducing the PaddleOCR VL Swift rollout startup and inspect the traceback locations in packages/torch/_dynamo/eval_frame.py and vllm/compilation/decorators.py. Collect the complete exception and environment details, then confirm that the rollout model starts successfully without the reported error.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, pytorch
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100