[Bug] Atlas 300IDUO下无法使用Qwen3 32B进行推理

### Checklist

- [x] 1. I have searched related issues but cannot get the expected help.
- [ ] 2. The bug has not been fixed in the latest version.
- [ ] 3. Please note that if the bug-related issue you submitted lacks corresponding environment info and a minimal reproducible demo, it will be challenging for us to reproduce and resolve the issue, reducing the likelihood of receiving feedback.

### Describe the bug

在昇腾设备Atlas 300IDUO上可以跑起来Qwen3-32B，但是无法进行推理；使用官方提供的镜像，在容器内通过命令启动，可以正常加载模型，但是无法进行推理

### Reproduction

镜像：使用[官方文档](https://lmdeploy.readthedocs.io/zh-cn/latest/get_started/ascend/get_started.html)提供[镜像](crpi-4crprmm5baj1v8iv.cn-hangzhou.personal.cr.aliyuncs.com/lmdeploy_dlinfer/ascend:310p-latest)
启动命令：ASCEND_RT_VISIBLE_DEVICES=4,5 lmdeploy serve api_server --model-name Qwen3-32B --backend pytorch --tp 2 --dtype bfloat16 --device ascend /root/models/Qwen3-32B --server-port 11025

### Environment

```Shell
使用官方提供的镜像
```

### Error traceback

```Shell
INFO: Started server process [23]
INFO: Waiting for application startup.
INFO: Application startup complete.
INFO: Uvicorn running on http://0.0.0.0:11025 (Press CTRL+C to quit)

('Warning: torch.save with "_use_new_zipfile_serialization = False" is not recommended for npu tensor, which may bring unexpected errors and hopefully set "_use_new_zipfile_serialization = True"', 'if it is necessary to use this, please convert the npu tensor to cpu tensor for saving')
(RayWorkerWrapper pid=883) mki_log log dir:/root/atb/log exist
(RayWorkerWrapper pid=883) [2025-08-08 03:39:45.508] [dicp] [error] [model.cpp:478] atb CreateOperation graph failed, st: 2
(pid=883) 2025-08-08 03:37:24,561 - lmdeploy - WARNING - __init__.py:10 - Disable DLSlime Backend
(RayWorkerWrapper pid=629) 2025-08-08 03:39:45,602 - lmdeploy - ERROR - model_agent.py:679 - Task failed
(RayWorkerWrapper pid=629) Traceback (most recent call last):
(RayWorkerWrapper pid=629) File "/opt/lmdeploy/lmdeploy/pytorch/engine/model_agent.py", line 674, in _on_finish_callback
(RayWorkerWrapper pid=629) task.result()
(RayWorkerWrapper pid=629) File "/opt/lmdeploy/lmdeploy/pytorch/engine/model_agent.py", line 646, in _async_loop_background
(RayWorkerWrapper pid=629) await self._async_step_background(**forward_inputs, )
(RayWorkerWrapper pid=629) File "/opt/lmdeploy/lmdeploy/pytorch/engine/model_agent.py", line 578, in _async_step_background
(RayWorkerWrapper pid=629) output = await self._async_model_forward(
(RayWorkerWrapper pid=629) File "/opt/lmdeploy/lmdeploy/pytorch/engine/model_agent.py", line 401, in _async_model_forward
(RayWorkerWrapper pid=629) ret = await __forward(inputs)
(RayWorkerWrapper pid=629) File "/opt/lmdeploy/lmdeploy/pytorch/engine/model_agent.py", line 361, in __forward
(RayWorkerWrapper pid=629) return await self.async_forward(inputs, swap_in_map=swap_in_map, swap_out_map=swap_out_map)
(RayWorkerWrapper pid=629) File "/opt/lmdeploy/lmdeploy/pytorch/engine/model_agent.py", line 848, in async_forward
(RayWorkerWrapper pid=629) output = self._forward_impl(inputs, swap_in_map=swap_in_map, swap_out_map=swap_out_map)
(RayWorkerWrapper pid=629) File "/opt/lmdeploy/lmdeploy/pytorch/engine/model_agent.py", line 832, in _forward_impl
(RayWorkerWrapper pid=629) output = model_forward(
(RayWorkerWrapper pid=629) File "/usr/local/python3.10.5/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
(RayWorkerWrapper pid=629) return func(*args, **kwargs)
(RayWorkerWrapper pid=629) File "/opt/lmdeploy/lmdeploy/pytorch/engine/model_agent.py", line 161, in model_forward
(RayWorkerWrapper pid=629) output = model(**input_dict)
(RayWorkerWrapper pid=629) File "/opt/lmdeploy/lmdeploy/pytorch/backends/graph_runner.py", line 31, in __call__
(RayWorkerWrapper pid=629) return self.model(**kwargs)
(RayWorkerWrapper pid=629) File "/usr/local/python3.10.5/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1532, in _wrapped_call_impl
(RayWorkerWrapper pid=629) return self._call_impl(*args, **kwargs)
(RayWorkerWrapper pid=629) File "/usr/local/python3.10.5/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1541, in _call_impl
(RayWorkerWrapper pid=629) return forward_call(*args, **kwargs)
(RayWorkerWrapper pid=629) File "/usr/local/python3.10.5/lib/python3.10/site-packages/torch/_dynamo/eval_frame.py", line 451, in _fn
(RayWorkerWrapper pid=629) return fn(*args, **kwargs)
(RayWorkerWrapper pid=629) File "/usr/local/python3.10.5/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1532, in _wrapped_call_impl
(RayWorkerWrapper pid=629) return self._call_impl(*args, **kwargs)
(RayWorkerWrapper pid=629) File "/usr/local/python3.10.5/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1541, in _call_impl
(RayWorkerWrapper pid=629) return forward_call(*args, **kwargs)
(RayWorkerWrapper pid=629) File "/usr/local/python3.10.5/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 921, in catch_errors
(RayWorkerWrapper pid=629) return callback(frame, cache_entry, hooks, frame_state, skip=1)
(RayWorkerWrapper pid=629) File "/usr/local/python3.10.5/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 400, in _convert_frame_assert
(RayWorkerWrapper pid=629) return _compile(
(RayWorkerWrapper pid=629) File "/usr/local/python3.10.5/lib/python3.10/contextlib.py", line 79, in inner
(RayWorkerWrapper pid=629) return func(*args, **kwds)
(RayWorkerWrapper pid=629) File "/usr/local/python3.10.5/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 676, in _compile
(RayWorkerWrapper pid=629) guarded_code = compile_inner(code, one_graph, hooks, transform)
(RayWorkerWrapper pid=629) File "/usr/local/python3.10.5/lib/python3.10/site-packages/torch/_dynamo/utils.py", line 262, in time_wrapper
(RayWorkerWrapper pid=629) r = func(*args, **kwargs)
(RayWorkerWrapper pid=629) File "/usr/local/python3.10.5/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 535, in compile_inner
(RayWorkerWrapper pid=629) out_code = transform_code_object(code, transform)
(RayWorkerWrapper pid=629) File "/usr/local/python3.10.5/lib/python3.10/site-packages/torch/_dynamo/bytecode_transformation.py", line 1036, in transform_code_object
(RayWorkerWrapper pid=629) transformations(instructions, code_options)
(RayWorkerWrapper pid=629) File "/usr/local/python3.10.5/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 165, in _fn
(RayWorkerWrapper pid=629) return fn(*args, **kwargs)
(RayWorkerWrapper pid=629) File "/usr/local/python3.10.5/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 500, in transform
(RayWorkerWrapper pid=629) tracer.run()
(RayWorkerWrapper pid=629) File "/usr/local/python3.10.5/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 2149, in run
(RayWorkerWrapper pid=629) super().run()
(RayWorkerWrapper pid=629) File "/usr/local/python3.10.5/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 810, in run
(RayWorkerWrapper pid=629) and self.step()
(RayWorkerWrapper pid=629) File "/usr/local/python3.10.5/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 773, in step
(RayWorkerWrapper pid=629) getattr(self, inst.opname)(inst)
(RayWorkerWrapper pid=629) File "/usr/local/python3.10.5/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 2268, in RETURN_VALUE
(RayWorkerWrapper pid=629) self.output.compile_subgraph(
(RayWorkerWrapper pid=629) File "/usr/local/python3.10.5/lib/python3.10/site-packages/torch/_dynamo/output_graph.py", line 981, in compile_subgraph
(RayWorkerWrapper pid=629) self.compile_and_call_fx_graph(tx, list(reversed(stack_values)), root)
(RayWorkerWrapper pid=629) File "/usr/local/python3.10.5/lib/python3.10/contextlib.py", line 79, in inner
(RayWorkerWrapper pid=629) return func(*args, **kwds)
(RayWorkerWrapper pid=629) File "/usr/local/python3.10.5/lib/python3.10/site-packages/torch/_dynamo/output_graph.py", line 1178, in compile_and_call_fx_graph
(RayWorkerWrapper pid=629) compiled_fn = self.call_user_compiler(gm)
(RayWorkerWrapper pid=629) File "/usr/local/python3.10.5/lib/python3.10/site-packages/torch/_dynamo/utils.py", line 262, in time_wrapper
(RayWorkerWrapper pid=629) r = func(*args, **kwargs)
(RayWorkerWrapper pid=629) File "/usr/local/python3.10.5/lib/python3.10/site-packages/torch/_dynamo/output_graph.py", line 1251, in call_user_compiler
(RayWorkerWrapper pid=629) raise BackendCompilerFailed(self.compiler_fn, e).with_traceback(
(RayWorkerWrapper pid=629) File "/usr/local/python3.10.5/lib/python3.10/site-packages/torch/_dynamo/output_graph.py", line 1232, in call_user_compiler
(RayWorkerWrapper pid=629) compiled_fn = compiler_fn(gm, self.example_inputs())
(RayWorkerWrapper pid=629) File "/usr/local/python3.10.5/lib/python3.10/site-packages/torch/_dynamo/repro/after_dynamo.py", line 117, in debug_wrapper
(RayWorkerWrapper pid=629) compiled_gm = compiler_fn(gm, example_inputs)
(RayWorkerWrapper pid=629) File "/usr/local/python3.10.5/lib/python3.10/site-packages/torch/_dynamo/repro/after_dynamo.py", line 117, in debug_wrapper
(RayWorkerWrapper pid=629) compiled_gm = compiler_fn(gm, example_inputs)
(RayWorkerWrapper pid=629) File "/usr/local/python3.10.5/lib/python3.10/site-packages/torch/__init__.py", line 1770, in __call__
(RayWorkerWrapper pid=629) return self.compiler_fn(model_, inputs_, **self.kwargs)
(RayWorkerWrapper pid=629) File "/usr/local/python3.10.5/lib/python3.10/site-packages/dlinfer/graph/dicp/vendor/AtbGraph/__init__.py", line 7, in atbgraph
(RayWorkerWrapper pid=629) return compile_fx(gm, fake_input_tensor, "atbgraph")
(RayWorkerWrapper pid=629) File "/usr/local/python3.10.5/lib/python3.10/site-packages/dlinfer/graph/dicp/dynamo_bridge/compile_fx.py", line 102, in compile_fx
(RayWorkerWrapper pid=629) return compile_fx_210(model_, example_inputs_, backend, inner_compile)
(RayWorkerWrapper pid=629) File "/usr/local/python3.10.5/lib/python3.10/site-packages/dlinfer/graph/dicp/dynamo_bridge/compile_fx.py", line 263, in compile_fx_210
(RayWorkerWrapper pid=629) return aot_autograd(
(RayWorkerWrapper pid=629) File "/usr/local/python3.10.5/lib/python3.10/site-packages/torch/_dynamo/backends/common.py", line 58, in compiler_fn
(RayWorkerWrapper pid=629) cg = aot_module_simplified(gm, example_inputs, **kwargs)
(RayWorkerWrapper pid=629) File "/usr/local/python3.10.5/lib/python3.10/site-packages/torch/_functorch/aot_autograd.py", line 903, in aot_module_simplified
(RayWorkerWrapper pid=629) compiled_fn = create_aot_dispatcher_function(
(RayWorkerWrapper pid=629) File "/usr/local/python3.10.5/lib/python3.10/site-packages/torch/_dynamo/utils.py", line 262, in time_wrapper
(RayWorkerWrapper pid=629) r = func(*args, **kwargs)
(RayWorkerWrapper pid=629) File "/usr/local/python3.10.5/lib/python3.10/site-packages/torch/_functorch/aot_autograd.py", line 628, in create_aot_dispatcher_function
(RayWorkerWrapper pid=629) compiled_fn = compiler_fn(flat_fn, fake_flat_args, aot_config, fw_metadata=fw_metadata)
(RayWorkerWrapper pid=629) File "/usr/local/python3.10.5/lib/python3.10/site-packages/torch/_functorch/_aot_autograd/runtime_wrappers.py", line 443, in aot_wrapper_dedupe
(RayWorkerWrapper pid=629) return compiler_fn(flat_fn, leaf_flat_args, aot_config, fw_metadata=fw_metadata)
(RayWorkerWrapper pid=629) File "/usr/local/python3.10.5/lib/python3.10/site-packages/torch/_functorch/_aot_autograd/runtime_wrappers.py", line 648, in aot_wrapper_synthetic_base
(RayWorkerWrapper pid=629) return compiler_fn(flat_fn, flat_args, aot_config, fw_metadata=fw_metadata)
(RayWorkerWrapper pid=629) File "/usr/local/python3.10.5/lib/python3.10/site-packages/torch/_functorch/_aot_autograd/jit_compile_runtime_wrappers.py", line 119, in aot_dispatch_base
(RayWorkerWrapper pid=629) compiled_fw = compiler(fw_module, updated_flat_args)
(RayWorkerWrapper pid=629) File "/usr/local/python3.10.5/lib/python3.10/site-packages/dlinfer/graph/dicp/dynamo_bridge/compile_fx.py", line 213, in fw_compiler_base
(RayWorkerWrapper pid=629) return _fw_compiler_base(model, example_inputs, is_inference)
(RayWorkerWrapper pid=629) File "/usr/local/python3.10.5/lib/python3.10/site-packages/dlinfer/graph/dicp/dynamo_bridge/compile_fx.py", line 226, in _fw_compiler_base
(RayWorkerWrapper pid=629) return inner_compile(
(RayWorkerWrapper pid=629) File "/usr/local/python3.10.5/lib/python3.10/contextlib.py", line 79, in inner
(RayWorkerWrapper pid=629) return func(*args, **kwds)
(RayWorkerWrapper pid=629) File "/usr/local/python3.10.5/lib/python3.10/site-packages/dlinfer/graph/dicp/dynamo_bridge/compile_fx.py", line 83, in compile_fx_inner
(RayWorkerWrapper pid=629) compiled_fn = gt.compile_to_fn()
(RayWorkerWrapper pid=629) File "/usr/local/python3.10.5/lib/python3.10/site-packages/dlinfer/graph/dicp/dynamo_bridge/graph.py", line 141, in compile_to_fn
(RayWorkerWrapper pid=629) return self.compile_to_module().call
(RayWorkerWrapper pid=629) File "/usr/local/python3.10.5/lib/python3.10/site-packages/dlinfer/graph/dicp/dynamo_bridge/graph.py", line 134, in compile_to_module
(RayWorkerWrapper pid=629) mod = PyCodeCache.load(code)
(RayWorkerWrapper pid=629) File "/usr/local/python3.10.5/lib/python3.10/site-packages/torch/_inductor/codecache.py", line 2137, in load
(RayWorkerWrapper pid=629) return cls.load_by_key_path(key, path, linemap, attrs)
(RayWorkerWrapper pid=629) File "/usr/local/python3.10.5/lib/python3.10/site-packages/torch/_inductor/codecache.py", line 2160, in load_by_key_path
(RayWorkerWrapper pid=629) exec(code, mod.__dict__, mod.__dict__)
(RayWorkerWrapper pid=629) File "/tmp/torchinductor_root/br/cbrwlrjtl5oovp6peqnpg2ib3l2g52c2phrd22elonxustfknrbb.py", line 21, in
(RayWorkerWrapper pid=629) kernel_cpp_0 = async_compile.compile_kernel(atb_compile_job)
(RayWorkerWrapper pid=629) File "/usr/local/python3.10.5/lib/python3.10/site-packages/dlinfer/graph/dicp/dynamo_bridge/compile.py", line 42, in compile_kernel
(RayWorkerWrapper pid=629) return DeviceKernelCache.get_kernel(device_compile_job).run
(RayWorkerWrapper pid=629) File "/usr/local/python3.10.5/lib/python3.10/site-packages/dlinfer/graph/dicp/dynamo_bridge/compile.py", line 34, in get_kernel
(RayWorkerWrapper pid=629) loaded = device_compile_job.get_compile_result()
(RayWorkerWrapper pid=629) File "/usr/local/python3.10.5/lib/python3.10/site-packages/dlinfer/graph/dicp/vendor/AtbGraph/compile_job.py", line 52, in get_compile_result
(RayWorkerWrapper pid=629) return load_and_run.AtbModel(self._input_path)
(RayWorkerWrapper pid=629) File "/usr/local/python3.10.5/lib/python3.10/site-packages/dlinfer/graph/dicp/vendor/AtbGraph/codegen/load_and_run.py", line 13, in __init__
(RayWorkerWrapper pid=629) self.model = torch.classes.DICPModel.DICPModel(model_path)
(RayWorkerWrapper pid=629) torch._dynamo.exc.BackendCompilerFailed: backend='atbgraph' raised:
(RayWorkerWrapper pid=629) RuntimeError: atb CreateOperation graph failed!
(RayWorkerWrapper pid=629)
(RayWorkerWrapper pid=629) Set TORCH_LOGS="+dynamo" and TORCHDYNAMO_VERBOSE=1 for more information
(RayWorkerWrapper pid=629)
(RayWorkerWrapper pid=629)
(RayWorkerWrapper pid=629) You can suppress this exception and fall back to eager by setting:
(RayWorkerWrapper pid=629) import torch._dynamo
(RayWorkerWrapper pid=629) torch._dynamo.config.suppress_errors = True
(RayWorkerWrapper pid=629)
```