deepspeedai / deepspeedai/DeepSpeed
[BUG] Guard check fails after deep-compiling a model that calls tensor.expand()
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 43.1k
- Forks
- 5k
- Avg merge
- 4d 15h
- Merged PRs (30d)
- 112
Description
Describe the bug
Trying to deep-compile a model calling tensor.expand() triggers the following guard error:
[rank0]: Traceback (most recent call last):
[rank0]: File "/Playground/gist/deepcompile/extend.py", line 46, in <module>
[rank0]: o = m(x)
[rank0]: File "/venv-3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1751, in _wrapped_call_impl
[rank0]: return self._call_impl(*args, **kwargs)
[rank0]: File "/venv-3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1762, in _call_impl
[rank0]: return forward_call(*args, **kwargs)
[rank0]: File "/Projects/deepspeed/deepspeed/utils/nvtx.py", line 20, in wrapped_fn
[rank0]: ret_val = func(*args, **kwargs)
[rank0]: File "/Projects/deepspeed/deepspeed/runtime/engine.py", line 2106, in forward
[rank0]: loss = self.module(*inputs, **kwargs)
[rank0]: File "/venv-3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1749, in _wrapped_call_impl
[rank0]: return self._compiled_call_impl(*args, **kwargs) # type: ignore[misc]
[rank0]: File "/venv-3.10/lib/python3.10/site-packages/torch/_dynamo/eval_frame.py", line 655, in _fn
[rank0]: return fn(*args, **kwargs)
[rank0]: File "/venv-3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1762, in _call_impl
[rank0]: return forward_call(*args, **kwargs)
[rank0]: File "/venv-3.10/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 1432, in __call__
[rank0]: return self._torchdynamo_orig_callable(
[rank0]: File "/venv-3.10/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 1213, in __call__
[rank0]: result = self._inner_convert(
[rank0]: File "/venv-3.10/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 598, in __call__
[rank0]: return _compile(
[rank0]: File "/venv-3.10/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 1059, in _compile
[rank0]: guarded_code = compile_inner(code, one_graph, hooks, transform)
[rank0]: File "/venv-3.10/lib/python3.10/site-packages/torch/_utils_internal.py", line 97, in wrapper_function
[rank0]: return function(*args, **kwargs)
[rank0]: File "/venv-3.10/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 761, in compile_inner
[rank0]: return _compile_inner(code, one_graph, hooks, transform)
[rank0]: File "/venv-3.10/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 906, in _compile_inner
[rank0]: check_fn = CheckFunctionManager(
[rank0]: File "/venv-3.10/lib/python3.10/site-packages/torch/_dynamo/guards.py", line 2514, in __init__
[rank0]: raise AssertionError(f"Guard check failed: {reasons}")
[rank0]: AssertionError: Guard check failed: 0/0: tensor 'self._parameters['cls_token']' rank mismatch. expected 3, actual 1. Guard failed on a parameter, consider using torch._dynamo.config.force_parameter_static_shapes = False to allow dynamism on parameters.
To Reproduce
Run this script: https://gist.github.com/eternalNight/89ad0639abba0d51ca7777a91d0b07a0
Expected behavior
Forward graph is generated without triggering any guard check error.
System info (please complete the following information):
- Python version: 3.10.13
- Deepspeed commit:
43f00ba3 Remove additional unused tests (human-eval) (#7445)(i.e.v0.17.2-16-g43f00ba3)
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Run the reproduction script from the linked gist and inspect the Guard check failure through torch/_dynamo/guards.py and the compilation path shown in the traceback. Verify that a model calling tensor.expand() can generate its forward graph without the parameter rank-mismatch error, then add or run a regression test if the relevant test location is identified.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, pytorch
- Domain
- machine-learning, performance
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 52/100