deepspeedai / deepspeedai/DeepSpeed
[BUG]RuntimeError: Called with more columns than supported, please report this bug and this limit will be increased."
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 43.1k
- Forks
- 5k
- Avg merge
- 4d 15h
- Merged PRs (30d)
- 112
Description
I want to integrate the Qwen model, but an error occurred:
File "/deepspeed/inference/v2/model_implementations/qwen14b/qwen_model.py", line 159, in _forward_transformer_layer
hidden_states = self.mlp_1(hidden_states, cur_params.mlp_1_w, b=None)
File "/usr/local/lib/python3.8/dist-packages/torch/nn/modules/module.py", line 1501, in _call_impl
return forward_call(*args, **kwargs)
File "/deepspeed/inference/v2/modules/implementations/linear/blas_fp_linear.py", line 89, in forward
self._act_fn(output, staging_output, b)
File "/deepspeed/inference/v2/kernels/core_ops/gated_activations/gated_activation.py", line 65, in call
self.kernel(output, input, bias, self.act_fn.value)
RuntimeError: Called with more columns than supported, please report this bug and this limit will be increased.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with deepspeed/inference/v2/model_implementations/qwen14b/qwen_model.py at _forward_transformer_layer, then follow the calls into blas_fp_linear.py and gated_activation.py. Reproduce the Qwen integration failure and identify the input shape that triggers the reported column limit; done means the model runs without this RuntimeError and the behavior is verified for the affected case.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, pytorch
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100