OpenMOSS / OpenMOSS/MOSS

NameError: name 'transpose_matmul_248_kernel' is not defined

Open
#215 3 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
12.3k
Forks
1.1k
PR merge metrics
No merged PRs in 30d

Description

代码中缺少了对transpose_matmul_248_kernel的定义。
代码中使用的库存在问题,这个错误出现在quantization.py文件中,这个文件似乎是Hugging Face模型缓存中的一部分。
quantization.py", line 265, 应该是忘记导入 transpose_matmul_248_kernel 了,我找了一个,文件头部没有导入。
这个代码是deepspeed 微调时触发的, 走到model_engine.backward(loss) 时出的问题
"""
File "fine_tune.py", line 117, in
model_engine.backward(loss)
File "/home/zeal/pytorch-venv/lib/python3.8/site-packages/deepspeed/utils/nvtx.py", line 15, in wrapped_fn
ret_val = func(*args, **kwargs)
File "/home/zeal/pytorch-venv/lib/python3.8/site-packages/deepspeed/runtime/engine.py", line 1796, in backward
self.optimizer.backward(loss, retain_graph=retain_graph)
File "/home/zeal/pytorch-venv/lib/python3.8/site-packages/deepspeed/utils/nvtx.py", line 15, in wrapped_fn
ret_val = func(*args, **kwargs)
File "/home/zeal/pytorch-venv/lib/python3.8/site-packages/deepspeed/runtime/zero/stage3.py", line 1923, in backward
self.loss_scaler.backward(loss.float(), retain_graph=retain_graph)
File "/home/zeal/pytorch-venv/lib/python3.8/site-packages/deepspeed/runtime/fp16/loss_scaler.py", line 62, in backward
scaled_loss.backward(retain_graph=retain_graph)
File "/home/zeal/pytorch-venv/lib/python3.8/site-packages/torch/_tensor.py", line 487, in backward
torch.autograd.backward(
File "/home/zeal/pytorch-venv/lib/python3.8/site-packages/torch/autograd/init.py", line 200, in backward
Variable._execution_engine.run_backward( # Calls into the C++ engine to run the backward pass
File "/home/zeal/pytorch-venv/lib/python3.8/site-packages/torch/autograd/function.py", line 274, in apply
return user_fn(self, *args)
File "/home/zeal/pytorch-venv/lib/python3.8/site-packages/torch/cuda/amp/autocast_mode.py", line 123, in decorate_bwd
return bwd(*args, **kwargs)
File "/home/zeal/.cache/huggingface/modules/transformers_modules/fnlp/moss-moon-003-sft-plugin-int4/353c499f7415575ba217704f3f28a1e817eb7487/quantization.py", line 292, in backward
grad_input = transpose_matmul248(grad_output, qweight, scales, qzeros, g_idx, bits, maxq)
File "/home/zeal/.cache/huggingface/modules/transformers_modules/fnlp/moss-moon-003-sft-plugin-int4/353c499f7415575ba217704f3f28a1e817eb7487/quantization.py", line 265, in transpose_matmul248
transpose_matmul_248_kernel[grid](input, qweight, output,
NameError: name 'transpose_matmul_248_kernel' is not defined

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by inspecting the cached quantization.py around lines 265 and 292, then trace the backward call from fine_tune.py line 117. Confirm where transpose_matmul_248_kernel is expected to be defined or imported, and verify that model_engine.backward(loss) completes without this NameError.

Written by the indexing model from the issue text.

Assessment

Tech stack
huggingface, python, pytorch
Domain
machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.