NVIDIA / NVIDIA/apex

No module named 'fast_self_multihead_attn'

Open
#958 4 comments 4 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
9k
Forks
1.5k
Avg merge
2d 4h
Merged PRs (30d)
3

Description

Hi,
Iwas trying run the fast-MHA example here:
https://github.com/NVIDIA/apex/tree/master/apex/contrib/multihead_attn

I got the following errors when running the performance:

Traceback (most recent call last):
  File "perf_test_multihead_attn.py", line 5, in <module>
    from apex.contrib.multihead_attn import SelfMultiheadAttn
  File "/home/apex/apex/contrib/multihead_attn/__init__.py", line 1, in <module>
    from .self_multihead_attn import SelfMultiheadAttn
  File "/home/apex/apex/contrib/multihead_attn/self_multihead_attn.py", line 9, in <module>
    from .fast_self_multihead_attn_func          import fast_self_attn_func
  File "/home/apex/apex/contrib/multihead_attn/fast_self_multihead_attn_func.py", line 2, in <module>
    import fast_self_multihead_attn
ModuleNotFoundError: No module named 'fast_self_multihead_attn'

The Apex packaged is installed with CPP and CUDA extension turned on. And I was not able to find the definition of this missing function anywhere.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with apex/contrib/multihead_attn/fast_self_multihead_attn_func.py and the perf_test_multihead_attn.py example, then inspect how the native extension is built and installed. Re-run the performance example after investigating the missing fast_self_multihead_attn import and confirm that the module imports successfully.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp, python, pytorch
Domain
machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.