Support for arbitrary sequence length in FMHA

Open
#1,395 2 comments 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
5/5
Estimated time
Over a week
Newbie friendliness
35/100
Issue type
Feature
Clarity
Mostly clear
Activity status
Stale
Tech stack
python, pytorch

Research direction

Start with run_fmha.py and the linked Apex patch, then compare the existing FMHA implementation with the benchmarked arbitrary-length approach in the gist. Done means supporting arbitrary sequence lengths for head_dim 64 and 128 while preserving the reported efficiency; use the provided benchmark procedure to verify the result.

Written by the indexing model from the issue text.

Description

We experimented with porting existing fmha implementation to efficiently support arbitrary sequence length for head_dim=64/128 and seem to have found a working solution by computing blocks one at a time and accumulating results.

Here's our approach and benchmarks: https://gist.github.com/krunt/72197074816dfe4035fcd9413e4afb22

Here's our implementation (patch to apex): https://github.com/krunt/apex/compare/727a6452c9b781930acee5e24e09efe9360b4890...3655f21606256efdcae092a0b4fd0ac1151d2de8
You can also find the implementation for head dim 128 in the above gist.
Here's how to run/benchmark it: https://github.com/krunt/apex/blob/arbitlen_fmha_headdim_64/run_fmha.py

Authors: @krunt, @TimDettmers, @xtinkt
We also thank @yjk21 @jdemouth and @jaredcasper for helpful discussions
Almost the same idea independently proposed in: https://arxiv.org/abs/2205.14135
Similar idea with full re-materialization proposed in: https://arxiv.org/abs/2112.05682

Dominant language
Python
Stars
9k
Forks
1.5k
Avg merge
2d 4h
Merged PRs (30d)
3

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from NVIDIA/apex

All issues in NVIDIA/apex

Similar issues

More Python issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.