lllyasviel / lllyasviel/FramePack

ROCm : Sage Attn Out Of Resources

Open
#470 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
17.3k
Forks
1.7k
PR merge metrics
No merged PRs in 30d

Description

Hi,

On Ubuntu, RX 7900 XTX (GFX1100). Sage Attn is installed by default.

```
sageattention 1.0.6
torch 2.7.0+rocm6.2.4
torchaudio 2.7.0+rocm6.2.4
triton 3.3.0
```

$dem_graio.py

```
Currently enabled native sdp backends: ['flash', 'math', 'mem_efficient', 'cudnn']
Xformers is not installed!
Flash Attn is not installed!
Sage Attn is installed!
```

These are the error messages:

```
Unloaded SiglipVisionModel as complete.
Moving DynamicSwap_HunyuanVideoTransformer3DModelPacked to cuda:0 with preserved memory: 6 GB
0%| | 0/25 [00:00

File "/home/user/sd/FramePack/diffusers_helper/models/hunyuan_video_packed.py", line 111, in attn_varlen_func
x = sageattn(q, k, v, tensor_layout='NHD')
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/user/miniconda3/lib/python3.12/site-packages/sageattention/core.py", line 110, in sageattn
o = attn_false(q_int8, k_int8, v, q_scale, k_scale, tensor_layout=tensor_layout, output_dtype=dtype)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/user/miniconda3/lib/python3.12/site-packages/sageattention/attn_qk_int8_per_block.py", line 113, in forward
_attn_fwd[grid](
File "/home/user/miniconda3/lib/python3.12/site-packages/triton/runtime/jit.py", line 347, in
return lambda *args, **kwargs: self.run(grid=grid, warmup=False, *args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/user/miniconda3/lib/python3.12/site-packages/triton/runtime/jit.py", line 591, in run
kernel.run(grid_0, grid_1, grid_2, stream, kernel.function, kernel.packed_metadata,
^^^^^^^^^^
File "/home/user/miniconda3/lib/python3.12/site-packages/triton/compiler/compiler.py", line 413, in __getattribute__
self._init_handles()
File "/home/user/miniconda3/lib/python3.12/site-packages/triton/compiler/compiler.py", line 401, in _init_handles
raise OutOfResources(self.metadata.shared, max_shared, "shared memory")
triton.runtime.errors.OutOfResources: out of resource: shared memory, Required: 90112, Hardware limit: 65536. Reducing block sizes or `num_stages` may help.
```

There is no error when using `flash_attn 2.7.4.post1`

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with diffusers_helper/models/hunyuan_video_packed.py at attn_varlen_func and reproduce the reported SageAttention path on the RX 7900 XTX with the listed PyTorch and ROCm versions. Trace the call into the installed SageAttention and Triton packages; done means the attention path runs without the shared-memory OutOfResources failure.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, pytorch
Domain
machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.