lllyasviel / lllyasviel/FramePack
CUDA running out of memory solution older cards
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 17.3k
- Forks
- 1.7k
- PR merge metrics
- No merged PRs in 30d
Description
So older cards aka series of 10xx and below have the issue of not being able to handle bfloat's.
The cmd will till you that it is just a simple CUDA OOM issue, but to solve it, all you have to do is to change the bfloat16 values to float16 and then it will actually parse on cards with 8gb ram.
I am still figuring out what side effects this has, but atleast it will not give the OOM exception,
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reproducing the CUDA out-of-memory failure on a 10xx-or-older card with 8GB of RAM, then trace where bfloat16 values are selected. Verify whether using float16 allows the workload to parse without the OOM and check for the side effects noted in the issue before considering the work complete.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning, performance
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100