pytorch / pytorch/benchmark

llama_7b model OOM issue

Open
#2,051 3 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
1k
Forks
346
PR merge metrics
No merged PRs in 30d

Description

Hi

I duplicate the llama model and rename it into llama_7b, changed the model parameters according to llama_7b specification, looks like this:
image

skiped the CPU eager mode, only run the cuda model.

it reports the following issue when running with this command:
python userbenchmark/dynamo/dynamobench/torchbench.py -dcuda --float16 -n1 --inductor --performance --inference --filter "llama" --batch_size 1 --in_slen 32 --out_slen 3 --output-dir=torchbench_llama_test_logs
image

If I want to run this model, how should I fix it? my hardware is A100-40G

thanks

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with userbenchmark/dynamo/dynamobench/torchbench.py and reproduce the reported CUDA command using the llama filter, float16, and batch size 1. Compare the duplicated llama_7b parameters with the available model configuration and inspect the reported A100-40G out-of-memory failure. Done means identifying the cause and documenting a verified configuration or limitation for running this model.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, pytorch
Domain
machine-learning, performance
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.