mlcommons / mlcommons/inference

GPT-J errors

Open
#1,826 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
1.6k
Forks
650
Avg merge
1d 22h
Merged PRs (30d)
6

Description

command:

cm run script --tags=run-mlperf,inference,_find-performance,_full,_r4.1 \
   --model=gptj-99 \
   --implementation=reference \
   --framework=pytorch \
   --category=edge \
   --scenario=Offline \
   --execution_mode=test \
   --device=cpu  \
   --docker --quiet \
   --test_query_count=50

error:

Encoding Samples
Finished constructing QSL.
Loading PyTorch model...
Loading checkpoint shards:   0%|                                                                                    | 0/3 [00:00<?, ?it/s]
Traceback (most recent call last):
  File "/home/cmuser/.local/lib/python3.10/site-packages/transformers/modeling_utils.py", line 575, in load_state_dict
    return torch.load(
  File "/home/cmuser/.local/lib/python3.10/site-packages/torch/serialization.py", line 1087, in load
    overall_storage = torch.UntypedStorage.from_file(os.fspath(f), shared, size)
RuntimeError: unable to mmap 10004248818 bytes from file </home/cmuser/CM/repos/local/cache/5de735f7d99448f8/checkpoint/checkpoint-final/pytorch_model-00001-of-00003.bin>: Cannot allocate memory (12)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/cmuser/.local/lib/python3.10/site-packages/transformers/modeling_utils.py", line 584, in load_state_dict
    if f.read(7) == "version":
  File "/usr/lib/python3.10/codecs.py", line 322, in decode
    (result, consumed) = self._buffer_decode(data, self.errors, final)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0x80 in position 128: invalid start byte

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/cmuser/CM/repos/local/cache/da5cd8ab8fe54ea3/inference/language/gpt-j/main.py", line 170, in <module>
    main()
  File "/home/cmuser/CM/repos/local/cache/da5cd8ab8fe54ea3/inference/language/gpt-j/main.py", line 109, in main
    sut = get_SUT(
  File "/home/cmuser/CM/repos/local/cache/da5cd8ab8fe54ea3/inference/language/gpt-j/backend_PyTorch.py", line 238, in get_SUT
    return SUT_Offline(model_path, dtype, dataset_path, scenario, max_examples, use_gpu, network, qsl)
  File "/home/cmuser/CM/repos/local/cache/da5cd8ab8fe54ea3/inference/language/gpt-j/backend_PyTorch.py", line 173, in __init__
    SUT_base.__init__(self, model_path, dtype, dataset_path, scenario, max_examples, use_gpu, network, qsl)
  File "/home/cmuser/CM/repos/local/cache/da5cd8ab8fe54ea3/inference/language/gpt-j/backend_PyTorch.py", line 45, in __init__
    self.model = AutoModelForCausalLM.from_pretrained(
  File "/home/cmuser/.local/lib/python3.10/site-packages/transformers/models/auto/auto_factory.py", line 564, in from_pretrained
    return model_class.from_pretrained(
  File "/home/cmuser/.local/lib/python3.10/site-packages/transformers/modeling_utils.py", line 3941, in from_pretrained
    ) = cls._load_pretrained_model(
  File "/home/cmuser/.local/lib/python3.10/site-packages/transformers/modeling_utils.py", line 4395, in _load_pretrained_model
    state_dict = load_state_dict(shard_file, is_quantized=is_quantized)
  File "/home/cmuser/.local/lib/python3.10/site-packages/transformers/modeling_utils.py", line 596, in load_state_dict
    raise OSError(
OSError: Unable to load weights from pytorch checkpoint file for '/home/cmuser/CM/repos/local/cache/5de735f7d99448f8/checkpoint/checkpoint-final/pytorch_model-00001-of-00003.bin' at '/home/cmuser/CM/repos/local/cache/5de735f7d99448f8/checkpoint/checkpoint-final/pytorch_model-00001-of-00003.bin'. If you tried to load a PyTorch model from a TF 2.0 checkpoint, please set from_tf=True.
Finished destroying SUT.
./run.sh: line 59: 1: command not found
./run.sh: line 65: 1: command not found

CM error: Portable CM script failed (name = benchmark-program, return code = 32512)

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing the reported cm run script command, then inspect inference/language/gpt-j/main.py, backend_PyTorch.py, and the referenced run.sh errors. Determine why the GPT-J checkpoint cannot be loaded and address the reported failure so the command completes without the checkpoint-loading or shell errors.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, pytorch
Domain
machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.