mlcommons / mlcommons/inference
GPT-J errors
Open
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 1.6k
- Forks
- 650
- Avg merge
- 1d 22h
- Merged PRs (30d)
- 6
Description
command:
cm run script --tags=run-mlperf,inference,_find-performance,_full,_r4.1 \
--model=gptj-99 \
--implementation=reference \
--framework=pytorch \
--category=edge \
--scenario=Offline \
--execution_mode=test \
--device=cpu \
--docker --quiet \
--test_query_count=50
error:
Encoding Samples
Finished constructing QSL.
Loading PyTorch model...
Loading checkpoint shards: 0%| | 0/3 [00:00<?, ?it/s]
Traceback (most recent call last):
File "/home/cmuser/.local/lib/python3.10/site-packages/transformers/modeling_utils.py", line 575, in load_state_dict
return torch.load(
File "/home/cmuser/.local/lib/python3.10/site-packages/torch/serialization.py", line 1087, in load
overall_storage = torch.UntypedStorage.from_file(os.fspath(f), shared, size)
RuntimeError: unable to mmap 10004248818 bytes from file </home/cmuser/CM/repos/local/cache/5de735f7d99448f8/checkpoint/checkpoint-final/pytorch_model-00001-of-00003.bin>: Cannot allocate memory (12)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/cmuser/.local/lib/python3.10/site-packages/transformers/modeling_utils.py", line 584, in load_state_dict
if f.read(7) == "version":
File "/usr/lib/python3.10/codecs.py", line 322, in decode
(result, consumed) = self._buffer_decode(data, self.errors, final)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0x80 in position 128: invalid start byte
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/cmuser/CM/repos/local/cache/da5cd8ab8fe54ea3/inference/language/gpt-j/main.py", line 170, in <module>
main()
File "/home/cmuser/CM/repos/local/cache/da5cd8ab8fe54ea3/inference/language/gpt-j/main.py", line 109, in main
sut = get_SUT(
File "/home/cmuser/CM/repos/local/cache/da5cd8ab8fe54ea3/inference/language/gpt-j/backend_PyTorch.py", line 238, in get_SUT
return SUT_Offline(model_path, dtype, dataset_path, scenario, max_examples, use_gpu, network, qsl)
File "/home/cmuser/CM/repos/local/cache/da5cd8ab8fe54ea3/inference/language/gpt-j/backend_PyTorch.py", line 173, in __init__
SUT_base.__init__(self, model_path, dtype, dataset_path, scenario, max_examples, use_gpu, network, qsl)
File "/home/cmuser/CM/repos/local/cache/da5cd8ab8fe54ea3/inference/language/gpt-j/backend_PyTorch.py", line 45, in __init__
self.model = AutoModelForCausalLM.from_pretrained(
File "/home/cmuser/.local/lib/python3.10/site-packages/transformers/models/auto/auto_factory.py", line 564, in from_pretrained
return model_class.from_pretrained(
File "/home/cmuser/.local/lib/python3.10/site-packages/transformers/modeling_utils.py", line 3941, in from_pretrained
) = cls._load_pretrained_model(
File "/home/cmuser/.local/lib/python3.10/site-packages/transformers/modeling_utils.py", line 4395, in _load_pretrained_model
state_dict = load_state_dict(shard_file, is_quantized=is_quantized)
File "/home/cmuser/.local/lib/python3.10/site-packages/transformers/modeling_utils.py", line 596, in load_state_dict
raise OSError(
OSError: Unable to load weights from pytorch checkpoint file for '/home/cmuser/CM/repos/local/cache/5de735f7d99448f8/checkpoint/checkpoint-final/pytorch_model-00001-of-00003.bin' at '/home/cmuser/CM/repos/local/cache/5de735f7d99448f8/checkpoint/checkpoint-final/pytorch_model-00001-of-00003.bin'. If you tried to load a PyTorch model from a TF 2.0 checkpoint, please set from_tf=True.
Finished destroying SUT.
./run.sh: line 59: 1: command not found
./run.sh: line 65: 1: command not found
CM error: Portable CM script failed (name = benchmark-program, return code = 32512)
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reproducing the reported cm run script command, then inspect inference/language/gpt-j/main.py, backend_PyTorch.py, and the referenced run.sh errors. Determine why the GPT-J checkpoint cannot be loaded and address the reported failure so the command completes without the checkpoint-loading or shell errors.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, pytorch
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100