OpenNMT / OpenNMT/CTranslate2

Exception when exporting bloomz model

Open
#1,324 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug
Dominant language
C++
Stars
4.7k
Forks
537
Avg merge
12h 12m
Merged PRs (30d)
4

Description

Operaring system: Ubuntu 22.04.2, Python 3.10.6, CTranslate2 3.16.

When exporting the bigscience/bloomz model using:

ct2-transformers-converter --force --model bigscience/bloomz --output_dir bloomz --quantization float16

The conversion process works well for other bloom models like bloomz-7b1.

Error:

Traceback (most recent call last):
File "/home/ubuntu/bloom-ctranslate2/generate/python3-env/bin/ct2-transformers-converter", line 8, in
sys.exit(main())
File "/home/ubuntu/bloom-ctranslate2/generate/python3-env/lib/python3.10/site-packages/ctranslate2/converters/transformers.py", line 1577, in main
converter.convert_from_args(args)
File "/home/ubuntu/bloom-ctranslate2/generate/python3-env/lib/python3.10/site-packages/ctranslate2/converters/converter.py", line 50, in convert_from_args
return self.convert(
File "/home/ubuntu/bloom-ctranslate2/generate/python3-env/lib/python3.10/site-packages/ctranslate2/converters/converter.py", line 104, in convert
model_spec.save(output_dir)
File "/home/ubuntu/bloom-ctranslate2/generate/python3-env/lib/python3.10/site-packages/ctranslate2/specs/model_spec.py", line 571, in save
super().save(output_dir)
File "/home/ubuntu/bloom-ctranslate2/generate/python3-env/lib/python3.10/site-packages/ctranslate2/specs/model_spec.py", line 324, in save
self._serialize(os.path.join(output_dir, "model.bin"))
File "/home/ubuntu/bloom-ctranslate2/generate/python3-env/lib/python3.10/site-packages/ctranslate2/specs/model_spec.py", line 363, in _serialize
model.write(struct.pack("I", value.nbytes))
struct.error: 'I' format requires 0 <= number <= 4294967295

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the converter flow in ctranslate2/converters/transformers.py and the serialization path in ctranslate2/specs/model_spec.py shown in the traceback. Reproduce the command with bigscience/bloomz, compare it with bloomz-7b, and trace the oversized serialized value. Done means the model exports successfully without breaking existing model conversion behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
cli, machine-learning
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.