Can't load models in reproducing the benchmark numbers from the README
Nobody has claimed this yet.
- Dominant language
- C++
- Stars
- 4.7k
- Forks
- 537
- Avg merge
- 12h 12m
- Merged PRs (30d)
- 4
Description
Step 5/5 : RUN touch /tmp/input.txt && /translate CPU /tmp/input.txt /tmp/output.txt && rm /tmp/input.txt
---> Running in 8ca5f61ea6e8
Traceback (most recent call last):
File "/translate", line 13, in
tokenizer = transformers.MarianTokenizer.from_pretrained(model_name)
File "/opt/conda/lib/python3.10/site-packages/transformers/tokenization_utils_base.py", line 1788, in from_pretrained
raise EnvironmentError(
OSError: Can't load tokenizer for 'Helsinki-NLP/opus-mt-en-de'. If you were trying to load it from 'https://huggingface.co/models', make sure you don't have a local directory with the same name. Otherwise, make sure 'Helsinki-NLP/opus-mt-en-de' is the correct path to a directory containing all relevant files for a MarianTokenizer tokenizer.
Traceback (most recent call last):
File "benchmark_all.py", line 117, in
Image(
File "benchmark_all.py", line 46, in init
raise RuntimeError(log["error"])
RuntimeError: The command '/bin/sh -c touch /tmp/input.txt && /translate CPU /tmp/input.txt /tmp/output.txt && rm /tmp/input.txt' returned a non-zero code: 1
im trying to reproducing the benchmark numbers from the README.it seem like the tokenizer did't exict at huggingface.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the README benchmark instructions and benchmark_all.py at lines 117 and 46, then inspect /translate at line 13 where MarianTokenizer loads Helsinki-NLP/opus-mt-en-de. Reproduce the Docker command and determine why the tokenizer cannot be loaded; done means the benchmark command completes and its expected output or numbers can be confirmed.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100