OpenNMT / OpenNMT/CTranslate2

Error while converting to Ctranslate2 from openNMTPy.

Open
#1,767 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
C++
Stars
4.7k
Forks
536
Avg merge
12h 12m
Merged PRs (30d)
4

Description

I have a openNMTPy model translator.pt. While converting it using ct2-opennmt-py-converter --model_path himangy_triton_repo/himangy-hi-pa/1/2.pt --output_dir ct2_model --quantization "int8" command. I get following error :-

  File "/opt/anaconda3/envs/NMT/bin/ct2-opennmt-py-converter", line 8, in <module>
    sys.exit(main())
  File "/opt/anaconda3/envs/NMT/lib/python3.10/site-packages/ctranslate2/converters/opennmt_py.py", line 355, in main
    OpenNMTPyConverter(args.model_path).convert_from_args(args)
  File "/opt/anaconda3/envs/NMT/lib/python3.10/site-packages/ctranslate2/converters/converter.py", line 50, in convert_from_args
    return self.convert(
  File "/opt/anaconda3/envs/NMT/lib/python3.10/site-packages/ctranslate2/converters/converter.py", line 89, in convert
    model_spec = self._load()
  File "/opt/anaconda3/envs/NMT/lib/python3.10/site-packages/ctranslate2/converters/opennmt_py.py", line 181, in _load
    check_opt(checkpoint["opt"], num_source_embeddings=len(src_vocabs))
  File "/opt/anaconda3/envs/NMT/lib/python3.10/site-packages/ctranslate2/converters/opennmt_py.py", line 55, in check_opt
    check.validate()
  File "/opt/anaconda3/envs/NMT/lib/python3.10/site-packages/ctranslate2/converters/utils.py", line 106, in validate
    raise_unsupported(self._unsupported_reasons)
  File "/opt/anaconda3/envs/NMT/lib/python3.10/site-packages/ctranslate2/converters/utils.py", line 93, in raise_unsupported
    raise ValueError(message)
ValueError: The model you are trying to convert is not supported by CTranslate2. We identified the following reasons:

- Options --encoder_type and --decoder_type must be 'transformer' or 'transformer_lm
- Options --position_encoding and --max_relative_positions cannot be both enabled or both disabled

Ctranslate2 - ctranslate2==4.3.1
OpenNMTPy - OpenNMT-py==1.2.0

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Reproduce the reported command with the OpenNMT-py checkpoint, then inspect ctranslate2/converters/opennmt_py.py and ctranslate2/converters/utils.py around check_opt and validate. Determine whether the encoder, decoder, and position-option checks correctly handle this model; done means the conversion failure has a confirmed resolution or documented compatibility outcome.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
machine-learning, tooling
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.