Plans to support "mlp" feature merge embeddings?
Nobody has claimed this yet.
- Dominant language
- C++
- Stars
- 4.7k
- Forks
- 537
- Avg merge
- 12h 12m
- Merged PRs (30d)
- 4
Description
Tried to convert a transformer model from OpenNMT-py which uses "mlp" for it's embeddings but MLP hasn't been incorporated into conversion, so I assume it's not supported in CTranslate2.
onmt_release_model --model ./saves/model_step_52000.pt --format ctranslate2 --quantization int8 --output ./ct2
Traceback (most recent call last):
File "C:\ProgramData\Anaconda3\lib\runpy.py", line 197, in _run_module_as_main
return _run_code(code, main_globals, None,
File "C:\ProgramData\Anaconda3\lib\runpy.py", line 87, in _run_code
exec(code, run_globals)
File "C:\ProgramData\Anaconda3\Scripts\onmt_release_model.exe\__main__.py", line 7, in <module>
File "C:\Users\admin\AppData\Roaming\Python\Python39\site-packages\onmt\bin\release_model.py", line 30, in main
converter.convert(opt.output, force=True,
File "C:\Users\admin\AppData\Roaming\Python\Python39\site-packages\ctranslate2\converters\converter.py", line 89, in convert
model_spec = self._load()
File "C:\Users\admin\AppData\Roaming\Python\Python39\site-packages\ctranslate2\converters\opennmt_py.py", line 175, in _load
return _get_model_spec_seq2seq(
File "C:\Users\admin\AppData\Roaming\Python\Python39\site-packages\ctranslate2\converters\opennmt_py.py", line 81, in _get_model_spec_seq2seq
embeddings_merge=_SUPPORTED_FEATURES_MERGE[feat_merge],
KeyError: 'mlp'
Wanted to trial and error mlp against the normal concat to compare and contrast how it may affect results, as they seemed to be promising
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in converters/opennmt_py.py at _SUPPORTED_FEATURES_MERGE and _get_model_spec_seq2seq, where the traceback shows the unsupported "mlp" value is handled. Read how OpenNMT-py represents this embedding merge mode, then verify that conversion succeeds and the resulting model can be used for comparison with concat.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp, python
- Domain
- machine-learning, tooling
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100