Apply ctranslate2 to KNN-MT
Nobody has claimed this yet.
- Dominant language
- C++
- Stars
- 4.7k
- Forks
- 536
- Avg merge
- 12h 12m
- Merged PRs (30d)
- 4
Description
Hi!
I want to apply ctranslate2 to KNN-MT (There are some pytorch implementations, knn-box,
and sockeye for example). Is there a corresponding interface to get the output hidden state of the model in order to do vector retrieval?
In addition, since KNN-MT needs to do vector retrieval for each decoding step, it needs to be decoded word by word, while currently ctranslate2 only provides an interface to decode the whole sentence at once. Is it possible to provide an interface to reuse the encoder output at each decoding step to reduce redundant calculations?
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Begin by reading CTranslate2's existing whole-sentence decoding interface and comparing it with the KNN-MT requirements described here. Determine how callers could obtain decoder hidden states for vector retrieval and reuse encoder output during word-by-word decoding; the work is done when both capabilities are available through a documented interface.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- machine-learning
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100