microsoft / microsoft/onnxruntime
[Feature Request] Beam Search for Encoder-Decoder model - Merging only Decoder & Decoder-Init with Beam Search Node
- Dominant language
- C++
- Stars
- 21.9k
- Forks
- 4.2k
- Avg merge
- 4d 11h
- Merged PRs (30d)
- 184
Description
### Describe the feature request
Currently there is support for beam search for T5 model. But the 3 models (encoder, decoder, decoder_with_past) or 2 models (encoder_decoder_init, decoder) would be joined into a single model. But I am looking at this from a different way. Instead of bundling encoder and decoder into a single model with beam search, if we can just use decoder and decoder init and bundle it with beam search node, we can switch different encoder models and have a single decoder model with beam search. Let me know if this is possible.
### Describe scenario use case
Switch different encoder models and have a single decoder model with beam search. If encoder and decoder are joined, the number of models increases.
Contributor guide
Research direction
No files or tests are identified in the issue. Start by reviewing the existing T5 beam-search support and the encoder, decoder, decoder_with_past, and encoder_decoder_init model arrangements mentioned here. Done would be a clear feasibility result and implementation scope for bundling decoder and decoder-init with beam search while allowing encoder substitution.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- machine-learning
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 20/100