microsoft / microsoft/onnxruntime

[Feature Request] Beam Search for Encoder-Decoder model - Merging only Decoder & Decoder-Init with Beam Search Node

Open
#19,718 1 comment 0 reactions 0 assignees View on GitHub
feature request model:transformer
Dominant language
C++
Stars
21.9k
Forks
4.2k
Avg merge
4d 11h
Merged PRs (30d)
184

Description

### Describe the feature request

Currently there is support for beam search for T5 model. But the 3 models (encoder, decoder, decoder_with_past) or 2 models (encoder_decoder_init, decoder) would be joined into a single model. But I am looking at this from a different way. Instead of bundling encoder and decoder into a single model with beam search, if we can just use decoder and decoder init and bundle it with beam search node, we can switch different encoder models and have a single decoder model with beam search. Let me know if this is possible.

### Describe scenario use case

Switch different encoder models and have a single decoder model with beam search. If encoder and decoder are joined, the number of models increases.

Contributor guide

Open the contributing guide

Research direction

No files or tests are identified in the issue. Start by reviewing the existing T5 beam-search support and the encoder, decoder, decoder_with_past, and encoder_decoder_init model arrangements mentioned here. Done would be a clear feasibility result and implementation scope for bundling decoder and decoder-init with beam search while allowing encoder substitution.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
machine-learning
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.