microsoft / microsoft/ai-dev-gallery
[BUG] Incompatible Model Combination Leads to Sample Can Not Be Loaded
Nobody has claimed this yet.
- Dominant language
- C#
- Stars
- 1.5k
- Forks
- 223
- PR merge metrics
- No merged PRs in 30d
Description
When a scenario contains multiple samples with different model type requirements, the model picker allows users to select model combinations that no individual sample supports, resulting in viableSamples.Count == 0 and preventing any sample from loading.
Steps to Reproduce
-
Navigate to a scenario that contains multiple samples with different
Model1Typesrequirements- For example, Text -> Retrieval Augmented Generation (RAG) might have:
- Sample A requiring
Model1Types = [LanguageModels] - Sample B requiring
Model1Types = [KnowledgeRetrieval]
- Sample A requiring
- For example, Text -> Retrieval Augmented Generation (RAG) might have:
-
Open the model picker - observe that it shows models from both
LanguageModelsandKnowledgeRetrievaltypes (union of all samples' requirements) -
Select the 1st model "Retrieval Augmented Generation (RAG)" (that belongs to
KnowledgeRetrievaltype) -
Select the 2nd model "all-MiniLM" (that belongs to
KnowledgeRetrievaltype, too)
-
Click "Run Sample"
-
The sample can not be loaded then always show the loading image:
Expected Behavior
Either:
- The model picker should only show models that are compatible with at least one complete sample configuration, OR
- The UI should provide clear feedback explaining why the selection is invalid
Actual Behavior
- The model picker closes and immediately reopens without any error message
- No sample loads
- User is left confused with no indication of what went wrong
- The debug output shows:
viableSamples.Count == 0
Code Analysis
Location: ScenarioPage.xaml.cs
Collection Logic (LoadPicker, lines ~99-106)
// Merges ALL model types from ALL samples (UNION)
List<List<ModelType>> modelDetailsList = [samples.SelectMany(s => s.Model1Types).ToList()];
This allows the picker to show models from any type that any sample needs.
Validation Logic (HandleModelSelectionChanged, lines ~249-251)
// Requires finding a sample that matches BOTH selected models (INTERSECTION)
List<Sample> viableSamples = samples!.Where(s =>
IsModelFromTypes(s.Model1Types, selectedModels[0]) &&
IsModelFromTypes(s.Model2Types, selectedModels[1])).ToList();
This requires at least one sample to support the entire selected model combination.
The Mismatch: Collection uses union logic, but validation uses intersection logic, creating a gap where users can select "theoretically valid but practically unsupported" model combinations.
Debug Output Example
[IsModelFromTypes] Checking model: 6a526fdd-359f-4eac-9aa6-f01db11ae542, Name: Retrieval Augmented Generation (RAG)
[IsModelFromTypes] Required types: LanguageModels, PhiSilica
[IsModelFromTypes] Total 14 model IDs collected. Model 6a526fdd-359f-4eac-9aa6-f01db11ae542 NOT FOUND
[IsModelFromTypes] Checking model: 6a526fdd-359f-4eac-9aa6-f01db11ae542
[IsModelFromTypes] Required types: KnowledgeRetrieval, PhiSilica
[IsModelFromTypes] Total 2 model IDs collected. Model 6a526fdd-359f-4eac-9aa6-f01db11ae542 FOUND
The RAG model is found in KnowledgeRetrieval type but not in LanguageModels type, yet both types were presented as valid options to the user.
Additional Notes
- The code comment states
// this should never happen, but this scenario does occur in practice - When
viableSamples.Count == 0, the code simply callsApp.MainWindow.ModelPicker.Show(selectedModels)without providing any user feedback about why the selection failed
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in ScenarioPage.xaml.cs with LoadPicker and HandleModelSelectionChanged, comparing how model types are collected with how viableSamples are validated. Reproduce the incompatible selection described in the issue, then make the picker prevent unsupported complete configurations or provide clear invalid-selection feedback, and verify that a valid sample can load.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- csharp
- Domain
- desktop
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100