[Requests and ideas] Multi-Model Subagent Orchestration System

**Mood:** 😊
**Category:** Requests and ideas

# MULTI-MODEL SUBAGENT ORCHESTRATION SYSTEM

## 1. Problem Statement

Currently, when a user interacts within a Session configured with a text-only Large Language Model (LLM), the system triggers an unsupported error if the input contains multimodal data such as images. This limits user experience and restricts the system's flexibility in handling complex Tasks that require distinct strengths from different models.

## 2. Proposed Solution

We propose a mechanism for **Dynamic Input-Based Model Configuration** and **Task-Based Subagent Orchestration**. Instead of forcing the main Session to process all types of data and tasks, the system will automatically decompose and delegate workloads to specialized Subagents running dedicated models, then aggregate the results back to the main Session.

## 3. Detailed Workflow

### Phase 1: Input Type-Based Routing

- **Trigger Condition:** The user uploads an image into a Session currently configured with a text-only model.
- **Workflow:**
1. The system detects that the input contains an image.
2. The system looks up the Configuration to find the pre-defined Vision-supported model for this specific data type.
3. A **Subagent** is initialized in the background, sending the image to the Vision model for analysis.
4. The Subagent converts the image analysis results into a detailed textual description.
5. This textual description is returned and fed into the Context of the original text-only model in the main Session to generate the final response for the user.

### Phase 2: Extension - Task-Based Routing

- **Trigger Condition:** The system identifies a complex workflow consisting of multiple stages (e.g., Analysis followed by Coding).
- **Workflow:**
1. The user configures the workflow pipeline, or the system automatically recognizes the task type via the Prompt.
2. During the "Analysis" stage, the system activates a Subagent running a model optimized for reasoning (e.g., GPT-5.3).
3. Once the analysis is complete, based on the task definition configuration, the system proactively triggers another Subagent running a model optimized for source code (e.g., Claude Opus) to execute the coding task.
4. All intermediate results are seamlessly consolidated and delivered back to the main Session.

## 4. Key Benefits

- **Cost and Performance Optimization:** Eliminates the need to use expensive Multimodal models for the entire conversation when only a few simple images need to be processed.
- **Enhanced Accuracy:** Leverages the specialized strengths of individual models (e.g., combining a strong logic/analysis model with a powerful coding model).
- **Seamless User Experience:** Users no longer encounter "format not supported" errors, as the system automatically handles the processing in the background through seamless integration.

---
| Field | Value |
| --- | --- |
| App version | 1.0.15 |
| OS | macOS 26.5.1 |
| Theme | GitHub |
| Path | /chat |
| Tenure | Week 4 |

Bắt đầu bằng việc xem xét trải nghiệm /chat và cách xử lý hiện tại đối với đầu vào đa phương thức không được hỗ trợ. Đề xuất này không nêu tên các tệp triển khai hoặc bài kiểm thử; để hoàn thành sẽ cần một thiết kế routing và orchestration được xác định rõ, có thể xử lý đầu vào hình ảnh và kết quả của subagent qua nhiều giai đoạn mà không gặp lỗi hiện tại.

Do mô hình lập chỉ mục viết ra từ nội dung của issue.