Add Support for Multiple Agents Within Single 'Agent Runtime' Server
- Dominant language
- Python
- Stars
- 670
- Forks
- 183
- Avg merge
- 17h 7m
- Merged PRs (30d)
- 358
Description
## Overview
Enable Backend.AI agent server to support multiple agents within a single physical agent process, now called ‘Agent Runtime’, allowing different scheduling units and backend configurations while maintaining backward compatibility with existing single-agent deployments.
## Background
Currently, Backend.AI uses a 1:1 mapping between physical agent servers and scheduling units. This limits flexibility when:
- Different workload types need different scheduling policies within the same physical machine
- Multiple isolated backend configurations are required on the same host
- Fine-grained resource partitioning is needed beyond the agent level
## Goals
1. **Agent Side**: Support multiple agents with backward compatibility
1. **Manager Side**: Handle mapping agent instances to the correct agent runtime in scheduling and event processing
1. **Gradual Migration**: Enable progressive rollout without disrupting existing deployments
## Architecture Changes
### Agent Server
- Store multiple agents within the same AgentRPCServer
- Add agent ID in server’s RPC functions as now the agent runtime server handles requests for multiple agents
- Ensure legacy single agent configuration works with zero config changes
### Manager Server
- Update scheduler and event handlers to make it handle agent runtime to agent mapping
## Success Criteria
- Existing single-agent deployments continue to work without changes
- New multiple agent deployments can be configured and scheduled correctly
- All agent events (heartbeat, started, terminated) properly interact with agent runtime with many agents
- Scheduler can route requests to correct agents within agent runtime
JIRA Issue: BA-2605
Contributor guide
Assessment
This issue has not been assessed yet.