hoangsonww / hoangsonww/AegisVision-Computer-Vision-System

Feature: GPU placement simulator for pipeline rollout planning

Open
#18 0 comments 0 reactions 1 assignee Claimed by @hoangsonww View on GitHub
bug documentation enhancement good first issue help wanted question
Dominant language
Go
Stars
1
Forks
0
PR merge metrics
No merged PRs in 30d

Description

## Summary
Add a GPU placement simulator that predicts MIG slice usage, latency risk, and admission outcomes before operators activate new streams, models, or pipeline revisions.

## Problem / Opportunity
The project has a GPU scheduler, MIG defaults, canary/shadow workflows, and cost accounting, but operators lack a dry-run planning tool. Today, a rollout can be syntactically valid yet operationally risky because the expected GPU slice mix, concurrency, fallback behavior, or tenant contention is not visible until execution.

## Proposed Feature
Expose a simulation API and console workflow that takes proposed streams, model versions, pipeline revisions, and tenant priorities, then returns expected placement, resource pressure, cost estimate, and admission/refusal reasons without changing live scheduler state.

## Scope
- Add scheduler dry-run inputs for candidate workloads and time windows.
- Model MIG slice availability, current reservations, tenant priority, and known model resource profiles.
- Return placement candidates, bottlenecks, and refusal explanations.
- Integrate with pipeline/model promotion and stream creation flows as an optional preflight.
- Add metrics and docs for interpreting simulation confidence.
- Add tests with synthetic cluster topologies.

## Acceptance Criteria
- [ ] Users can dry-run a candidate pipeline/model/stream rollout against current scheduler state.
- [ ] Results include projected MIG allocation, unresolved constraints, and estimated incremental GPU cost.
- [ ] The simulator distinguishes hard admission failures from soft risk warnings.
- [ ] No live reservations are created during simulation.
- [ ] Console rollout screens can invoke simulation before activation.
- [ ] Documentation describes simulator assumptions and failure modes.

## Non-Goals
- Perfect performance prediction for every model/hardware combination.
- Replacing live canary, shadow inference, or SLO gates.
- Adding spot bidding or budget guardrails, which are already roadmap topics.

## Dependencies / Risks
- Requires accurate model resource profiles; missing profiles should degrade gracefully.
- Simulated results can become stale quickly in busy clusters.
- Needs clear UX to avoid implying guarantees the simulator cannot provide.

## Open Questions
- Should simulations be persisted for auditability when used before regulated rollouts?
- How should the simulator account for bursty stream FPS and quality changes?

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.