vllm-project / vllm-project/aibrix

[Batch] Make one logical job independent of provider and capacity type

Open
#2,610 0 comments 0 reactions 0 assignees View on GitHub
area/batch kind/feature
Dominant language
Go
Stars
5.1k
Forks
697
Avg merge
1d 19h
Merged PRs (30d)
104

Description

## Problem

A batch job is currently coupled to a single physical resource choice and allocation. Users must decide whether to use spot or on-demand capacity and which provider should run the job. If that allocation expires, is preempted, or becomes unavailable, the job cannot transparently continue on different capacity.

This exposes infrastructure decisions to the user and makes long-running jobs operationally fragile.

## Expected outcome

A user submits one logical job. AIBrix may use spot capacity, on-demand capacity, or capacity from different configured providers during the lifetime of that job without requiring the user to resubmit it.

Provider and capacity changes must remain internal execution events. They must not create a new user-visible job or discard completed work.

Users may still provide hard constraints such as allowed providers, regions, compliance boundaries, or whether interruptible capacity is permitted.

## Acceptance criteria

- A job can start on one provider and capacity type and continue after that capacity becomes unavailable.
- The continuation keeps the same user-visible job identity, progress, cancellation operation, and final result.
- Completed work survives resource replacement.
- The final result contains no missing or duplicate committed records caused by capacity replacement.
- Job status exposes the physical capacity attempts used for operational visibility without requiring the user to manage them.
- Cancelling the logical job stops active execution and releases all capacity owned by the job.
- Provider or allocation failure is not treated as a terminal job failure while eligible capacity and retry budget remain.

## Out of scope

This issue does not choose a provider-ranking, pricing, bidding, or placement algorithm.

Contributor guide

Open the contributing guide

Research direction

No files, tests, or entry points are named. Start by locating the batch job lifecycle and its provider and capacity allocation paths, then trace identity, progress, retry, cancellation, and result-commit handling. Done means one logical job survives capacity replacement with preserved progress, consistent cancellation and results, visible physical attempts, and no duplicate or missing committed records.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
distributed-systems, infrastructure
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.