kvcache-ai / kvcache-ai/AgentENV

Add locality-based scheduling using node cache awareness

Open
#15 1 comment 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
Rust
Stars
3.5k
Forks
309
Avg merge
1d 9h
Merged PRs (30d)
66

Description

## Problem

AgentENV should add a locality-based scheduling strategy that places new sandboxes on nodes that already hold the most relevant cache for the requested workload.

Today the distributed scheduler can choose strategies such as round-robin, but it does not account for per-node caches. That can place a sandbox on a cold node even when another node already has useful local artifacts, such as overlaybd layers, converted image commits, memory snapshot commits, or fixed snapshot artifacts.

The scheduler should prefer warm nodes when they have enough available resources. It should only place the sandbox on colder or new nodes when the warmer candidates lack sufficient CPU, memory, disk, or other required capacity.

## Proposal

Add a locality-aware scheduling strategy, for example `locality` or `cache_aware`, that:

- Filters candidate nodes by hard resource availability.
- Scores feasible nodes by cache coverage for the requested template, snapshot, or `userImage`.
- Chooses the feasible node with the highest cache-locality score.
- Falls back to colder or new nodes only when warm nodes do not have enough resources, or when no useful cache metadata is known.

The cache score can consider local availability of rootfs/image layers, overlaybd commits, memory snapshot commits, fixed snapshot artifacts, and related P2P-published artifacts where applicable.

Contributor guide

Open the contributing guide

Research direction

Start by locating the distributed scheduler and the node-cache metadata used for rootfs layers, snapshot commits, and related artifacts. Define how candidate filtering, cache scoring, and fallback behavior should integrate with the existing scheduling strategies, then identify tests that can verify warm-node preference and resource-based fallback.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
backend, distributed-systems
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.