Make trajectory cache opt-in per workload (cache_safe contract)
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 99
- Forks
- 62
- Avg merge
- 6d 12h
- Merged PRs (30d)
- 17
Description
Problem
CloudAIGymEnv.get_cached_trajectory_result() (in src/cloudai/configurator/cloudai_gym.py) returns the previously recorded (reward, observation) whenever a step's action matches an earlier entry's action. The returned tuple is then written to trajectory.csv and returned to the caller as if the workload had been re-executed.
This is correct only if the workload is deterministic given the action. For a stochastic workload, a cached (reward, observation) is a single sample from a distribution; reusing it instead of re-executing silently biases the recorded trajectory and any downstream consumer (DSE analysis, offline training corpora, leaderboards).
There is currently no way for a workload to declare that caching is unsafe for it.
Proposed change
Add a workload-level cache_safe: bool = True declaration on TestDefinition. CloudAIGymEnv.get_cached_trajectory_result() returns None whenever self.test_run.test.test_definition.cache_safe is False, forcing re-execution.
- Default remains
Trueto preserve current behavior for the existing deterministic workloads. - Stochastic workloads override to
Falsein theirTestDefinitionsubclass (or in TOML). - No change to consumers of
trajectory.csv: the cache becomes a property of the workload, not a property the consumer has to reason about.
Acceptance criteria
TestDefinitionexposescache_safe: bool = True.get_cached_trajectory_result()returnsNonewhencache_safeis False, regardless of trajectory contents.- Unit test: a
TestDefinitionwithcache_safe=Falsere-executes on a duplicate action;cache_safe=True(default) returns the cached entry as today. - Documentation note next to
cache_safestating the determinism contract.
Out of scope
Changing any existing workload's cache_safe value. Each workload owner decides separately.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with src/cloudai/configurator/cloudai_gym.py and locate the TestDefinition declaration and existing tests for get_cached_trajectory_result(). Trace how duplicate actions are handled, then add coverage for both cache_safe=False and the default behavior. Done means unsafe workloads re-execute while safe workloads retain the current cached result, with the determinism contract documented next to the declaration.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- tooling
- Issue type
- Feature
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Quiet
- Clarity
- Clearly specified
- Newbie friendliness
- 72/100