single_sbatch routing: rework agent-driven (DSE/live-RL) handling and mixed-job guard ordering
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 99
- Forks
- 62
- Avg merge
- 6d 12h
- Merged PRs (30d)
- 17
Description
Context
handle_dry_run_and_run (src/cloudai/cli/handlers.py) routes scenarios as:
agent_driven = [_is_dse_or_live_rl(tr) for tr in test_scenario.test_runs]
if args.single_sbatch or not any(agent_driven): # in this mode cases are unrolled using grid search
handle_non_dse_job(runner, args)
return 0
if all(agent_driven):
return handle_dse_job(runner, args)
logging.error("Mixing agent-driven (DSE / live-RL) and plain jobs is not allowed.")
return 1
The args.single_sbatch short-circuit is evaluated before the mixed-job guard. This ordering predates the live-RL work (it keyed on tr.is_dse_job originally) and is benign for DSE, because SingleSbatchRunner deliberately grid-unrolls DSE param spaces (unroll_dse / all_trs / handle_dse). But it has two rough edges:
- Mixed agent-driven + plain under
--single-sbatchbypasses the "mixing not allowed" error and silently routes everything throughhandle_non_dse_job. SingleSbatchRunnerhas no live-RL path — it only branches ontr.is_dse_job. Alive_rl_modejob under--single-sbatchwould silently run once as a static job, because live-RL needs the in-processGymServerloop, not a batch grid.
A narrow stopgap guard was added for case (2) (error on live_rl_mode + single_sbatch) with a TODO pointing at this issue. This issue tracks the proper fix.
Proposed fix (dedicated PR)
- Reject mixed agent-driven + plain before the
single_sbatchshort-circuit. - Make single_sbatch semantics explicit per agent type: DSE -> grid-unroll (current behavior), live-RL -> unsupported/error (or a live-RL-aware single-sbatch path if ever needed).
- Add regression tests covering
single_sbatch=Truefor DSE, live-RL, plain, and mixed scenarios.
Files
src/cloudai/cli/handlers.py(handle_dry_run_and_run,_is_dse_or_live_rl)src/cloudai/systems/slurm/single_sbatch_runner.py(all_trs,handle_dse)tests/test_handlers.py(_run_routing+ routing tests)
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in src/cloudai/cli/handlers.py at handle_dry_run_and_run and _is_dse_or_live_rl, then inspect all_trs and handle_dse in src/cloudai/systems/slurm/single_sbatch_runner.py. Use the _run_routing tests in tests/test_handlers.py as the entry point. Done means mixed jobs are rejected before single_sbatch routing, with regression coverage for DSE, live-RL, plain, and mixed scenarios.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- cli, testing
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Quiet
- Clarity
- Clearly specified
- Newbie friendliness
- 74/100