HewlettPackard / HewlettPackard/FastSim

chore: pre-feature cleanup — regression baseline, config landmines, logging/serialization hygiene

Open
#2 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
4
Forks
2
PR merge metrics
No merged PRs in 30d

Description

## What / Why

Before starting the jobs-data input feature (changing FastSim's input from raw SLURM dumps to a jobs-data format), clean up known landmines and establish the regression safety net the refactor depends on. All items below were verified against the code on `feature/jobs-data-input` (see `dev/202607062252-determinism-check.md` for the determinism verification this builds on).

## Subtasks

### Regression safety net (do first)

- [x] Promote the canonicalized results-comparison script into the repo as `scripts/compare_results.py` (canonicalizes set order, compares `Job`s by `jid`, handles enums/`MappingProxyType`, depth-capped recursion — see dev note for rationale)
- [x] Capture and document a baseline results pickle for a reference config so future refactors can diff against it

### Bugs / landmines

- [x] Add a default for `save_interval_steps` in `scheduler/config.py` — `controller.py:750` reads it unconditionally and a YAML that omits it crashes mid-run
- [x] Fix U+2010 unicode hyphen in skip reason `"NOT-ENOUGH-NODES‐NOW"` (`controller.py:1212`) — lands in `wait_history` output and breaks ASCII matching
- [x] Replace the seven bare `except:` blocks in `controller.py` (lines 771, 888, 937, 1417, 1488, 1530, 1711) with specific exceptions or at least logged failures

### Hygiene

- [x] Fix hardcoded log dir `'../'` in `main.py:246` — with repo-root CWD convention, logs scatter into the repo's parent; derive from output path or default to `./logs`
- [x] Deduplicate job-history serialization (checkpoint path `controller.py:752-775` vs final save `main.py:257-278`) into one function
- [x] Open the power log once at init instead of re-checking/reopening the CSV every simulation step (`controller.py:2185+`)
- [x] Remove duplicate `re_fp` key in `configs/kestrel_conf.yaml` (lines 40-41; YAML silently keeps the second, `""`)
- [x] Add `*.zip` to `.gitignore` (`power_data.zip` / `slurm_dump.zip`, ~108 MB, currently one `git add .` from being committed)

### Deferred (fold into jobs-data input feature design)

- [ ] Output format: stop pickling live simulator objects (`Dependency`, `JobState`, `Job` refs) — results pickles currently require `scheduler/` importable to load

## Notes

- Determinism of the simulator was verified 2026-07-06: two identical 3-day Kestrel runs produce identical job histories after canonicalization (`dev/202607062252-determinism-check.md`).
- Work happens on a branch off `feature/jobs-data-input`, PR'd back into that branch, one commit per subtask where practical.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by checking the completed subtasks in scripts/compare_results.py, scheduler/config.py, scheduler/controller.py, main.py, configs/kestrel_conf.yaml, and .gitignore against dev/202607062252-determinism-check.md and the reference results baseline. Then identify what remains, especially the deferred output-format work, and run the documented determinism comparison to confirm the regression baseline still holds.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
backend
Issue type
Refactor
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.