HewlettPackard / HewlettPackard/FastSim
chore: pre-feature cleanup — regression baseline, config landmines, logging/serialization hygiene
- Dominant language
- Python
- Stars
- 4
- Forks
- 2
- PR merge metrics
- No merged PRs in 30d
Description
## What / Why
Before starting the jobs-data input feature (changing FastSim's input from raw SLURM dumps to a jobs-data format), clean up known landmines and establish the regression safety net the refactor depends on. All items below were verified against the code on `feature/jobs-data-input` (see `dev/202607062252-determinism-check.md` for the determinism verification this builds on).
## Subtasks
### Regression safety net (do first)
- [x] Promote the canonicalized results-comparison script into the repo as `scripts/compare_results.py` (canonicalizes set order, compares `Job`s by `jid`, handles enums/`MappingProxyType`, depth-capped recursion — see dev note for rationale)
- [x] Capture and document a baseline results pickle for a reference config so future refactors can diff against it
### Bugs / landmines
- [x] Add a default for `save_interval_steps` in `scheduler/config.py` — `controller.py:750` reads it unconditionally and a YAML that omits it crashes mid-run
- [x] Fix U+2010 unicode hyphen in skip reason `"NOT-ENOUGH-NODES‐NOW"` (`controller.py:1212`) — lands in `wait_history` output and breaks ASCII matching
- [x] Replace the seven bare `except:` blocks in `controller.py` (lines 771, 888, 937, 1417, 1488, 1530, 1711) with specific exceptions or at least logged failures
### Hygiene
- [x] Fix hardcoded log dir `'../'` in `main.py:246` — with repo-root CWD convention, logs scatter into the repo's parent; derive from output path or default to `./logs`
- [x] Deduplicate job-history serialization (checkpoint path `controller.py:752-775` vs final save `main.py:257-278`) into one function
- [x] Open the power log once at init instead of re-checking/reopening the CSV every simulation step (`controller.py:2185+`)
- [x] Remove duplicate `re_fp` key in `configs/kestrel_conf.yaml` (lines 40-41; YAML silently keeps the second, `""`)
- [x] Add `*.zip` to `.gitignore` (`power_data.zip` / `slurm_dump.zip`, ~108 MB, currently one `git add .` from being committed)
### Deferred (fold into jobs-data input feature design)
- [ ] Output format: stop pickling live simulator objects (`Dependency`, `JobState`, `Job` refs) — results pickles currently require `scheduler/` importable to load
## Notes
- Determinism of the simulator was verified 2026-07-06: two identical 3-day Kestrel runs produce identical job histories after canonicalization (`dev/202607062252-determinism-check.md`).
- Work happens on a branch off `feature/jobs-data-input`, PR'd back into that branch, one commit per subtask where practical.
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by checking the completed subtasks in scripts/compare_results.py, scheduler/config.py, scheduler/controller.py, main.py, configs/kestrel_conf.yaml, and .gitignore against dev/202607062252-determinism-check.md and the reference results baseline. Then identify what remains, especially the deferred output-format work, and run the documented determinism comparison to confirm the regression baseline still holds.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- backend
- Issue type
- Refactor
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100