ContextLab / ContextLab/clustrix
Restore SGE support: removed from v0.2.0 as never verified against a real scheduler
- Dominant language
- Python
- Stars
- 10
- Forks
- 4
- Avg merge
- 6h 27m
- Merged PRs (30d)
- 9
Description
SGE (Sun Grid Engine / Grid Engine) support is implemented and removed from v0.2.0 as unverified. It has never been run against a real Grid Engine installation.
## What exists, and where
Introduced in `4b2aba5` (2025-06-24). At `299109f`:
| Code | Location |
|-|-|
| Job submission | `clustrix/executor_schedulers.py:251` `submit_sge_job` |
| Status polling | `clustrix/executor_scheduler_status.py:517` `_check_sge_status` |
| Dispatch | `clustrix/executor_core.py:115,553` |
| Script generation | `clustrix/utils.py:2803` `_create_sge_script`; dispatch at `:2511,2548` |
| Tutorial | `docs/source/notebooks/sge_tutorial.ipynb` |
## Specific risk to check on restoration
SGE shares the SLURM/PBS staging and environment-setup path, but its directive syntax (`#$ -l h_rt=`, `#$ -pe`) and its `qstat` output format are distinct from both. The status parser and the resource-directive generator are the two places where an untested assumption would show up first: a misparsed `qstat` line means clustrix reports a running job as finished, which is a silent wrong answer rather than an error.
## Why it is being removed rather than fixed
Not because the code is known to be wrong. Because it has **never been run against the real thing**, and shipping it in the cluster-type dropdown states otherwise. A user who selects it gets a code path no one has ever seen succeed.
v0.2.0 keeps exactly the four backends that have been demonstrated end to end -- `local`, `ssh`, `slurm`, `huggingface` -- and the documentation now says the rest are planned for a future release rather than currently supported.
## Restoring it
Nothing is lost: every line cited above stays reachable in git history at the commits named. Reinstating it means reverting the removal commit and then doing the part that was never done -- running it against real hardware and recording the evidence in this issue.
## Definition of done
- [ ] Backend restored from history
- [ ] A real job submitted, executed and its result returned, with the transcript pasted into this issue
- [ ] Failure paths exercised (job rejected, job killed, node lost)
- [ ] Re-added to `SUPPORTED_CLUSTER_TYPES`, the widget dropdown and the CLI
- [ ] Documentation moved from "planned" to "supported"
Contributor guide
Research direction
Start with the cited SGE entry points in clustrix/executor_schedulers.py, executor_scheduler_status.py, executor_core.py, and utils.py, and restore the implementation from the named git history. Run a real submission and result-return workflow, then exercise rejected, killed, and lost-node jobs while checking qstat parsing and resource directives. Done includes updating SUPPORTED_CLUSTER_TYPES, the widget and CLI, moving the documentation to supported, and recording the evidence transcript here.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- distributed-systems
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 30/100