con / con/mechababs

iterate decide(): merge / failure-handling policy refinements (defer to real-data)

Open
#66 0 comments 0 reactions 0 assignees View on GitHub
automation decision
Dominant language
Python
Stars
1
Forks
4
Avg merge
15h 39m
Merged PRs (30d)
24

Description

Collects the deferred refinements to the reconciler's `decide()` policy — how a scaffolded-but-unmerged `(dataset, pipeline)` cell resolves once its jobs finish. These are **mechababs-only policy** (no babs change, no provenance / data-quality impact), so they're deliberately deferred: the friction they'd reduce is only knowable once we're processing real data, and the whole point is to be able to tweak them mid-campaign.

`decide()` (in `mechababs/babs_status.py`) is the single tweak-point.

## Current default (shipping with `babs status --json` consumption)
From the `babs status --json` counts:
- `unsubmitted > 0` → submit
- `in_progress > 0` (derived as `submitted - done - failed`) → skip / wait
- `failed > 0` (all ended, some failed) → **fail**: surface loudly (which jobs, where the logs are), do **not** auto-merge; the cell re-surfaces each tick until resolved
- else → merge

Conservative by design: never silently publishes a partial set, and loses no information.

## Deferred refinements
1. **Merge-on-partial-failure.** There's real value in merging the *successful* jobs even when some failed:
- selection bug — some jobs *shouldn't* succeed (wrong data for the job); the rest are valid and worth merging.
- want to inspect successful outputs, which requires a merge first.

Against: `babs merge` may become a SLURM compute job (#46), so it's not free to trigger speculatively.
2. **`merged` ledger column: boolean → count.** `babs merge` is re-runnable and a partial merge is a real state — track *how many* jobs are merged, not just true/false, so a rerun after fixing failures merges the newcomers cleanly. Touches the `state.py` schema.
3. **Failure information-flow.** The operator should learn which jobs failed → why (a quick path to logs) → act → rerun. Overlaps the FAIL.txt report (#59) and retries (M4).

## Why defer (not design now)
This is friction-tuning with no data-quality / provenance impact. The intent is to adjust it *after* data collection starts, and the requirements aren't fully knowable until then — so starting simple loses nothing.

Related: #59 (FAIL.txt report), #13 (done-detection), #46 (merge as a compute job), M4-E2E-Automation (retries). The submit-side "keep ≥N jobs in flight" cap is a separate concurrency-governor concern, not this issue.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start in mechababs/babs_status.py at decide() and review the current babs status --json counts and related issues #59, #13, #46, and M4. Examine state.py for the merged ledger schema; done requires an agreed real-data policy, with the selected refinements implemented without changing babs, provenance, or data-quality behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
tooling
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.