galaxyproject / galaxyproject/loom
Brain should flag basic count invariants in generated analysis summaries (parts must sum to the whole)
- Dominant language
- TypeScript
- Stars
- 14
- Forks
- 12
- Avg merge
- 6d 5h
- Merged PRs (30d)
- 17
Description
### Observed
During a local analysis session the agent generated a classifier and a summary report. One section reported a parent total of 172 items while the per-category breakdown summed to 174. The notebook itself noted this as "a small artifact from the classifier's final else branch" -- i.e. the model *recognized* the discrepancy but still wrote the inconsistent summary into the notebook and moved on.
### Expected
When generating analysis scripts and summary tables, the brain should apply cheap arithmetic invariant checks: category counts must sum exactly to their stated parent total. On mismatch, the generated script should fail loudly, or Orbit should clearly flag the output as internally inconsistent rather than silently committing it.
### Why this is a tractable guardrail
This is a generalizable, domain-agnostic numeric invariant (parts sum to whole), unlike domain-specific output-correctness cases. It fits the existing verify/guardrail precedents (#23 verify artifacts, #288/#289 DESeq2 guardrails). Suggested as brain-guidance: steer generated count/summary tables to be self-consistency-checked (sum-to-total, no orphaned else-branch leakage) before being written to the notebook, and flagged inline if they aren't.
### Environment
Orbit 0.4.1, Linux x64, provider anthropic / claude-sonnet-4-6.
Contributor guide
No contributing guide indexed for this repository
Research direction
No implementation files or tests are named. Start by reading the existing verify/guardrail precedents in issues #23, #288, and #289, then locate the brain guidance and notebook-summary generation entry points; done means mismatched parent and category counts are rejected or clearly flagged before the output is committed.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- typescript
- Domain
- ai
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100