galaxyproject / galaxyproject/loom

Brain should flag basic count invariants in generated analysis summaries (parts must sum to the whole)

Open
#318 0 comments 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
TypeScript
Stars
14
Forks
12
Avg merge
6d 5h
Merged PRs (30d)
17

Description

### Observed
During a local analysis session the agent generated a classifier and a summary report. One section reported a parent total of 172 items while the per-category breakdown summed to 174. The notebook itself noted this as "a small artifact from the classifier's final else branch" -- i.e. the model *recognized* the discrepancy but still wrote the inconsistent summary into the notebook and moved on.

### Expected
When generating analysis scripts and summary tables, the brain should apply cheap arithmetic invariant checks: category counts must sum exactly to their stated parent total. On mismatch, the generated script should fail loudly, or Orbit should clearly flag the output as internally inconsistent rather than silently committing it.

### Why this is a tractable guardrail
This is a generalizable, domain-agnostic numeric invariant (parts sum to whole), unlike domain-specific output-correctness cases. It fits the existing verify/guardrail precedents (#23 verify artifacts, #288/#289 DESeq2 guardrails). Suggested as brain-guidance: steer generated count/summary tables to be self-consistency-checked (sum-to-total, no orphaned else-branch leakage) before being written to the notebook, and flagged inline if they aren't.

### Environment
Orbit 0.4.1, Linux x64, provider anthropic / claude-sonnet-4-6.

Contributor guide

No contributing guide indexed for this repository

Research direction

No implementation files or tests are named. Start by reading the existing verify/guardrail precedents in issues #23, #288, and #289, then locate the brain guidance and notebook-summary generation entry points; done means mismatched parent and category counts are rejected or clearly flagged before the output is committed.

Written by the indexing model from the issue text.

Assessment

Tech stack
typescript
Domain
ai
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.