galaxyproject / galaxyproject/loom
Brain prompt: respect local machine capacity when scheduling tools (CPU threads, memory)
- Dominant language
- TypeScript
- Stars
- 14
- Forks
- 12
- Avg merge
- 6d 5h
- Merged PRs (30d)
- 17
Description
## Behavior to fix
The agent runs heavy local tools — \`fastp\`, \`bwa\`, \`samtools sort\`, \`bowtie2\`, \`STAR\`, etc. — without passing thread / parallelism flags. Most of these default to **single-threaded** if no flag is given, so a 16-core laptop sits at ~6% CPU while the user waits ~16× longer than necessary.
Reported during testing: \"fastp is running, I'm not sure if it's running multi threaded.\" (For reference: fastp's flag is \`-w N\` / \`--thread N\`, default = 3 — better than single-threaded but still well under what's available.)
## Proposed change
Two parts.
**A. System prompt guidance.** Add to \`extensions/loom/context.ts\` (alongside the existing local-tool-environment block):
> **Machine capacity:** before launching a heavy local tool, check \`nproc\` (Linux) / \`sysctl -n hw.ncpu\` (macOS) and \`free -m\` (Linux) / \`vm_stat\` (macOS). Leave **2 cores** free for the OS and agent — pass the rest to the tool's thread flag (\`fastp -w\`, \`bwa mem -t\`, \`samtools sort -@\`, \`STAR --runThreadN\`, etc.). For memory-bound steps (large alignments, sort), check available RAM and don't oversubscribe — STAR genome generation alone wants ~30 GB.
Plus a small concrete table in the prompt mapping common tools → thread flag. \`fastp -w\`, \`bwa mem -t\`, \`bowtie2 -p\`, \`STAR --runThreadN\`, \`samtools view/sort -@\`, \`hisat2 -p\`, \`salmon -p\`, \`kraken2 --threads\`, \`minimap2 -t\`, \`fastqc -t\`, \`cutadapt -j\`.
**B. \`machine_info\` tool (optional).** A minimal brain tool that returns \`{ cores, totalMemoryGB, availableMemoryGB, platform, arch }\`. Avoids per-call \`nproc\`/\`free\` shell calls and lets the agent reason about it explicitly. Implementation in \`extensions/loom/tools.ts\` using \`os.cpus()\`, \`os.totalmem()\`, \`os.freemem()\`.
## Why both
The prompt change is the high-leverage move (90% of the value, 10% of the work). The tool gives the agent a deterministic, cached view of capacity without scattering shell calls — and lets us extend later (GPU detection, disk free space, etc.).
## Out of scope
- **Auto-throttling** based on live CPU/RAM during a run — too invasive; rely on the agent picking sane up-front values.
- **Concurrent job scheduling** (\"run 4 fastp at once across 4 samples\") — different concern, file separately.
- **Galaxy steps** — Galaxy server handles its own scheduling; this is local-only.
## Files
- \`extensions/loom/context.ts\` — system prompt block.
- \`extensions/loom/tools.ts\` — new \`machine_info\` tool (if going with part B).
## Verification
- Manual: in a fresh project, ask the agent to run fastp on a paired-end FASTQ. Watch \`top\` / Process tab. Before: ~3 threads. After: should pass \`-w (nproc - 2)\`.
- Same for bwa, samtools sort.
## Related
- #59 (keep FASTQ gzipped) — same theme: agent should use the canonical, performant invocation by default.
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.