multica-ai / multica-ai/multica
[Feature]: Dry-run mode for autopilots and tasks — render the prompt without spending tokens
- Dominant language
- Go
- Stars
- 50k
- Forks
- 6.5k
- Avg merge
- 1d 36m
- Merged PRs (30d)
- 500
Description
### What do you want and why?
A `--dry-run` mode (or equivalent task-level flag) that lets an operator **rehearse** an autopilot or one-off task without actually invoking the agent CLI, so the cost and shape of "what would have happened" is visible *before* spending tokens.
### The current gap
When you wire up a new autopilot — cron trigger, agent assignment, prompt template — there's no way to ask the platform "if I turned this on, what would the next 24h look like?" The first real signal you get is the agent CLI actually running, billing tokens, and writing to the issue. If the prompt template references an issue field that's empty for most issues, or the trigger fires more often than intended, you find out by paying for it.
Combined with how usage data surfaces — task_usage is recorded on completion and rolled up hourly (`taskusagebackfill`) — by the time you see the burn on the dashboard, it's already an hour stale and several runs deep.
This shows up in three places where it would meaningfully change operator behavior:
1. **Autopilot validation.** "I just created a trigger that fires every weekday at 9am. Show me the actual rendered prompt for next Monday's run, without spending money."
2. **Spend forecasting.** Pairs naturally with the spend-cap proposal in #4076: "if I set this autopilot live, the projected daily cost is ~$X / ~Y tokens / ~Z turns at the configured `--max-turns`."
3. **Tightening prompt templates.** Right now the round-trip is: edit prompt → wait for next fire → read transcript. A dry-run shortcut is "render the prompt for these last 5 issues" and read them side-by-side.
### Proposal
Add `ExecOptions.DryRun bool` in `server/pkg/agent/agent.go`. When set, the agent backends (claude.go, codex.go, etc.) **build the full command line and prompt exactly as production**, but instead of spawning the CLI, return a structured `DryRunResult{ RenderedPrompt, CommandArgs, EnvKeys, EstimatedInputTokens }` to the daemon. The daemon writes this into the task's `result` JSON with a `dry_run: true` marker. UI badges the task as a rehearsal.
Surface points:
- CLI: `multica autopilot trigger --dry-run` — useful for ops.
- CLI: `multica issue create … --dry-run` — useful for prompt iteration.
- API: a `dry_run` field on the trigger / chat-message endpoints.
- Optional: `multica autopilot forecast --window 7d` — compute projected runs from the cron expression × estimated tokens per run.
### Scope discipline
- No new schema. `result` JSON is already opaque; the marker lives there.
- No changes to the agent CLI integrations beyond a `DryRun` short-circuit in the spawn step.
- Token *estimate* uses a coarse `len(rendered_prompt) / 4` heuristic — exact accounting requires hitting the model, which defeats the purpose.
- Skill discovery and runtime config injection still run (so dry-run faithfully shows what an executor would actually receive), they're just not followed by a real model call.
### Companion PR
I'm posting a draft implementation in a follow-up PR (linked when ready). Wanted to file this issue first so the design conversation has a home that isn't a 400-line diff.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Contributor guide
Research direction
Start in server/pkg/agent/agent.go and inspect how ExecOptions reaches the agent backends in claude.go, codex.go, and related files. Trace the autopilot trigger and issue-create CLI entry points plus the API endpoints named in the issue. Done means production-equivalent prompts and command inputs are returned without spawning the agent CLI, with the dry-run result recorded in task result JSON.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- api, backend, cli
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100