code-yeongyu / code-yeongyu/oh-my-openagent

Feature: add runtime smoke-check mode to omo doctor (ping configured agents/provider paths)

Open
#3,333 1 comment 0 reactions 1 assignee View on GitHub

@code-yeongyu is already working on this.

Since May 16, 2026.

enhancement triage:feature-request
Dominant language
TypeScript
Stars
69.2k
Forks
5.7k
Avg merge
3h 2m
Merged PRs (30d)
620

Description

Summary

omo doctor validates config shape, but it still leaves a major operator blind spot: it does not prove that each configured agent can actually start and answer with the current provider/model/auth wiring.

Discord context

This came up again in UltraWorkers Discord #feedback and related OMO setup discussion.
Relevant user feedback included:

  • it would be amazing if the omo doctor command actually just pinged each agent with "hi" so we can make sure that it is working with our config/models
  • users asking how they are supposed to test config when some agents cannot even be selected or when latest versions regress in specific setups

Channel context:

  • #feedback
  • guild: UltraWorkers (1452487457085063218)
  • representative message id: 1491951789602508852

Problem

Right now omo doctor can still report a config that looks valid while the real runtime is broken in ways users only discover later:

  • agent selection/UI mismatch
  • provider auth mismatch
  • model alias mismatch
  • provider-specific startup failures
  • delegation/runtime regressions after upgrades

That creates a fake-normal layer: config appears healthy, but the actual agent path is dead.

Expected behavior

Add a mode like one of these:

  • omo doctor --smoke
  • omo doctor --ping-agents
  • omo doctor --verify-runtime

That mode should:

  1. enumerate configured agents
  2. try a tiny real invocation per agent (or per unique provider/model path)
  3. report success/failure with the exact provider/model used
  4. show failures in a compact actionable way

Minimal smoke contract

For each agent (or deduped runtime path), run something intentionally tiny like:

  • prompt: Reply with exactly: OK
  • low token budget
  • short timeout

Report:

  • agent name
  • provider
  • model
  • success/failure
  • error summary if failed

Why this matters

This would catch the failures users actually care about before they start a real task:

  • broken delegation paths
  • stale/bad provider wiring
  • broken aliases
  • auth regressions after version changes

Notes / design guardrails

  • This should probably be opt-in, not the default lightweight doctor path
  • Deduping by runtime path may be useful to avoid spamming expensive calls
  • Even one tiny real round-trip is more valuable than pure config validation in many broken setups

Workaround today

Users must manually discover runtime failure by launching real tasks or poking agents one by one, which is slow and noisy.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.