Add runtime blueprint startup coverage with hardware/config-aware gating

Summary

During macOS release QA, I added a local runtime smoke test that launches every registered DimOS blueprint for several seconds and fails if the process exits during startup.

This test is not currently tracked on main. The local QA file was:

dimos/robot/test_blueprint_startup.py

Running that local startup coverage exposed that a large number of registered blueprints cannot be started in a generic environment because they require hardware, robot IPs, credentials, external assets, platform-specific dependencies, or intentionally fail by design.

The existing tracked dimos/robot/test_all_blueprints.py test validates blueprint importability and already maintains a SELF_HOSTED_BLUEPRINTS classification. This issue is therefore about a runtime QA/CI coverage gap and classification problem, not a regression in an existing tracked startup smoke test.

What the local QA test did

For every blueprint in all_blueprints, the local test effectively ran:

uv run dimos run <blueprint>

and allowed the process to stay alive for approximately 8 seconds.

If the process exited during startup, the test captured the startup log and reported the blueprint as failed.

Findings

The runtime QA pass exposed several categories of blueprints that need explicit handling if runtime startup coverage is added to CI:

Hardware / robot configuration required

Examples include:

• physical Unitree G1 / Go2 blueprints requiring robot IPs or network access
• XArm coordinator blueprints requiring an arm IP
• Spot blueprints requiring credentials or robot access
• OpenYAM / Damiao hardware-dependent blueprints
• R1Pro paths requiring ROS/native runtime dependencies

External assets required

Examples include:

• demo-mid360-pcap-replay requires DIMOS_MID360_PCAP or an explicit PCAP path
• spot-replay requires a Spot recording
• go2-zenoh-nav-baked requires the prebuilt dist/go2-nav executable

Platform / optional dependency limitations

Examples include:

• real-sense-camera-vis pulls v4l-utils, which is unsupported on aarch64-darwin
• demo-object-scene-registration requires the ZED SDK / pyzed

Intentional failure/demo cases

Example:

• demo-error-on-name-conflicts intentionally constructs conflicting stream names and is expected to fail startup

Why this matters

Import validation alone does not catch runtime wiring, deployment, configuration, or startup failures.

The local runtime QA pass found real issues that import validation did not detect, including:

• demo-gps-nav missing a required WebsocketVisSpec provider
• unitree-g1-groot-wbc Rerun config validation failure
• unitree-go2-multi / unitree-go2-multi-teleop constructing empty blueprints under the QA environment
• unitree-go2-vlm-stream-test passing an invalid g kwarg to VlmStreamTester

These are being tracked separately.

Proposed direction

Add a tracked runtime blueprint startup smoke test, but make it explicitly aware of blueprint requirements instead of assuming every registered blueprint can start in a generic CI environment.

Possible approaches:

1. Maintain explicit runtime categories such as:

   • generic CI
   • self-hosted
   • hardware-required
   • external-asset-required
   • platform-specific
   • intentional-failure

2. Reuse or extend blueprint requirement metadata so the startup test can skip unsupported environments automatically.

3. Keep runtime startup validation separate from the existing import/validity test in test_all_blueprints.py.

The goal is to make it possible to catch real runtime wiring/startup regressions without producing large numbers of expected failures from blueprints that require unavailable hardware or assets.

Notes

This issue was originally worded as though dimos/robot/test_blueprint_startup.py already existed on main. That was incorrect — it was a local QA test used during the macOS release pass. This description has been updated to clarify that distinction.

Start by reading dimos/robot/test_all_blueprints.py and its SELF_HOSTED_BLUEPRINTS classification, then compare it with the local QA file dimos/robot/test_blueprint_startup.py described in the issue. Define how runtime categories or requirement metadata distinguish generic CI from unsupported and intentional-failure cases. Done means tracked startup coverage runs without treating unavailable hardware, assets, platform dependencies, or known intentional failures as regressions.