Port remaining capsem-bench legacy modes to Rust and retire Python benchmark wrapper
- Dominant language
- Rust
- Stars
- 72
- Forks
- 13
- Avg merge
- 1d 2h
- Merged PRs (30d)
- 5
Description
## Problem
`capsem-bench protocol` must be Rust-backed so HTTP/model/credential/MCP/DNS hot-path benchmarks use the same implementation in host and VM lanes. The current release fix makes the Rust protocol binary mandatory in the guest as `capsem-bench-rs` and makes the existing `capsem-bench` wrapper delegate `protocol` to it.
The remaining `capsem-bench` modes are still implemented by the legacy Python package:
- `disk`
- `rootfs`
- `storage`
- `startup`
- `snapshot`
- `all`
- legacy load modes
This keeps release compatibility, but it is not the desired end state.
## Required end state
- One Rust `capsem-bench` implementation owns all benchmark modes.
- No Python benchmark wrapper is needed in the VM image.
- Host and VM lanes use the same Rust code for all hot protocol and release benchmarks.
- `capsem-bench all` remains available and writes `/tmp/capsem-benchmark.json` with the existing release artifact contract or an explicitly migrated schema.
- Existing benchmark gates and docs are updated to the Rust implementation.
## Gates
- Rust unit tests cover every mode.
- In-VM `capsem-bench all` passes and archives the expected JSON artifact.
- Protocol delta can run HTTP, model, credential, MCP/tools, and DNS in both host and guest lanes.
- No `guest/artifacts/capsem_bench` Python package remains after the port.
- Docker/rootfs/initrd packaging tests prove the Rust binary is the only benchmark executable shipped.
Contributor guide
Research direction
Start by tracing the Rust `capsem-bench` implementation and the existing `capsem-bench` wrapper across the listed modes: `disk`, `rootfs`, `storage`, `startup`, `snapshot`, `all`, and legacy load modes. Review the Rust unit tests, in-VM `capsem-bench all` gate, and Docker/rootfs/initrd packaging tests. Done means all modes use Rust, the expected `/tmp/capsem-benchmark.json` artifact is archived, protocol lanes pass, and `guest/artifacts/capsem_bench` is gone.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- docker, python, rust
- Domain
- build-system, devtools, release, testing
- Issue type
- Refactor
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 32/100