alandtse / alandtse/devbench

devbench: surface + extend CS config-menu control for QA (discoverability, describe, section nav)

Đang mở
#37 2 bình luận 1 reaction 0 người được giao Xem trên GitHub
Ngôn ngữ chính
C++
Star
10
Fork
4
Merge trung bình
1 ngày 10 giờ
Pull request đã merge (30 ngày)
17

Mô tả

## Background

devbench *can* drive the Community Shaders ImGui config menu — `menu {action:invoke, name:"CommunityShaders"}` toggles it (CS registers itself; it shows under `menu {action:list}` → `.registered`). This is the right tool for QA-ing the CS menu (e.g. verifying i18n string coverage, exercising every `DrawSettings` section).

## Cost: wasted MCP round-trips before real work

Opening the CS menu took **four** MCP calls to get right because the verb/param shape isn't guessable, and each wrong call is a full game round-trip:

1. `menu {action:open, name:"CommunityShaders"}` -> `400` (open is engine-menus-only)
2. `menu {action:invoke, menu:"CommunityShaders"}` -> `400` (wrong param name)
3. `menu {action:invoke, name:"CommunityShaders"}` -> works

For an agent driving the bench, every failed call is wasted latency and tokens before any real testing happens. The fix is to make the *first* guess work: a discoverable `openshaders.menu` tool and/or a one-line doc + a clearer error that names the exact working call. Lowering this first-call cost is the main ask.

## Problem — it's under-discoverable / under-powered

While QA-testing i18n menu-wrap, the menu-open path was not obvious and cost real time (fell back to blocked keyboard injection first):

1. **No `openshaders.*` entry.** Consumers look under the `openshaders.*` namespace for CS-specific controls (`openshaders.capture/feature/settings/shadercache` exist). There is **no `openshaders.menu`**, so the natural guess 404s. The CS menu is instead toggled through the *generic* built-in `menu` tool via `invoke` — non-obvious.
2. **`open` vs `invoke` is subtle.** The `menu` tool's description says *"open/close: menu to show/hide (e.g. TweenMenu)"* (engine RE::UI menus) and *"invoke: a registered menu"*. `open name:CommunityShaders` returns 400 (it's not an engine menu); only `invoke` works. The param is `name`, not `menu` — discovered only via the 400 error text.
3. **`describe` is empty for registered menus.** `menu {action:describe, name:"CommunityShaders"}` returns `{}`. There's no way to read the menu's section/tab tree or the visible label/value text, so QA must screenshot + eyeball instead of asserting on text.
4. **`invoke` only toggles visibility.** No way to navigate to a specific tab/section (Advanced, Upscaling, Light Limit Fix, …), so a script can't exercise each `DrawSettings` section programmatically — exactly what i18n / UI-regression QA needs.

## Suggestions

- **Docs:** add to `devbench-usage.md` (and the bridge README): *"Toggle the CS config menu: `menu {action:invoke, name:\"CommunityShaders\"}`."}* One line removes most of the friction.
- **Alias:** register an `openshaders.menu` tool (`open` / `close` / `toggle`, optional `section`) so it's discoverable in the namespace people actually search. It can delegate to the same registration.
- **`describe`:** make `menu describe name:CommunityShaders` return the menu descriptor — the section/tab list and, ideally, each rendered label + current value. This enables text-level assertions (e.g. "every visible string is non-empty / present in en.json") without screenshots.
- **Navigation:** let `invoke`/`openshaders.menu` take a `section`/`tab` arg to open the menu *and* select a section, so QA can sweep all `DrawSettings` sections (the i18n coverage use case).

## Why it matters

The fork's QA gate is SE+VR runtime smoke. For UI/i18n PRs, the only way to verify menu rendering today is screenshot + manual read; the menu can't be driven section-by-section, and its text can't be inspected. The above turns "open it and squint" into scriptable, assertable menu QA.

## Repro / evidence

- `menu {action:list}` → `registered: ["CommunityShaders", "devbench.selftest"]`
- `menu {action:open, name:"CommunityShaders"}` → `400 Bad Request`
- `menu {action:invoke, menu:"CommunityShaders"}` → `400 "action 'invoke' requires a 'name'"`
- `menu {action:invoke, name:"CommunityShaders"}` → `{"op":"toggle","queued":true}` ✅ (menu opens)
- `menu {action:describe, name:"CommunityShaders"}` → `{}` (no descriptor)

---

*Originally filed at alandtse/open-shaders#134 — moved here as this is a devbench API issue. Requested by @alandtse.*

Hướng dẫn đóng góp

Chưa lập chỉ mục được hướng dẫn đóng góp cho kho mã nguồn này

Đánh giá

Issue này chưa được đánh giá.

Nhận issue mới trong hộp thư của bạn

Bản tóm tắt ngắn những issue GitHub phù hợp với người mới.