Improve alpha testing coverage for MCP output, host compatibility, and npm smoke tests
- Dominant language
- TypeScript
- Stars
- 3
- Forks
- 1
- PR merge metrics
- No merged PRs in 30d
Description
## Context
The current alpha test suite is credible: it covers compiler discovery, MCP runtime methods, auth, pagination, widget bundling, platform-specific files, diagnostics, package metadata, and npm install/create smoke tests. Before treating Sidecar as stable, we should raise the bar around generated artifacts and real host compatibility.
## Current coverage
- Unit/runtime tests for tools, resources, prompts, auth, proxy, pagination, schema enforcement, and the MCP Apps bridge.
- Compiler tests for reserved files, folder-derived ids/URIs, widget resources, generated clients, platform-specific file selection, and diagnostics.
- E2E build tests for plain MCP, ChatGPT target, Claude plugin output, widget URI cache busting, path containment, and plugin filename/frontmatter sanitization.
- Packaging checks for exports/bins and smoke tests for `npm install sidecar-ai` and `npm create sidecar-app@latest`.
## Work to add
- Add golden artifact snapshots for every generated manifest/plugin file.
- Validate emitted MCP/plugin artifacts against official JSON Schemas where available.
- Add live host acceptance tests for ChatGPT and Claude where automation is possible.
- Add browser visual regression tests for native components across light/dark mode and host recipes.
- Add CI smoke tests that install from npm, scaffold an app, run `sidecar check`, and run `sidecar build`.
- Add more adversarial/fuzz tests for JSON-RPC request handling, auth, proxy middleware, and path handling.
## Acceptance criteria
- CI has a job that validates generated artifacts structurally, not only through hand-written expectations.
- CI has a published-package smoke test path that exercises `sidecar-ai` and `create-sidecar-app` from npm.
- We have a documented distinction between fast unit tests, build artifact tests, visual tests, and live host acceptance tests.
- Failures identify whether the breakage is compiler output, MCP protocol behavior, host-specific behavior, packaging, or visual/native UI parity.
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.