spec-kitty / spec-kitty/spec-kitty

Add Bob Shell and IDE smoke-test checklist

Open
#895 1 comment 0 reactions 0 assignees View on GitHub
documentation enhancement priority:P1 workflow
Dominant language
Python
Stars
1.6k
Forks
165
Avg merge
14h 22m
Merged PRs (30d)
343

Description

## Context

Bob is proprietary/auth-gated and does not have public product source, so runtime validation must be explicit and reproducible.

Parent epic: #890

## Scope

Create the manual or semi-automated test checklist required before promoting Bob from research-only to partial support.

## Bob Shell Checks

- [ ] Install or run Bob Shell `1.0.1`.
- [ ] Accept or inspect license through `--show-license` / `--accept-license`.
- [ ] Authenticate with `BOBSHELL_API_KEY` or IBM login.
- [ ] Verify `.bob/commands` command discovery.
- [ ] Verify command names with dots vs hyphens.
- [ ] Verify argument passing for free-form Spec Kitty prompts.
- [ ] Verify `AGENTS.md` loading and `/memory show` / `/memory refresh` behavior.
- [ ] Verify sandbox behavior does not hide required `.kittify` files.

## Bob IDE Checks

- [ ] Verify `.bob/skills//SKILL.md` discovery.
- [ ] Verify Bob skills activate manually or by relevance.
- [ ] Verify `.bob/commands` slash commands appear in Bob IDE.
- [ ] Verify `.bob/mcp.json` schema recognition.
- [ ] Verify approval behavior for read/write/execute/MCP/skills.

## Acceptance Criteria

- [ ] Checklist is committed in a stable docs or test-plan location.
- [ ] Smoke results are captured before closing the parent epic.
- [ ] Any runtime mismatch creates a follow-up issue or blocks the relevant implementation issue.

Contributor guide

Open the contributing guide

Research direction

Start with parent epic #890, install or run Bob Shell 1.0.1, and work through the listed Shell and IDE checks, including license handling, authentication, discovery, prompts, memory, sandboxing, skills, MCP, and approvals. Choose a stable docs or test-plan location for the checklist. Done means the checklist is committed, smoke results are captured, and any mismatch has a follow-up issue or blocks the relevant implementation issue.

Written by the indexing model from the issue text.

Assessment

Domain
documentation, testing
Issue type
Documentation
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.