finos / finos/morphir-dotnet

Test skill emulation in Windsurf

Open
#268 0 comments 0 reactions 0 assignees View on GitHub
documentation enhancement
Dominant language
F#
Stars
14
Forks
12
PR merge metrics
No merged PRs in 30d

Description

# Test: Skill Emulation in Windsurf

## Objective

Verify that morphir-dotnet skills (gurus) are accessible and functional in Windsurf through documentation-based emulation, leveraging Windsurf's Cascade AI and context awareness features.

## Background

Windsurf is Codeium's agentic IDE with Cascade AI, featuring:
1. **Deep codebase awareness** with automatic context gathering
2. **Multi-file editing** capabilities
3. **Command palette** integration
4. **Supercomplete** for intelligent code completion
5. **Chat and inline edit modes**

## Test Scope

**Skills to Test**:
- QA Tester (\`.claude/skills/qa-tester/\`)
- AOT Guru (\`.claude/skills/aot-guru/\`)
- Release Manager (\`.claude/skills/release-manager/\`)

**Test Environment**: Windsurf IDE with morphir-dotnet repository open

## Windsurf-Specific Features to Test

### 1. Cascade AI Context Awareness

Windsurf's Cascade AI automatically gathers relevant context. Test if it discovers skills without explicit prompting.

#### Test 1.1: Automatic Skill Discovery
**User Action**: Ask "How should I test this codebase?"

**BDD Scenario**:
\`\`\`gherkin
Feature: Cascade AI Skill Discovery

Scenario: User asks testing question without mentioning skills
Given Windsurf Cascade AI is active
And .agents/skills-reference.md exists in repository
When I ask "How should I test this codebase?"
Then Cascade should discover .claude/skills/ directory
And Cascade should identify QA Tester skill
And Cascade should reference QA Tester SKILL.md
And Cascade should apply testing best practices from skill
\`\`\`

**Acceptance Criteria**:
- [ ] Cascade discovers skills directory
- [ ] QA Tester skill identified as relevant
- [ ] Skill guidance applied

#### Test 1.2: Codebase Understanding
**User Action**: Ask "What expert guidance is available in this project?"

**BDD Scenario**:
\`\`\`gherkin
Feature: Project Structure Awareness

Scenario: User inquires about available expertise
Given Cascade AI indexes the codebase
When I ask "What expert guidance is available?"
Then Cascade should discover:
| Resource |
| AGENTS.md |
| .agents/skills-reference.md |
| .claude/skills/ directory |
And Cascade should list all available skills
And Cascade should explain how to use each skill
\`\`\`

**Acceptance Criteria**:
- [ ] All skill documentation discovered
- [ ] Skills listed with descriptions
- [ ] Usage guidance provided

### 2. Chat Mode Skill Invocation

#### Test 2.1: Natural Language Skill Request
**User Action**: In chat, ask "Use the QA Tester skill to create a test plan for PR #123"

**BDD Scenario**:
\`\`\`gherkin
Feature: Chat Mode Skill Invocation

Scenario: User requests skill assistance in chat
Given Windsurf chat mode is active
And PR #123 exists
When I ask "Use the QA Tester skill to create a test plan for PR #123"
Then Cascade should locate .claude/skills/qa-tester/SKILL.md
And Cascade should read Test Plan Development section
And Cascade should create comprehensive test plan
And Test plan should include:
| Component |
| Happy path scenarios |
| Edge cases |
| Error conditions |
| Priority labels |
\`\`\`

**Acceptance Criteria**:
- [ ] Skill file located and read
- [ ] Guidance correctly applied
- [ ] Output matches skill templates

#### Test 2.2: Multi-Turn Skill Conversation
**User Action**: Follow-up questions about skill guidance

**BDD Scenario**:
\`\`\`gherkin
Feature: Multi-Turn Skill Interaction

Scenario: User has follow-up questions about skill
Given I previously asked about QA Tester
When I ask "What playbooks does this skill have?"
Then Cascade should remember QA Tester context
And Cascade should list playbooks:
| Playbook |
| PR Verification |
| Regression Testing |
| Feature Testing |
| Build System Testing |
| Package Testing |
And Cascade should offer to walk through any playbook
\`\`\`

**Acceptance Criteria**:
- [ ] Context maintained across turns
- [ ] Playbooks correctly identified
- [ ] Interactive guidance offered

### 3. Inline Edit with Skill Context

#### Test 3.1: Skill-Guided Code Generation
**User Action**: Select code, use inline edit: "Apply AOT Guru best practices"

**BDD Scenario**:
\`\`\`gherkin
Feature: Inline Edit with Skill Guidance

Scenario: User requests skill-guided code improvements
Given I select code in Program.cs
And I invoke inline edit
When I enter "Apply AOT Guru best practices"
Then Cascade should read .claude/skills/aot-guru/SKILL.md
And Cascade should apply AOT-ready patterns
And Code should be refactored to:
| Improvement |
| Remove reflection usage |
| Use source generators |
| Add trim-friendly attributes |
| Optimize for binary size |
\`\`\`

**Acceptance Criteria**:
- [ ] Skill guidance accessed during edit
- [ ] Code follows skill best practices
- [ ] Changes are AOT-compatible

### 4. Command Palette Integration

#### Test 4.1: Custom Command for Skills
**Goal**: Determine if Windsurf can create custom commands for skills

**User Action**: Try creating custom command or macro for "List Skills"

**BDD Scenario**:
\`\`\`gherkin
Feature: Command Palette Skill Access

Scenario: User creates custom skill command
Given Windsurf command palette is accessible
When I attempt to create custom command "List Skills"
Then Command should execute chat query to Cascade
And Cascade should read .agents/skills-reference.md
And Cascade should display available skills
\`\`\`

**Acceptance Criteria** (if supported):
- [ ] Custom command created
- [ ] Command triggers skill listing
- [ ] Skills displayed correctly

**Note**: Test if Windsurf supports custom commands/macros. If not, document limitation.

### 5. Supercomplete Skill Awareness

#### Test 5.1: Skill-Aware Code Completion
**User Action**: Start typing test code, observe if Supercomplete suggests skill patterns

**BDD Scenario**:
\`\`\`gherkin
Feature: Skill-Aware Code Completion

Scenario: User writes test code
Given I'm in a test file
When I type "// Create test for"
Then Supercomplete may suggest completions referencing QA Tester patterns
And Suggestions should follow AAA pattern (Arrange-Act-Assert)
\`\`\`

**Acceptance Criteria** (exploratory):
- [ ] Test if Supercomplete references skill patterns
- [ ] Document whether skill context improves completions

**Note**: This is exploratory - document actual behavior.

### 6. Multi-File Editing with Skills

#### Test 6.1: Coordinated Multi-Skill Workflow
**User Action**: Ask Cascade to "Prepare release 1.0.0 with full QA verification"

**BDD Scenario**:
\`\`\`gherkin
Feature: Multi-File Skill Coordination

Scenario: Complex workflow requiring multiple skills
Given Cascade has multi-file editing capability
When I ask "Prepare release 1.0.0 with full QA verification"
Then Cascade should identify required skills:
| Skill |
| Release Manager |
| QA Tester |
And Cascade should read both SKILL.md files
And Cascade should coordinate workflow:
| Step | Skill | Action |
| 1 | Release Manager | Update CHANGELOG.md |
| 2 | Release Manager | Validate version |
| 3 | QA Tester | Run smoke tests |
| 4 | QA Tester | Run regression tests |
| 5 | Release Manager | Trigger deployment |
\`\`\`

**Acceptance Criteria**:
- [ ] Both skills identified
- [ ] Workflow coordinated correctly
- [ ] Multi-file edits suggested
- [ ] Steps follow skill playbooks

## Implementation Requirements

### 1. Optimize Documentation for Cascade Discovery

Ensure files are easily discoverable:
- **README.md**: Mention skills prominently
- **AGENTS.md**: Clear section on "Available Skills"
- **.agents/skills-reference.md**: Comprehensive skill index
- **Each SKILL.md**: Clear skill name and purpose at top

### 2. Update AGENTS.md

Add section: **"Using Skills with Windsurf"**

\`\`\`markdown
## Using Skills with Windsurf

Windsurf's Cascade AI automatically discovers and applies skill guidance.

### Automatic Discovery
Cascade indexes the codebase and discovers:
- \`AGENTS.md\` - Project guidance
- \`.agents/skills-reference.md\` - Skills index
- \`.claude/skills/*/SKILL.md\` - Individual skills

### Using Skills

**Method 1: Natural Language (Recommended)**
Ask: "Use [skill-name] to [task]"
Example: "Use QA Tester to create a test plan for PR #123"

**Method 2: Context-Aware Questions**
Ask domain-specific questions. Cascade will discover relevant skills.
Example: "How do I optimize binary size?" → Discovers AOT Guru

**Method 3: Inline Edit**
Select code, use inline edit: "Apply [skill-name] best practices"

### Multi-Skill Workflows
Cascade can coordinate multiple skills for complex tasks:
"Prepare release with QA verification" → Engages Release Manager + QA Tester

### Skill Locations
- **Skills Index**: \`.agents/skills-reference.md\`
- **QA Tester**: \`.claude/skills/qa-tester/SKILL.md\`
- **AOT Guru**: \`.claude/skills/aot-guru/SKILL.md\`
- **Release Manager**: \`.claude/skills/release-manager/SKILL.md\`
\`\`\`

### 3. Update skills-reference.md

Add Windsurf-specific section:

\`\`\`markdown
### For Windsurf Users

**Cascade AI automatically discovers skills** when relevant to your task.

**Natural Language Invocation**:
- "Use QA Tester to validate this PR"
- "Apply AOT Guru optimization to this code"
- "Follow Release Manager playbook for version 1.0.0"

**Context-Aware Discovery**:
Ask domain questions - Cascade finds relevant skills:
- "How should I test this?" → QA Tester
- "Optimize binary size" → AOT Guru
- "Prepare release" → Release Manager

**Inline Editing**:
Select code → Inline edit → "Apply [skill] best practices"

**Multi-Skill Coordination**:
Complex tasks engage multiple skills automatically.
\`\`\`

## Acceptance Criteria

**Functional Requirements**:
- [ ] Cascade discovers skills automatically
- [ ] Natural language skill invocation works
- [ ] Chat mode applies skill guidance
- [ ] Inline edit respects skill best practices
- [ ] Multi-file editing coordinates skills
- [ ] Script locations identified correctly

**Documentation Requirements**:
- [ ] AGENTS.md includes Windsurf section
- [ ] skills-reference.md includes Windsurf usage
- [ ] Documentation optimized for Cascade discovery
- [ ] README mentions skill support

**Quality Requirements**:
- [ ] All BDD scenarios documented
- [ ] Test results captured with screenshots/videos
- [ ] Cascade discovery effectiveness measured
- [ ] Issues/limitations identified

## Deliverables

- [ ] Test execution report (pass/fail for each scenario)
- [ ] Windsurf interaction recordings/screenshots
- [ ] Cascade discovery effectiveness analysis
- [ ] Documentation updates (AGENTS.md, skills-reference.md)
- [ ] List of identified issues and workarounds
- [ ] Comparison with Cursor and Copilot

## Success Metrics

- **Automatic Discovery**: 90%+ skill discovery without explicit mention
- **Natural Language**: 95%+ correct skill invocation
- **Chat Mode**: 90%+ accurate skill guidance application
- **Inline Edit**: 85%+ skill-compliant code changes
- **Multi-Skill**: 80%+ successful coordination

## Related

- #266 - GitHub Copilot skill testing
- #267 - Cursor skill testing
- Skills: qa-tester, aot-guru, release-manager

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.