ruvnet / ruvnet/ruflo

[EPIC] Claude Agent SDK Integration v2.5.0-alpha.130 - Migrate to SDK Foundation

Open
#780 12 comments 0 reactions 0 assignees View on GitHub
Dominant language
TypeScript
Stars
72.7k
Forks
8.6k
Avg merge
3d 3h
Merged PRs (30d)
85

Description

# ๐ŸŽฏ Epic: Claude Agent SDK Integration for Claude-Flow v2.5.0-alpha.130

## Executive Summary
Integrate Claude Agent SDK (@anthropic-ai/claude-code) as the foundation layer for Claude-Flow, eliminating redundant custom implementations and positioning Claude-Flow as the premier multi-agent orchestration layer.

**Value Proposition:** "Claude Agent SDK handles single agents brilliantly. Claude-Flow makes them work as a swarm."

## ๐ŸŽฏ Success Metrics
- โœ… 50% reduction in custom retry/checkpoint code (15k โ†’ 7.5k lines)
- โœ… Zero regression in existing functionality
- โœ… 30% performance improvement in core operations
- โœ… 100% backward compatibility with migration path
- โœ… 95%+ test coverage for migrated components

## ๐Ÿ“‹ Implementation Phases

### Phase 1: Foundation Setup (Week 1)
**Install and Configure SDK**
```bash
npm install @anthropic-ai/claude-code@latest
```

**Tasks:**
- Install Claude Agent SDK package
- Create SDK configuration adapter
- Build compatibility layer for backward compatibility
- Set up SDK wrapper classes

**Files to create:**
- `src/sdk/sdk-config.ts`
- `src/sdk/compatibility-layer.ts`
- `src/sdk/__tests__/sdk-config.test.ts`

### Phase 2: Retry Mechanism Migration (Week 1-2)
**Refactor retry logic to use SDK primitives**

**Current Implementation (REMOVE):**
```typescript
// src/api/claude-client.ts - 200+ lines of custom retry
private calculateBackoff(attempt: number): number {
const baseDelay = this.config.retryDelay || 1000;
const jitter = Math.random() * 1000;
return Math.min(baseDelay * Math.pow(2, attempt - 1) + jitter, 30000);
}
```

**New Implementation (ADD):**
```typescript
// src/api/claude-client-v3.ts - SDK handles retry
constructor(config: ClaudeAPIConfig) {
this.sdk = new ClaudeCodeSDK({
retryPolicy: {
maxAttempts: config.retryAttempts || 3,
backoffMultiplier: 2,
initialDelay: config.retryDelay || 1000
}
});
}

async makeRequest(request: ClaudeRequest): Promise {
// SDK automatically handles retry with exponential backoff
return this.sdk.messages.create(request);
}
```

**Files to modify:**
- `src/api/claude-client.ts` โ†’ `src/api/claude-client-v3.ts`
- `src/swarm/executor.ts` โ†’ `src/swarm/executor-sdk.ts`
- `src/swarm/strategies/*.ts`

### Phase 3: Artifact Management Migration (Week 2)
**Migrate memory system to SDK artifacts**

**Tasks:**
- Replace custom memory manager with SDK artifacts
- Implement batch operations using SDK
- Update swarm memory coordination
- Ensure data compatibility

**New Memory Manager:**
```typescript
// src/swarm/memory-manager-sdk.ts
export class MemoryManagerSDK {
async store(key: string, value: any): Promise {
await this.sdk.artifacts.store({
key: `swarm:${key}`,
value,
metadata: { timestamp: Date.now(), version: '3.0.0' }
});
}

async batchStore(items: Array<{key: string, value: any}>): Promise {
await this.sdk.artifacts.batchStore(items);
}
}
```

### Phase 4: Checkpoint System Integration (Week 2-3)
**Integrate SDK checkpoints with swarm extensions**

**Tasks:**
- Use SDK checkpoints as base
- Add swarm-specific metadata layer
- Enable auto-checkpointing for long-running swarms
- Migrate existing checkpoint data

**New Checkpoint System:**
```typescript
// src/verification/checkpoint-manager-sdk.ts
export class CheckpointManagerSDK {
async createCheckpoint(description: string, swarmData?: SwarmMetadata): Promise {
const sdkCheckpoint = await this.sdk.checkpoints.create({
description,
metadata: { ...swarmData, createdBy: 'claude-flow' }
});

// Add swarm-specific extensions
this.swarmMetadata.set(sdkCheckpoint.id, swarmData);
return sdkCheckpoint.id;
}

async enableAutoCheckpoint(swarmId: string, interval: number = 60000): Promise {
this.sdk.checkpoints.enableAuto({ interval, filter: ctx => ctx.swarmId === swarmId });
}
}
```

### Phase 5: Tool Governance Migration (Week 3)
**Migrate hook system to SDK permissions**

**Tasks:**
- Configure SDK tool permissions
- Migrate custom hooks to SDK events
- Implement swarm-specific hooks on top
- Update security policies

**SDK Permission Configuration:**
```typescript
// src/services/hook-manager-sdk.ts
this.sdk.permissions.configure({
fileSystem: {
read: { allowed: true, paths: ['./src', './tests'] },
write: { allowed: true, paths: ['./dist'], beforeWrite: this.validateWrite }
},
network: {
allowed: true,
domains: ['api.anthropic.com', 'github.com'],
beforeRequest: this.rateLimit
}
});
```

### Phase 6: Regression Testing (Week 3-4)
**Comprehensive test suite to prevent regressions**

**Test Coverage Requirements:**
- Unit tests: 98%+
- Integration tests: 95%+
- E2E tests: 90%+
- Performance benchmarks

**Key Test Files:**
- `src/__tests__/regression/sdk-migration.test.ts`
- `src/__tests__/performance/sdk-benchmarks.test.ts`
- `src/__tests__/compatibility/backward-compat.test.ts`

### Phase 7: Migration & Documentation (Week 4)
**Automated migration and comprehensive docs**

**Deliverables:**
- Migration script: `scripts/migrate-to-v3.js`
- Breaking changes doc: `BREAKING_CHANGES.md`
- Migration guide: `MIGRATION_GUIDE.md`
- API documentation updates

## ๐Ÿšจ Breaking Changes

### API Changes
**Before (v2.x):**
```typescript
client.executeWithRetry(request)
memory.persistToDisk()
checkpoints.executeValidations()
```

**After (v3.x):**
```typescript
client.makeRequest(request) // Retry is automatic
memory.store(key, value) // Persistence is automatic
checkpoints.create() // Validation is automatic
```

### Configuration Changes
**Before:**
```js
{ retryAttempts: 3, retryDelay: 1000 }
```

**After:**
```js
{ retryPolicy: { maxAttempts: 3, initialDelay: 1000 } }
```

## ๐Ÿ“Š Performance Improvements

### Expected Benchmarks
- **Retry Operations:** 30% faster (1250ms โ†’ 875ms avg)
- **Memory Operations:** 73% faster (45ms โ†’ 12ms per op)
- **Batch Operations:** 4x faster with SDK batching
- **Checkpoint Creation:** 50% faster with SDK

## ๐Ÿ”„ Migration Strategy

### Step 1: Install Dependencies
```bash
npm install @anthropic-ai/claude-code@latest
npm update claude-flow@3.0.0-alpha.130
```

### Step 2: Run Migration Script
```bash
npm run migrate:v3
```

### Step 3: Test Migration
```bash
npm run test:migration
npm run test:regression
npm run benchmark:performance
```

### Step 4: Rollback Plan
```bash
# If issues arise
npm install claude-flow@2.0.0-alpha.129
npm run rollback:v2
```

## ๐Ÿ“ Key Files

### New Files
- `src/sdk/sdk-config.ts` - SDK configuration adapter
- `src/sdk/compatibility-layer.ts` - Backward compatibility
- `src/api/claude-client-v3.ts` - SDK-based client
- `src/swarm/executor-sdk.ts` - SDK-based executor
- `src/swarm/memory-manager-sdk.ts` - SDK memory manager
- `src/verification/checkpoint-manager-sdk.ts` - SDK checkpoints

### Modified Files
- `src/api/claude-client.ts` - Mark deprecated
- `src/swarm/executor.ts` - Extend with SDK
- `src/verification/checkpoint-manager.ts` - Wrap SDK

### Migration Scripts
- `scripts/migrate-to-v3.js` - Automated migration
- `scripts/rollback-v2.js` - Rollback script

## ๐Ÿ† Definition of Done

- [ ] All SDK dependencies installed
- [ ] Compatibility layer implemented
- [ ] Retry logic migrated to SDK
- [ ] Memory system using SDK artifacts
- [ ] Checkpoints using SDK with swarm extensions
- [ ] Hook system migrated to SDK permissions
- [ ] Zero regression in test suite
- [ ] 30% performance improvement verified
- [ ] Migration script tested and working
- [ ] Documentation updated
- [ ] Breaking changes documented
- [ ] Rollback plan tested

## ๐Ÿ“ˆ Risk Mitigation

### Identified Risks
1. **Breaking changes impact users** โ†’ Compatibility layer + migration script
2. **Performance regression** โ†’ Comprehensive benchmarks before/after
3. **Data compatibility issues** โ†’ Migration tests + rollback plan
4. **SDK limitations** โ†’ Maintain swarm extensions layer

## ๐Ÿ”— Related Links

- SDK Documentation: https://docs.claude.com/en/docs/claude-code/sdk
- NPM Package: https://www.npmjs.com/package/@anthropic-ai/claude-code
- Migration Guide: /docs/epic-sdk-integration.md
- Claude-Flow Docs: https://github.com/ruvnet/claude-flow

## ๐Ÿ“ Notes

This epic represents a major architectural shift that:
1. Validates Claude-Flow's pioneering concepts now in SDK
2. Reduces maintenance burden by 50%
3. Improves performance by 30%
4. Positions Claude-Flow as the swarm orchestration leader
5. Maintains 100% backward compatibility

**Remember:** "Claude Agent SDK handles single agents. Claude-Flow orchestrates swarms."

---
*Full implementation details with 500+ lines of code examples available in `/docs/epic-sdk-integration.md`*

@ruvnet - Ready for implementation in alpha-130 branch

Contributor guide

Open the contributing guide

Research direction

Start by reading /docs/epic-sdk-integration.md and the listed entry points such as src/api/claude-client.ts, src/swarm/executor.ts, and src/verification/checkpoint-manager.ts. The work spans the seven migration phases; done requires the SDK integration, compatibility and migration paths, regression and performance tests, and updated documentation to satisfy the checklist.

Written by the indexing model from the issue text.

Assessment

Tech stack
node.js, typescript
Domain
ai, backend
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.