kamiazya / kamiazya/web-csv-toolbox

Design extensibility mechanism for ecosystem packages without adding core dependencies

Open
#528 0 comments 0 reactions 1 assignee Claimed by @kamiazya View on GitHub
enhancement help wanted
Dominant language
TypeScript
Stars
21
Forks
10
PR merge metrics
No merged PRs in 30d

Description

## Background

The `web-csv-toolbox` library has a strong commitment to **zero dependencies**, which provides:
- Small bundle size
- Fast installation
- No transitive dependency issues
- Cross-platform compatibility (Node.js, Deno, browsers)

However, users have requested advanced features that would naturally require external dependencies:
- Schema validation (Zod, TypeBox, JSON Schema)
- Data serialization (CSV writing/formatting)
- Type conversion and transformation
- Data quality reporting and statistics

## Problem Statement

**How can we enable a rich ecosystem of extensions while keeping the core library zero-dependency?**

### Current Constraints
1. ✅ Core `web-csv-toolbox` must remain zero-dependency
2. ✅ Core API should not be breaking for existing users
3. ✅ Extensions should be opt-in (don't affect users who don't need them)
4. ❓ Need mechanism for extensions to hook into parsing pipeline
5. ❓ Need clear boundaries between core and extensions

### Desired Outcomes
- Users can install only the features they need
- Community can build custom extensions
- Type safety across core + extensions
- Minimal performance overhead for unused features
- Clear documentation on extensibility

## Potential Approaches

### Option 1: Plugin System with Hooks
Add extension points to core library (e.g., `onRecord`, `onComplete` hooks) that allow external packages to inject behavior.

**Pros:**
- Flexible, composable
- No core dependencies
- Users control what they install

**Cons:**
- Adds complexity to core
- Plugin API needs careful design
- Potential performance overhead

### Option 2: Wrapper/Adapter Pattern
Extensions wrap or extend core parsing functions without modifying core.

**Pros:**
- Core remains simple
- Extensions are independent
- Easy to understand

**Cons:**
- Might require parsing twice
- Harder to compose multiple extensions
- Less integrated experience

### Option 3: Monorepo with Separate Packages
Move to monorepo structure with `@web-csv-toolbox/*` scoped packages.

**Pros:**
- Clear separation of concerns
- Independent versioning
- Shared tooling and CI

**Cons:**
- More complex project structure
- Need monorepo tooling (already using pnpm)
- Publishing complexity

### Option 4: Hybrid Approach
Combination of above approaches - minimal hooks in core, monorepo for extensions, wrapper patterns where appropriate.

## Key Design Questions

### 1. Extension Points
**Where should extensions be able to hook into the parsing process?**
- Before parsing starts?
- For each record during parsing?
- After parsing completes?
- On errors?
- For validation?
- For transformation?

### 2. Data Flow
**How should data flow between core and extensions?**
- Extensions modify records in-place?
- Extensions return new records?
- Extensions can skip/filter records?
- Async vs sync extension functions?

### 3. Type Safety
**How to maintain TypeScript type safety across packages?**
- How do extensions affect return types?
- How to infer types from extension configurations?
- Should core export utility types for extensions?

### 4. Performance
**How to minimize performance impact?**
- Zero overhead when no extensions used?
- Efficient pipeline for multiple extensions?
- Streaming-friendly?

### 5. Developer Experience
**How easy should it be to create extensions?**
- What's the minimal API surface?
- How much boilerplate is required?
- Are there helper utilities?

### 6. Package Organization
**How should packages be structured?**
- Monorepo vs separate repos?
- Naming conventions (`@web-csv-toolbox/*`)?
- Version synchronization strategy?
- Shared vs independent release cycles?

## Example Use Cases to Consider

### Use Case 1: Schema Validation
```typescript
// User wants to validate with Zod
// Should work WITHOUT installing zod in core
import { parse } from 'web-csv-toolbox';
import { withZodValidation } from '@web-csv-toolbox/zod';
import { z } from 'zod';

const schema = z.object({ name: z.string(), age: z.number() });
const records = await parse.toArray(csv, /* how to integrate schema? */);
```

### Use Case 2: CSV Serialization
```typescript
// User wants to write CSV
// Should work as standalone feature
import { stringify } from '@web-csv-toolbox/serialize';

const csv = stringify([
{ name: 'Alice', age: 42 }
]);
```

### Use Case 3: Multiple Transformations
```typescript
// User wants to chain multiple transformations
// Should be composable
import { parse } from 'web-csv-toolbox';
import { trim, convertNumbers, filterEmpty } from '@web-csv-toolbox/transform';

const records = await parse.toArray(csv, /* how to compose? */);
```

### Use Case 4: Custom Extension
```typescript
// Third-party developer wants to create extension
// Should have clear API to implement
export function createMyCustomExtension(options) {
// What interface should this implement?
// What utilities does core provide?
}
```

## Related Issues
- #517 - CSV stringify/serialize functionality
- #520 - Transform option to modify records during parsing
- #326 - Column count mismatch validation
- #116 - Add Format Support

## Success Criteria

A successful design should:
- [ ] Maintain zero dependencies in core package
- [ ] Enable rich ecosystem of extension packages
- [ ] Preserve backward compatibility
- [ ] Provide clear extension API
- [ ] Maintain type safety across packages
- [ ] Have minimal performance overhead
- [ ] Be well-documented with examples
- [ ] Be easy for community to extend

## Discussion Points

This issue is for **design discussion** before implementation. Please share thoughts on:

1. Which approach (or combination) seems best?
2. What extension points are most important?
3. What should the extension API look like?
4. How should packages be organized?
5. Are there examples from other ecosystems we should study?

## Next Steps

1. Gather feedback and requirements
2. Research similar patterns in other libraries
3. Create detailed RFC with specific API design
4. Prototype to validate design decisions
5. Implement based on approved design

---
**Strategic Decision**: This will shape the future of the library and its ecosystem.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.