aRustyDev / aRustyDev/ccgram

Create train/val/test splits

Open
#7 0 comments 0 reactions 0 assignees View on GitHub
data
Dominant language
Python
Stars
0
Forks
0
PR merge metrics
No merged PRs in 30d

Description

## Summary
Split the normalized dataset into train, validation, and test sets with proper stratification.

## Success Criteria
- [ ] Split ratios defined (e.g., 80/10/10)
- [ ] Stratification by commit type
- [ ] No data leakage (same repo shouldn't span splits)
- [ ] Splits saved to data/splits/
- [ ] Split statistics documented
- [ ] Reproducible with fixed random seed

## Stratification Strategy
- Balance commit types across splits
- Consider stratifying by scope frequency
- Ensure test set has good coverage of rare types

## Output Files
\`\`\`
data/splits/
train.jsonl
val.jsonl
test.jsonl
splits_metadata.json
\`\`\`

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.