andreagrandi / andreagrandi/draftomen
Build a leakage-safe pick-ranking dataset pipeline
- Vorherrschende Sprache
- Python
- Sterne
- 0
- Forks
- 0
- Ø Merge
- 1 Std. 21 Min.
- Gemergte PRs (30 T.)
- 188
Beschreibung
## Problem
The research completed in #37 defines viable 17Lands sources and feature constraints, but Draftgoblin has no reproducible workflow for converting public draft dumps into ML-ready ranking examples. Using the raw rows directly risks future-information leakage, inconsistent pool state, and retaining pseudonymous draft identifiers.
## Proposed change
Add an offline dataset-building workflow that reads a local 17Lands public draft dump and Draftgoblin's existing card metadata. Produce one candidate row per offered card, grouped by pick, with deterministic chronological splits and a source manifest.
The pipeline must enforce the feature, licensing, and privacy constraints documented in `docs/ml-pick-recommendations.md`. It is research tooling only and must not affect live recommendations or collect user draft data.
## Acceptance Criteria
- [ ] Read supported 17Lands draft CSV, gzip, or archive inputs for an explicit set and format.
- [ ] Produce one ranking group per pick and one candidate row per positive `pack_card_*` entry, with relevance `1` for `pick` and `0` for other offered cards.
- [ ] Use `pool_*` as the pre-pick pool and cross-check it against prior picks; skip and count inconsistent rows.
- [ ] Exclude and report multi-pick rows with a non-empty `pick_2`, unresolved cards, missing picks, and malformed pack data.
- [ ] Include only recommendation-time features from the pack, pre-pick pool, position, set/format, and existing card metadata.
- [ ] Exclude outcomes, later picks, final deck data, timestamps as unique values, rank/history buckets, and identifiers from model features.
- [ ] Remap `draft_id` and assign complete drafts chronologically to deterministic 70/15/15 train, validation, and test splits without cross-split leakage.
- [ ] Write a manifest containing source URL, retrieval date, stated license, schema version, preprocessing rules, split ranges, and skip counts.
- [ ] Do not commit raw public rows, remapped training matrices, or row-level predictions.
- [ ] Add deterministic fixture coverage and verify the complete dataset-building workflow against a local public dump.
Beitragsleitfaden
Für dieses Repository ist kein Beitragsleitfaden indexiert
Rechercherichtung
Start with docs/ml-pick-recommendations.md and the findings in #37, then inspect the existing card metadata and the local 17Lands dump formats. Build deterministic fixture coverage for grouping, leakage exclusions, skips, remapped chronological splits, and manifest output, then verify the complete workflow against a local public dump.
Vom Indexierungsmodell aus dem Issue-Text verfasst.
Bewertung
- Tech-Stack
- python
- Bereich
- data-engineering, machine-learning, tooling
- Issue-Typ
- Feature
- Schwierigkeit
- 4/5
- Geschätzter Aufwand
- 3-5 Tage
- Aktivitätsstatus
- Ruhig
- Klarheit
- Größtenteils klar
- Anfängerfreundlichkeit
- 48/100