DataEvolve — LLM-Agent-Driven Autonomous Curation Strategy Evolution

What:
Add an orchestration layer that uses an LLM agent to autonomously discover, test, and refine data curation strategies per content category. The agent maintains an experience pool (discovered issues) and strategy pool (performance metrics), running 30 iterative generations per category to evolve domain-specific curation rules without human intervention.

Why:
DataEvolve (arXiv:2603.14420, Mar 2026) produces Darwin-CC (504B tokens from 672B Nemotron-CC), scoring 44.13 average across 18 benchmarks at 3B/500B-token scale — outperforming DCLM, Ultra-FineWeb, and FineWeb-Edu by +3.96 points. Manual curation strategy design is brittle and does not generalize across diverse content types. Evolved strategies consistently beat suboptimal strategies by ~3 points in ablations.

Definition of Done:

• DataEvolveOrchestrator under nemo_curator/evolution/
• DataObserver: samples documents per category, calls LLM to identify quality issues (configurable judge model)
• StrategyGenerator: given issues, LLM proposes curation strategies (noise removal, format normalization, preservation rules)
• StrategyEvaluator: applies strategy on sampled data, scores with proxy model perplexity or small-scale training
• ExperiencePool + StrategyPool: persistent JSON stores tracking discovered issues and strategy performance history
• Configurable: number of categories, iterations per category (default: 30), LLM endpoint
• Ships with default 8-category taxonomy (STEM, code, news, web, books, medical, legal, social)
• Ray-native execution; each evaluation run is a distributed Ray task
• Tutorial: evolving strategies on a 10B-token CommonCrawl subset
• Integration test: verify strategy pool grows over iterations and retained token quality improves

Begin with the proposed nemo_curator/evolution/ area and the Ray-native execution requirements. Use the requested integration test and the 10B-token CommonCrawl tutorial as acceptance targets, verifying that strategy history grows across iterations and retained token quality improves.