aai-institute / aai-institute/pyDVL
Implement NTK scorer
- Dominant language
- Python
- Stars
- 146
- Forks
- 10
- PR merge metrics
- No merged PRs in 30d
Description
Introduced in _Zhaoxuan Wu, Yao Shu, and Bryan Kian Hsiang Low, “[DAVINZ: Data Valuation Using Deep Neural Networks at Initialization](https://proceedings.mlr.press/v162/wu22j.html),” in Proceedings of the 39th International Conference on Machine Learning (International Conference on Machine Learning, PMLR, 2022), 24150–76._
It's a scorer using the Neural Tangent Kernel on untrained networks with a correction to estimate generalisation error. Implementation available in the supp. materials of the paper.
(Maybe relevant, JAX library for NTK: https://github.com/google/neural-tangents, and [paper](https://proceedings.mlr.press/v162/novak22a/novak22a.pdf) about fast finite-width NTK computation)
Contributor guide
Research direction
The issue references a specific paper and supplementary materials for the NTK scorer. First, examine the pyDVL codebase structure for existing scorer implementations. Review the linked paper and its supplementary materials for the algorithm details. Check the neural-tangents JAX library for potential integration. Determine where to add the new scorer class, likely in a module like pydvl.value/scoring. Implementation will involve adapting the NTK-based generalization error estimation from the paper into the pyDVL framework.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- jupyter-notebook, machine-learning, python
- Domain
- ai-infra-agents, devtools, machine-learning
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100