aai-institute / aai-institute/pyDVL

Implement NTK scorer

Open
#274 0 comments 0 reactions 0 assignees View on GitHub
new-method
Dominant language
Python
Stars
146
Forks
10
PR merge metrics
No merged PRs in 30d

Description

Introduced in _Zhaoxuan Wu, Yao Shu, and Bryan Kian Hsiang Low, “[DAVINZ: Data Valuation Using Deep Neural Networks at Initialization](https://proceedings.mlr.press/v162/wu22j.html),” in Proceedings of the 39th International Conference on Machine Learning (International Conference on Machine Learning, PMLR, 2022), 24150–76._

It's a scorer using the Neural Tangent Kernel on untrained networks with a correction to estimate generalisation error. Implementation available in the supp. materials of the paper.

(Maybe relevant, JAX library for NTK: https://github.com/google/neural-tangents, and [paper](https://proceedings.mlr.press/v162/novak22a/novak22a.pdf) about fast finite-width NTK computation)

Contributor guide

Open the contributing guide

Research direction

The issue references a specific paper and supplementary materials for the NTK scorer. First, examine the pyDVL codebase structure for existing scorer implementations. Review the linked paper and its supplementary materials for the algorithm details. Check the neural-tangents JAX library for potential integration. Determine where to add the new scorer class, likely in a module like pydvl.value/scoring. Implementation will involve adapting the NTK-based generalization error estimation from the paper into the pyDVL framework.

Written by the indexing model from the issue text.

Assessment

Tech stack
jupyter-notebook, machine-learning, python
Domain
ai-infra-agents, devtools, machine-learning
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.