huggingface / huggingface/paperswithcode-feedback
[Dataset Request] Add nvidia/Nemotron-PII — PII detection benchmark
- Dominant language
- No language data
- Stars
- 10
- Forks
- 1
- PR merge metrics
- No merged PRs in 30d
Description
Hi! I'd like to request indexing the **Nemotron-PII** dataset and a leaderboard for **PII Detection**.
### Dataset
- **HF:** [`nvidia/Nemotron-PII`](https://huggingface.co/datasets/nvidia/Nemotron-PII)
- **Publisher:** NVIDIA
- **Size:** 200,000 rows (100K train / 100K test), synthetic, ~30 domains
- **Labels:** 50+ PII types; character-level spans (`{start, end, text, label}`) + inline-tagged text; English (US locale)
- **Task:** PII Detection / span extraction (token classification)
- **Primary metric:** token/span-level F1 (micro-averaged)
- **License:** CC-BY-4.0
### Suggested results to seed the leaderboard
⚠️ **Single-source, self-reported.** NVIDIA's dataset card publishes **no** baselines. The only public numbers I could find come from one community author (Kalyan KS), who trained the winning models — treat as a community claim, not an independent benchmark. Micro-F1 on 10K test instances:
| Model | Precision | Recall | F1 |
|---|---|---|---|
| GPT-4o-mini (zero-shot) | 95.21 | 67.05 | 78.69 |
| Ettin-32M (fine-tuned) | 95.96 | 95.49 | 95.73 |
| Ettin-68M (fine-tuned) | 96.35 | 96.19 | 96.27 |
Source: https://huggingface.co/blog/kalyan-ks/tiny-pii-entity-detection-models . Given the single-source provenance, it may be cleaner to **request the dataset/task first and leave results for the community/maintainer to populate** from `pwc-archive` or peer-reviewed work.
### Note
A large, recent, industry-domain PII dataset with fine-grained labels — good complement to AI4Privacy and Privy. Thanks!
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by inspecting this repository's existing dataset and leaderboard indexing workflow, then review the nvidia/Nemotron-PII dataset card for its schema, labels, license, and evaluation task. Confirm how similar PII or token-classification benchmarks are represented, and leave results unseeded unless an accepted source is available; done means the dataset and PII Detection task are indexed with the stated metric and metadata.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- huggingface
- Domain
- machine-learning, security
- Issue type
- Feature
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 65/100