huggingface / huggingface/paperswithcode-feedback
[Dataset Request] Index ImmigrationQA for U.S. Immigration Law QA benchmarking (arXiv:2605.30589)
- Dominant language
- No language data
- Stars
- 10
- Forks
- 1
- PR merge metrics
- No merged PRs in 30d
Description
Hi there,
I'm the author of a paper I recently uploaded to arXiv and I'd love to get the dataset properly indexed on Papers with Code so the benchmark results can be tracked.
### Paper
- **Title:** ImmigrationQA: A Source-Grounded Dataset and Small-Model Adaptation for U.S. Immigration Law
- **arXiv:** https://arxiv.org/abs/2605.30589
- **Author:** Nazarii Shportun
### About the Dataset
ImmigrationQA is a source-grounded QA dataset built on top of U.S. immigration law — covering official policy documents, federal regulations, and procedural guidance. It's designed to support retrieval-augmented and fine-tuning approaches for legal QA in a high-stakes domain.
- **Size:** 17,058 QA pairs across 13 immigration subdomains
- Each answer is grounded in a specific source passage from the official documents
### Evaluation
We evaluated models using an LLM-as-judge approach on a held-out 101-example split, scoring answers from 0–3. Here are the results reported in the paper:
| Model | Mean Score (0–3) |
|---|---|
| Fine-tuned Llama 3.2 3B Instruct (LoRA) | ~1.08 |
| Llama 3 8B base (zero-shot) | ~0.85 |
| Claude Sonnet (zero-shot) | ~1.52 |
### What I'm Asking
Could you index **ImmigrationQA** as a dataset in Papers with Code and link it to the paper above under the **Question Answering** task? That would make it possible to properly register the benchmark results and allow comparison with other QA work.
Happy to provide any additional info if needed. Thanks so much!
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.