huggingface / huggingface/paperswithcode-feedback

[Dataset Request] Index ImmigrationQA for U.S. Immigration Law QA benchmarking (arXiv:2605.30589)

Open
#16 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
No language data
Stars
10
Forks
1
PR merge metrics
No merged PRs in 30d

Description

Hi there,

I'm the author of a paper I recently uploaded to arXiv and I'd love to get the dataset properly indexed on Papers with Code so the benchmark results can be tracked.

### Paper
- **Title:** ImmigrationQA: A Source-Grounded Dataset and Small-Model Adaptation for U.S. Immigration Law
- **arXiv:** https://arxiv.org/abs/2605.30589
- **Author:** Nazarii Shportun

### About the Dataset
ImmigrationQA is a source-grounded QA dataset built on top of U.S. immigration law — covering official policy documents, federal regulations, and procedural guidance. It's designed to support retrieval-augmented and fine-tuning approaches for legal QA in a high-stakes domain.

- **Size:** 17,058 QA pairs across 13 immigration subdomains
- Each answer is grounded in a specific source passage from the official documents

### Evaluation
We evaluated models using an LLM-as-judge approach on a held-out 101-example split, scoring answers from 0–3. Here are the results reported in the paper:

| Model | Mean Score (0–3) |
|---|---|
| Fine-tuned Llama 3.2 3B Instruct (LoRA) | ~1.08 |
| Llama 3 8B base (zero-shot) | ~0.85 |
| Claude Sonnet (zero-shot) | ~1.52 |

### What I'm Asking
Could you index **ImmigrationQA** as a dataset in Papers with Code and link it to the paper above under the **Question Answering** task? That would make it possible to properly register the benchmark results and allow comparison with other QA work.

Happy to provide any additional info if needed. Thanks so much!

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.