PolicyEngine / PolicyEngine/policyengine-core
download_huggingface_dataset hardcodes repo_type="model", so it cannot fetch datasets repos
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 22
- Forks
- 30
- Avg merge
- 14h 33m
- Merged PRs (30d)
- 7
Description
policyengine_core.tools.hugging_face.download_huggingface_dataset (policyengine-core 3.32.5, policyengine_core/tools/hugging_face.py:92-94) always calls hf_hub_download(..., repo_type="model"). Run against the canonical US population release policyengine/populace-us (a Hugging Face datasets repo; certified tag populace-us-2024-spm-20260915), it raises RepositoryNotFoundError: 401, because there is no model repo of that name. Observed today while re-qualifying the PolicyEngine 6.0.0 bundle (registry re-qualification lane, 2026-09-15); the wrapper policyengine 6.0.0 avoids the helper by reading repo_type: "dataset" from its bundle manifest and downloading with the correct type, and recomputing the certified hash independently.
Suggested fix: accept a repo_type argument (default "model" for backward compatibility with policyengine-uk-data-private and the other model-typed data repos), or infer it by probing repo_info for both types; pass it through to hf_hub_download and to the private/gated check. This is separate from #538, which concerns token handling for public-but-gated repos.
🤖 Generated with Claude Code
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in policyengine_core/tools/hugging_face.py at download_huggingface_dataset and inspect the hf_hub_download call and private/gated check around lines 92-94. Add repository-type handling while preserving the model default, then verify that policyengine/populace-us downloads as a dataset and existing model-typed repositories still work.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- huggingface, python
- Domain
- data
- Issue type
- Bug
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 78/100