Inconsistent host annotation for ON022733.1 in datasets CLI and NCBI website
Nobody has claimed this yet.
- Dominant language
- Jupyter Notebook
- Stars
- 559
- Forks
- 70
- Avg merge
- 1m
- Merged PRs (30d)
- 2
Description
Before opening an issue, please:
- Make sure you are using the latest version using
datasets --version - Review our documentation
Describe the bug
When ingesting NCBI data to mirror on Pathoplexus, we noticed a difference between the data produced by the datasets CLI versus the NCBI website.
Specifically, we noticed that accession ON022733.1 has a annotation for host (Callithrix) on the website, which is not present in the datasets download output. In the past (2026-03-12), the host was included in the datasets download output. See here for a comparison of an older version of the ON022733 accession we ingested in the past, versus what we received today: https://pathoplexus.org/seq/PP_005MRGX.2/versions?compare=1%2C2
In parallel, we've seen quite a few updates to host information for other sequences we ingest from NCBI (would be happy to provide specifics if you like).
Could this be due to a bug in the datasets CLI? Or does it maybe mean there has been an update to the data and it just takes a little while for the website and the datasets outputs to sync?
Indicate what operating system you're using
macOS
To Reproduce
Steps to reproduce the behavior:
datasets download virus genome taxon 3046277
unzip ncbi_dataset.zip
# when I look at this output, there is no `host` annotated
grep "ON022733.1" ncbi_dataset/data/data_report.jsonl
# (or with jq for formatting, if installed)
grep "ON022733.1" ncbi_dataset/data/data_report.jsonl | jq '.'
Compare to:
- Go to https://www.ncbi.nlm.nih.gov/nuccore/ON022733.1
- Search for 'host'
- See
/host="Callithrix"
Expected behavior
I would expect both sources of information to be consistent, i.e., for both the datasets output and the website have host: Callithrix or neither of them to have it.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Run datasets download virus genome taxon 3046277 and inspect ncbi_dataset/data/data_report.jsonl for ON022733.1. Compare its host annotation with the NCBI page at nuccore/ON022733.1, then determine whether the sources are out of sync or the CLI omits the field; done means the discrepancy is explained and the expected output is consistent.
Written by the indexing model from the issue text.
Assessment
- Domain
- cli, data
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100