ncbi / ncbi/datasets

Inconsistent host annotation for ON022733.1 in datasets CLI and NCBI website

Open
#595 5 comments 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

bug
Dominant language
Jupyter Notebook
Stars
559
Forks
70
Avg merge
1m
Merged PRs (30d)
2

Description

Before opening an issue, please:
  • Make sure you are using the latest version using datasets --version
  • Review our documentation

Describe the bug
When ingesting NCBI data to mirror on Pathoplexus, we noticed a difference between the data produced by the datasets CLI versus the NCBI website.

Specifically, we noticed that accession ON022733.1 has a annotation for host (Callithrix) on the website, which is not present in the datasets download output. In the past (2026-03-12), the host was included in the datasets download output. See here for a comparison of an older version of the ON022733 accession we ingested in the past, versus what we received today: https://pathoplexus.org/seq/PP_005MRGX.2/versions?compare=1%2C2

In parallel, we've seen quite a few updates to host information for other sequences we ingest from NCBI (would be happy to provide specifics if you like).

Could this be due to a bug in the datasets CLI? Or does it maybe mean there has been an update to the data and it just takes a little while for the website and the datasets outputs to sync?

Indicate what operating system you're using
macOS

To Reproduce

Steps to reproduce the behavior:

datasets download virus genome taxon 3046277

unzip ncbi_dataset.zip

# when I look at this output, there is no `host` annotated
grep "ON022733.1" ncbi_dataset/data/data_report.jsonl

# (or with jq for formatting, if installed)
grep "ON022733.1" ncbi_dataset/data/data_report.jsonl | jq '.'

Compare to:

  1. Go to https://www.ncbi.nlm.nih.gov/nuccore/ON022733.1
  2. Search for 'host'
  3. See /host="Callithrix"

Expected behavior
I would expect both sources of information to be consistent, i.e., for both the datasets output and the website have host: Callithrix or neither of them to have it.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Run datasets download virus genome taxon 3046277 and inspect ncbi_dataset/data/data_report.jsonl for ON022733.1. Compare its host annotation with the NCBI page at nuccore/ON022733.1, then determine whether the sources are out of sync or the CLI omits the field; done means the discrepancy is explained and the expected output is consistent.

Written by the indexing model from the issue text.

Assessment

Domain
cli, data
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.