DDBJ study ids (DRP123456) do not map correctly
Nobody has claimed this yet.
- Dominant language
- Nextflow
- Stars
- 202
- Forks
- 93
- Avg merge
- 1d 18m
- Merged PRs (30d)
- 15
Description
Description of the bug
When the input list includes DRP****** ids or PRJDB***** ids, the sra_ids_to_runid.py script fails with error or returns the incorrect data (for a different ID).
It appears that the efetch query fails in cls._id_to_srx(identifier)
You can take for example the id mentioned in the script: DRP004793 will return the incorrect results - a different study is returned. It returns the information for SRP000742. Here is the query and result:
curl "https://eutils.ncbi.nlm.nih.gov/entrez/eutils/efetch.fcgi?id=004793&db=sra&rettype=runinfo&retmode=text"
Run,ReleaseDate,LoadDate,spots,bases,spots_with_mates,avgLength,size_MB,AssemblyName,download_path,Experiment,LibraryName,LibraryStrategy,LibrarySelection,LibrarySource,LibraryLayout,InsertSize,InsertDev,Platform,Model,SRAStudy,BioProject,Study_Pubmed_id,ProjectID,Sample,BioSample,SampleType,TaxID,ScientificName,SampleName,g1k_pop_code,source,g1k_analysis_group,Subject_ID,Sex,Disease,Tumor,Affection_Status,Analyte_Type,Histological_Type,Body_Site,CenterName,Submission,dbgap_study_accession,Consent,RunHash,ReadHash
SRR016082,2009-06-12 17:55:38,2014-05-26 05:26:24,5832504,209970144,0,36,1149,,https://sra-downloadb.be-md.ncbi.nlm.nih.gov/sos5/sra-pub-zq-11/SRR000/016/SRR016082/SRR016082.sralite.1,SRX004537,Cryptosporidium_nref,FL-cDNA,cDNA,TRANSCRIPTOMIC,SINGLE,0,0,ILLUMINA,Illumina Genome Analyzer,SRP000742,PRJDA33427,,33427,SRS002838,SAMN00004429,simple,5807,Cryptosporidium parvum,Cryptosporidium_nref,,,,,,,no,,,,,UT-MGS,SRA002056,,public,C39D154DF2031E73C9578A3317C42931,4166465432361F26049099FFC7742B15
I am not sure the best solution other than doing eutils esearch for example, term=DRP004793, and using the returned ids.
curl "https://eutils.ncbi.nlm.nih.gov/entrez/eutils/esearch.fcgi?db=sra&term=DRP004793"
<?xml version="1.0" encoding="UTF-8" ?>
<!DOCTYPE eSearchResult PUBLIC "-//NLM//DTD esearch 20060628//EN" "https://eutils.ncbi.nlm.nih.gov/eutils/dtd/20060628/esearch.dtd">
<eSearchResult><Count>645</Count><RetMax>20</RetMax><RetStart>0</RetStart><IdList>
<Id>7938219</Id>
<Id>7938218</Id>
<Id>7507857</Id>
<Id>7507856</Id>
<Id>7507855</Id>
<Id>7507854</Id>
<Id>7507853</Id>
<Id>7507852</Id>
<Id>7507851</Id>
<Id>7507850</Id>
<Id>7507849</Id>
<Id>7507848</Id>
<Id>7507847</Id>
<Id>7507846</Id>
<Id>7507845</Id>
<Id>7507844</Id>
<Id>7507843</Id>
<Id>7507842</Id>
<Id>7507841</Id>
<Id>7507840</Id>
</IdList><TranslationSet/><TranslationStack> <TermSet> <Term>DRP004793[All Fields]</Term> <Field>All Fields</Field> <Count>645</Count> <Explode>N</Explode> </TermSet> <OP>GROUP</OP> </TranslationStack><QueryTranslation>DRP004793[All Fields]</QueryTranslation></eSearchResult>
Command used and terminal output
nextflow run main.nf --input 'input.txt'
This will fail to return results from DRP004793.
just running the sra_ids_to_runinfo.py directly with input.txt gives this error:
Traceback (most recent call last):
File "bin/sra_ids_to_runinfo.py", line 498, in <module>
sys.exit(main())
^^^^^^
File "/bin/sra_ids_to_runinfo.py", line 494, in main
fetch_sra_runinfo(args.file_in, args.file_out, ena_metadata_fields)
File "bin/sra_ids_to_runinfo.py", line 470, in fetch_sra_runinfo
ids = DatabaseResolver.expand_identifier(db_id)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "bin/sra_ids_to_runinfo.py", line 223, in expand_identifier
return cls._id_to_srx(identifier)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "bin/sra_ids_to_runinfo.py", line 241, in _id_to_srx
cls._content_check(response, identifier)
File "bin/sra_ids_to_runinfo.py", line 232, in _content_check
if response.status == 204:
^^^^^^^^^^^^^^^
AttributeError: 'NoneType' object has no attribute 'status'
Relevant files
input.txt:
SRP102769
DRP004793
System information
This would happen on any system - have tested locally, awsbatch, docker, linux, macOS.
Version of nf-core/fetchngs: 1.12.0
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with bin/sra_ids_to_runinfo.py, especially DatabaseResolver.expand_identifier, _id_to_srx, and _content_check. Reproduce the failure using input.txt with SRP102769 and DRP004793, then compare the efetch and esearch responses described in the issue. Done means DDBJ DRP and PRJDB identifiers resolve to the correct metadata without the NoneType error or unrelated study results.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- data, databases
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Clearly specified
- Newbie friendliness
- 45/100