nf-core / nf-core/variantbenchmarking
unexpected F1 score running somatic snv/indel variants benchmarking
Nobody has claimed this yet.
- Dominant language
- Nextflow
- Stars
- 51
- Forks
- 31
- Avg merge
- 1d 22h
- Merged PRs (30d)
- 1
Description
Description of the bug
Hi,
I am running the following
nextflow run nf-core/variantbenchmarking
-r 1.3.0
-c $inDIR/yale.config
-work-dir $workDIR
--input $inDIR/${smplshtFILE}.csv
--outdir $outPATH/$outDIR
--analysis $analysisTYPE
--variant_type snv
--method sompy \
--truth_id ${smpID}
--truth_vcf $refPATH/${refFILE}.vcf.gz
--preprocess split_multiallelic,normalize,deduplicate,prepy
--fasta $GENOME
--fai $GENOMEIDX
I am testing a VCF derived from RNAseq and one from WGS, against the same WGS. Again, this is to have a sort of control
The resulting sompy.summary.csv is below
Tool,File,Type,TP_base,TP,FN,TP_call,FP,UNK,Recall,Precision,recall_lower,recall_upper,recall2,precision_lower,precision_upper,na,ambiguous,fp.region.size,F1
RNAsentieonGATK,RNAsentieonGATK.661T.GATK.stats.csv,SNVs,5,1,4,67,66,0,0.2,0.0149253731343283,0.0,0.6286264006319943,0.2,0.0,0.0676321414491473,0.0,0.0,675359842.0,0.0977256802900045
WGSsentieonGATK,WGSsentieonGATK.661T.GATK.stats.csv,SNVs,5,5,0,5,0,0,1.0,1.0,0.4781762498950185,1.0,1.0,0.4781762498950185,1.0,0.0,0.0,675359842.0,0.0
The second row VCF is the same as the reference. Beyond the
recall_lower,recall_upper,recall2,precision_lower,precision_upper which I will have to read about, I have F1=0, but Recall and Precision are both 1.0
Am I doing something. wrong ?
Thanks
Command used and terminal output
Relevant files
No response
System information
No response
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Reproduce the supplied Nextflow command and inspect the generated sompy.summary.csv, starting with how F1 is calculated from TP, FP, FN, Recall, and Precision. No relevant files or tests are identified; done means the discrepancy is explained and the benchmark reports a consistent F1 value or documents why it is 0.
Written by the indexing model from the issue text.
Assessment
- Domain
- bioinformatics
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100