nf-core / nf-core/hic

samplesheet check too stringent for header check

Open
#152 8 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug
Dominant language
Nextflow
Stars
112
Forks
68
PR merge metrics
No merged PRs in 30d

Description

Description of the bug

I'm starting to use v2.0.0 of the nf-core HiC. I used the previous version but always submitted one sample at a time. This time, I created a samplesheet and am running into an issue where hic doesn't think the file has a header. It does. The has_header() function of the cvs module used in check_samplesheet.py is overly stringent in how it defines headers and seems like it would fail for must samplesheets, as it does for mine.

The following sample sheets will fail and succeed, respectively:

sample,fastq_1,fastq_2
RH41_B6,1,2
SMS_A3,1,2
sample,fastq_1,fastq_2
RH41_B6,1,2
SMS_A3,p1,q2
Command used and terminal output
nextflow run /home/ass6094/bin/nextflow_modules/hic_v2.0.0/main.nf \
--digestion 'qiagen' \
--input /projects/b1103/HIC_Macquarrie/hic_round2/samplesheet.csv \
  --outdir $outdir \
  --fasta /projects/genomicsshare/AWS_iGenomes/references/Homo_sapiens/NCBI/GRCh38/Sequence/WholeGenomeFasta/genome.fa  \
 --bwt2_index /projects/genomicsshare/AWS_iGenomes/references/Homo_sapiens/NCBI/GRCh38/Sequence/Bowtie2Index/   \
--split_fastq --fastq_chunks_size 10000000   --max_memory 64.GB   --bin_size \ 20000,40000,150000,500000,1000000  \ --bwt2_opts_end2end \
'--very-sensitive -L 30 --score-min L,-0.6,-0.2 --end-to-end --reorder -p 14'   --bwt2_opts_trimmed ' \
--very-sensitive -L 20 --score-min L,-0.6,-0.2 --end-to-end --reorder -p 14' \
-profile singularity,slurmshort   -with-report hic_report.html -with-trace \
-with-timeline hic_timeline.html   -with-dag hic_dag.png -bg   -w $scratch

Output:

------------------------------------------------------
                                        ,--./,-.
        ___     __   __   __   ___     /,-._.--~'
  |\ | |__  __ /  ` /  \ |__) |__         }  {
  | \| |       \__, \__/ |  \ |___     \`-._,-`-,
                                        `._,._,'
  nf-core/hic v2.0.0
------------------------------------------------------
Core Nextflow options
  runName                      : magical_hamilton
  containerEngine              : singularity
  launchDir                    : /projects/b1103/HIC_Macquarrie/hic_round2
  workDir                      : /scratch/ass6094/hic/nextflow
  projectDir                   : /home/ass6094/bin/nextflow_modules/hic_v2.0.0
  userName                     : ass6094
  profile                      : singularity,slurmshort
  configFiles                  : /home/ass6094/bin/nextflow_modules/hic_v2.0.0/nextflow.config

Input/output options
  input                        : /projects/b1103/HIC_Macquarrie/hic_round2/samplesheet.csv
  outdir                       : /projects/b1103/HIC_Macquarrie/hic_round2/NextflowResults/

Reference genome options
  fasta                        : /projects/genomicsshare/AWS_iGenomes/references/Homo_sapiens/NCBI/GRCh38/Sequence/WholeGenomeFasta/genome.fa
  bwt2_index                   : /projects/genomicsshare/AWS_iGenomes/references/Homo_sapiens/NCBI/GRCh38/Sequence/Bowtie2Index/

Digestion Hi-C
  digestion                    : qiagen

DNAse Hi-C
  min_cis_dist                 : 0

Alignments
  split_fastq                  : true
  fastq_chunks_size            : 10000000
  bwt2_opts_end2end            : --very-sensitive -L 30 --score-min L,-0.6,-0.2 --end-to-end --reorder -p 14
  bwt2_opts_trimmed            : --very-sensitive -L 20 --score-min L,-0.6,-0.2 --end-to-end --reorder -p 14

Valid Pairs Detection
  max_insert_size              : 0
  min_insert_size              : 0
  max_restriction_fragment_size: 0
  min_restriction_fragment_size: 0

Contact maps
  bin_size                     : 20000,40000,150000,500000,1000000
  ice_filter_high_count_perc   : 0
  res_zoomify                  : null

Downstream Analysis
  res_dist_decay               : 250000
  tads_caller                  : insulation
  res_tads                     : 40000

Max job request options
  max_cpus                     : 14
  max_memory                   : 64.GB
  max_time                     : 10d

!! Only displaying parameters that differ from the pipeline defaults !!
------------------------------------------------------
If you use nf-core/hic for your analysis please cite:

* The pipeline
  https://doi.org/10.5281/zenodo.2669513

* The nf-core framework
  https://doi.org/10.1038/s41587-020-0439-x

* Software dependencies
  https://github.com/nf-core/hic/blob/master/CITATIONS.md
------------------------------------------------------
WARN: A process with name 'BOWTIE2_ALIGN_TRIMMED' is defined more than once in module script: /home/ass6094/bin/nextflow_modules/hic_v2.0.0/./workflows/../subworkflows/local/./hicpro_mapping.nf -- Make sure to not define the same function as process
[65/1f640c] Submitted process > NFCORE_HIC:HIC:PREPARE_GENOME:GET_RESTRICTION_FRAGMENTS (^GATC)
[11/c26455] Submitted process > NFCORE_HIC:HIC:INPUT_CHECK:SAMPLESHEET_CHECK (samplesheet.csv)
[10/5e7883] Submitted process > NFCORE_HIC:HIC:PREPARE_GENOME:CUSTOM_GETCHROMSIZES (genome.fa)
Error executing process > 'NFCORE_HIC:HIC:INPUT_CHECK:SAMPLESHEET_CHECK (samplesheet.csv)'

Caused by:
  Process `NFCORE_HIC:HIC:INPUT_CHECK:SAMPLESHEET_CHECK (samplesheet.csv)` terminated with an error exit status (1)

Command executed:

  check_samplesheet.py \
      samplesheet.csv \
      samplesheet.valid.csv
  
  cat <<-END_VERSIONS > versions.yml
  "NFCORE_HIC:HIC:INPUT_CHECK:SAMPLESHEET_CHECK":
      python: $(python --version | sed 's/Python //g')
  END_VERSIONS

Command exit status:
  1

Command output:
  

Command error:
  WARNING: While bind mounting '/projects/b1103/HIC_Macquarrie/hic_round2:/projects/b1103/HIC_Macquarrie/hic_round2': destination is already in the mount point list
  WARNING: While bind mounting '/home/ass6094/bin/nextflow_modules/hic_v2.0.0/bin:/home/ass6094/bin/nextflow_modules/hic_v2.0.0/bin': destination is already in the mount point list
  WARNING: While bind mounting '/scratch/ass6094/hic/nextflow/11/c26455104fe4b102d7953120fa3a65:/scratch/ass6094/hic/nextflow/11/c26455104fe4b102d7953120fa3a65': destination is already in the mount point list
  WARNING: Skipping mount /hpc/software/singularity/3.8.1/var/singularity/mnt/session/etc/resolv.conf [files]: /etc/resolv.conf doesn't exist in container
  [CRITICAL] The given sample sheet does not appear to contain a header.
Relevant files

No response

System information

nextflow version 22.10.5.5840
Hardware: Slurm HPC
Executor: slurm
Container engine:Singularity
OS: Redhat Linux 7.9
Version of nf-core/hic 2.0.0

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with check_samplesheet.py and trace its use of the has_header() function in the csv module. Reproduce the two sample sheets from the issue, then verify that the first is accepted as a valid header while preserving rejection of genuinely invalid headers.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
bioinformatics
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.