cwl-runner re-executes cached jobs

I am using the `cwl-runner` to process real genomic data.
The caching system has been very helpful when I need to fix some input parameter.
However, sometimes the software re-executes some of the jobs that were already completed.

## Expected Behavior
The `cwl-runner` should use cached data instead of re-executing jobs.

## Actual Behavior
The `cwl-runner` unexpectedly re-executes some jobs. I guess it happens with tools that use path of the input files within result files such as gzip files and report files.

## Workflow Code

I wrote a workflow example that:

1. Download FASTQ files;
2. Perform quality assessment;
3. Trim files;
3. Perform quality assessment on trimmed files.

``` yml
cwlVersion: v1.0
class: Workflow

requirements:
ScatterFeatureRequirement: {}
StepInputExpressionRequirement: {}

inputs:
urls: string[]
workers: int
groups: string[]
reportFileRaw: string
small_rna: boolean
length: int
max_length: int
max_n: int
reportFileTrim: string

outputs:
files:
type: File[]
outputSource: retrieve/files
qa_report_raw:
type: File
outputSource: qa_raw/qc_report
trim_files:
type: File[]
outputSource: trim/trim_file
trim_stats_files:
type: File[]
outputSource: trim/stats_file
qc_report_trim:
type: File
outputSource: qa_trim/qc_report

steps:
retrieve:
run: wget.cwl
in:
urls: urls
out: [files]
qa_raw:
run: https://raw.githubusercontent.com/labbcb/docker-rqc/master/Rqc.cwl
in:
files: retrieve/files
groups: groups
workers: workers
reportFile: reportFileRaw
out: [qc_report]
trim:
run: https://raw.githubusercontent.com/labbcb/docker-trimgalore/master/TrimGalore.cwl
scatter: file
in:
file: retrieve/files
small_rna: small_rna
length: length
max_length: max_length
max_n: max_n
out: [trim_file, stats_file]
qa_trim:
run: https://raw.githubusercontent.com/labbcb/docker-rqc/master/Rqc.cwl
in:
files: trim/trim_file
groups: groups
workers: workers
reportFile: reportFileTrim
out: [qc_report]
```

`wget.cwl`:

``` yml
cwlVersion: v1.0
class: CommandLineTool

baseCommand: [wget]

inputs:
urls:
type: string[]
inputBinding: {}

outputs:
files:
type: File[]
outputBinding:
glob: "*"
```

## Tests

I ran this workflow three times.
In the first time, all jobs were executed.
In the second time, one job was re-executed (`trim job` for the second file, but not the first file).
Since output file from second `trim job` was changed, the `cwl-runner` correctly re-execute the last step.
In the last time, `cwlref-runner` collected all cached data.

``` bash
cwl-runner --cachedir cache/ --outdir result1 \
workflow_cache.cwl workflow_cache.yml > stdout.1.txt 2> stderr.1.txt
cwl-runner --cachedir cache/ --outdir result2 \
workflow_cache.cwl workflow_cache.yml > stdout.2.txt 2> stderr.2.txt
cwl-runner --cachedir cache/ --outdir result3 \
workflow_cache.cwl workflow_cache.yml > stdout.3.txt 2> stderr.3.txt
```

Input:

``` yml
urls: [
"https://github.com/Bioconductor-mirror/ShortRead/raw/release-3.5/inst/extdata/E-MTAB-1147/ERR127302_1_subset.fastq.gz",
"https://github.com/Bioconductor-mirror/ShortRead/raw/release-3.5/inst/extdata/E-MTAB-1147/ERR127302_2_subset.fastq.gz"
]
workers: 2
groups: [None, None]
reportFileRaw: qa_report_raw
small_rna: true
length: 18
max_length: 30
max_n: 2
reportFileTrim: qa_report_trim
```

## Outputs

First execution.

```
/usr/local/bin/cwl-runner 1.0.20170717120410
Resolved 'workflow_cache.cwl' to 'file:///home/welliton/lab/cwl_workflow_cache/workflow_cache.cwl'
/usr/local/lib/python2.7/dist-packages/urllib3/util/ssl_.py:335: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
SNIMissingWarning
/usr/local/lib/python2.7/dist-packages/urllib3/util/ssl_.py:133: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
InsecurePlatformWarning
/usr/local/lib/python2.7/dist-packages/urllib3/util/ssl_.py:133: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
InsecurePlatformWarning
/usr/local/lib/python2.7/dist-packages/urllib3/util/ssl_.py:133: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
InsecurePlatformWarning
[workflow workflow_cache.cwl] start
[step retrieve] start
[job retrieve] Output of job will be cached in /home/welliton/lab/cwl_workflow_cache/cache/58d8a9f042a3f2bac76bc2c7b6e72bf9
[job retrieve] /home/welliton/lab/cwl_workflow_cache/cache/58d8a9f042a3f2bac76bc2c7b6e72bf9$ wget \
https://github.com/Bioconductor-mirror/ShortRead/raw/release-3.5/inst/extdata/E-MTAB-1147/ERR127302_1_subset.fastq.gz \
https://github.com/Bioconductor-mirror/ShortRead/raw/release-3.5/inst/extdata/E-MTAB-1147/ERR127302_2_subset.fastq.gz
--2017-07-22 11:59:34-- https://github.com/Bioconductor-mirror/ShortRead/raw/release-3.5/inst/extdata/E-MTAB-1147/ERR127302_1_subset.fastq.gz
Resolving github.com (github.com)... 192.30.253.113, 192.30.253.112
Connecting to github.com (github.com)|192.30.253.113|:443... connected.
HTTP request sent, awaiting response... 302 Found
Location: https://raw.githubusercontent.com/Bioconductor-mirror/ShortRead/release-3.5/inst/extdata/E-MTAB-1147/ERR127302_1_subset.fastq.gz [following]
--2017-07-22 11:59:35-- https://raw.githubusercontent.com/Bioconductor-mirror/ShortRead/release-3.5/inst/extdata/E-MTAB-1147/ERR127302_1_subset.fastq.gz
Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 151.101.56.133
Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|151.101.56.133|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 1408765 (1.3M) [application/octet-stream]
Saving to: 'ERR127302_1_subset.fastq.gz'

0K .......... .......... .......... .......... .......... 3% 153K 9s
50K .......... .......... .......... .......... .......... 7% 424K 6s
100K .......... .......... .......... .......... .......... 10% 996K 4s
150K .......... .......... .......... .......... .......... 14% 1.32M 3s
200K .......... .......... .......... .......... .......... 18% 717K 3s
250K .......... .......... .......... .......... .......... 21% 1.92M 2s
300K .......... .......... .......... .......... .......... 25% 2.24M 2s
350K .......... .......... .......... .......... .......... 29% 1.94M 2s
400K .......... .......... .......... .......... .......... 32% 911K 2s
450K .......... .......... .......... .......... .......... 36% 3.30M 1s
500K .......... .......... .......... .......... .......... 39% 3.63M 1s
550K .......... .......... .......... .......... .......... 43% 3.81M 1s
600K .......... .......... .......... .......... .......... 47% 4.46M 1s
650K .......... .......... .......... .......... .......... 50% 4.60M 1s
700K .......... .......... .......... .......... .......... 54% 3.00M 1s
750K .......... .......... .......... .......... .......... 58% 805K 1s
800K .......... .......... .......... .......... .......... 61% 3.25M 1s
850K .......... .......... .......... .......... .......... 65% 2.99M 0s
900K .......... .......... .......... .......... .......... 69% 3.02M 0s
950K .......... .......... .......... .......... .......... 72% 3.04M 0s
1000K .......... .......... .......... .......... .......... 76% 2.93M 0s
1050K .......... .......... .......... .......... .......... 79% 3.29M 0s
1100K .......... .......... .......... .......... .......... 83% 3.13M 0s
1150K .......... .......... .......... .......... .......... 87% 1.29M 0s
1200K .......... .......... .......... .......... .......... 90% 3.11M 0s
1250K .......... .......... .......... .......... .......... 94% 3.07M 0s
1300K .......... .......... .......... .......... .......... 98% 3.14M 0s
1350K .......... .......... ..... 100% 4.56M=1.1s

2017-07-22 11:59:36 (1.25 MB/s) - 'ERR127302_1_subset.fastq.gz' saved [1408765/1408765]

--2017-07-22 11:59:36-- https://github.com/Bioconductor-mirror/ShortRead/raw/release-3.5/inst/extdata/E-MTAB-1147/ERR127302_2_subset.fastq.gz
Connecting to github.com (github.com)|192.30.253.113|:443... connected.
HTTP request sent, awaiting response... 302 Found
Location: https://raw.githubusercontent.com/Bioconductor-mirror/ShortRead/release-3.5/inst/extdata/E-MTAB-1147/ERR127302_2_subset.fastq.gz [following]
--2017-07-22 11:59:37-- https://raw.githubusercontent.com/Bioconductor-mirror/ShortRead/release-3.5/inst/extdata/E-MTAB-1147/ERR127302_2_subset.fastq.gz
Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|151.101.56.133|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 1402866 (1.3M) [application/octet-stream]
Saving to: 'ERR127302_2_subset.fastq.gz'

0K .......... .......... .......... .......... .......... 3% 178K 7s
50K .......... .......... .......... .......... .......... 7% 495K 5s
100K .......... .......... .......... .......... .......... 10% 1.13M 3s
150K .......... .......... .......... .......... .......... 14% 1.54M 3s
200K .......... .......... .......... .......... .......... 18% 835K 2s
250K .......... .......... .......... .......... .......... 21% 2.24M 2s
300K .......... .......... .......... .......... .......... 25% 2.61M 2s
350K .......... .......... .......... .......... .......... 29% 2.27M 1s
400K .......... .......... .......... .......... .......... 32% 1.01M 1s
450K .......... .......... .......... .......... .......... 36% 3.91M 1s
500K .......... .......... .......... .......... .......... 40% 4.29M 1s
550K .......... .......... .......... .......... .......... 43% 4.77M 1s
600K .......... .......... .......... .......... .......... 47% 4.84M 1s
650K .......... .......... .......... .......... .......... 51% 5.56M 1s
700K .......... .......... .......... .......... .......... 54% 3.29M 1s
750K .......... .......... .......... .......... .......... 58% 954K 1s
800K .......... .......... .......... .......... .......... 62% 3.54M 0s
850K .......... .......... .......... .......... .......... 65% 3.51M 0s
900K .......... .......... .......... .......... .......... 69% 3.50M 0s
950K .......... .......... .......... .......... .......... 72% 3.48M 0s
1000K .......... .......... .......... .......... .......... 76% 3.37M 0s
1050K .......... .......... .......... .......... .......... 80% 3.77M 0s
1100K .......... .......... .......... .......... .......... 83% 3.40M 0s
1150K .......... .......... .......... .......... .......... 87% 1.54M 0s
1200K .......... .......... .......... .......... .......... 91% 3.45M 0s
1250K .......... .......... .......... .......... .......... 94% 4.01M 0s
1300K .......... .......... .......... .......... .......... 98% 3.65M 0s
1350K .......... ......... 100% 6.22M=0.9s

2017-07-22 11:59:38 (1.45 MB/s) - 'ERR127302_2_subset.fastq.gz' saved [1402866/1402866]

FINISHED --2017-07-22 11:59:38--
Total wall clock time: 4.5s
Downloaded: 2 files, 2.7M in 2.0s (1.35 MB/s)
[job retrieve] completed success
[step retrieve] completed success
[step qa_raw] start
[job qa_raw] Output of job will be cached in /home/welliton/lab/cwl_workflow_cache/cache/3e1def7ec528ae55597d12d7db3fcfa4
[job qa_raw] /home/welliton/lab/cwl_workflow_cache/cache/3e1def7ec528ae55597d12d7db3fcfa4$ docker \
run \
-i \
--volume=/home/welliton/lab/cwl_workflow_cache/cache/3e1def7ec528ae55597d12d7db3fcfa4:/var/spool/cwl:rw \
--volume=/tmp/tmpleNRpN:/tmp:rw \
--volume=/home/welliton/lab/cwl_workflow_cache/cache/58d8a9f042a3f2bac76bc2c7b6e72bf9/ERR127302_1_subset.fastq.gz:/var/lib/cwl/stg025fbd87-3079-4446-a2e3-df9fc201bb9e/ERR127302_1_subset.fastq.gz:ro \
--volume=/home/welliton/lab/cwl_workflow_cache/cache/58d8a9f042a3f2bac76bc2c7b6e72bf9/ERR127302_2_subset.fastq.gz:/var/lib/cwl/stgaa190545-e671-4547-91d9-5fae63d2f122/ERR127302_2_subset.fastq.gz:ro \
--workdir=/var/spool/cwl \
--read-only=true \
--user=1000 \
--rm \
--env=TMPDIR=/tmp \
--env=HOME=/var/spool/cwl \
welliton/rqc \
Rqc \
--files \
/var/lib/cwl/stgaa190545-e671-4547-91d9-5fae63d2f122/ERR127302_2_subset.fastq.gz,/var/lib/cwl/stg025fbd87-3079-4446-a2e3-df9fc201bb9e/ERR127302_1_subset.fastq.gz \
--groups \
None,None \
--sample \
--reads \
1000000 \
--workers \
2 \
--report_file \
qa_report_raw \
--rds_file \
rqc.rds
[1] "/var/spool/cwl/qa_report_raw.html"
Warning messages:
1: In exists(sym, ns, inherits = FALSE) :
closing unused connection 4 (/var/lib/cwl/stg025fbd87-3079-4446-a2e3-df9fc201bb9e/ERR127302_1_subset.fastq.gz)
2: In exists(sym, ns, inherits = FALSE) :
closing unused connection 3 (/var/lib/cwl/stgaa190545-e671-4547-91d9-5fae63d2f122/ERR127302_2_subset.fastq.gz)
[job qa_raw] completed success
[step qa_raw] completed success
[step trim] start
[job trim] Output of job will be cached in /home/welliton/lab/cwl_workflow_cache/cache/694cbde9f5690336d246b18f673236d4
[job trim] /home/welliton/lab/cwl_workflow_cache/cache/694cbde9f5690336d246b18f673236d4$ docker \
run \
-i \
--volume=/home/welliton/lab/cwl_workflow_cache/cache/694cbde9f5690336d246b18f673236d4:/var/spool/cwl:rw \
--volume=/tmp/tmpr3TMdg:/tmp:rw \
--volume=/home/welliton/lab/cwl_workflow_cache/cache/58d8a9f042a3f2bac76bc2c7b6e72bf9/ERR127302_2_subset.fastq.gz:/var/lib/cwl/stg7a303a48-36eb-4b40-9ebf-0bbe8fadcbb6/ERR127302_2_subset.fastq.gz:ro \
--workdir=/var/spool/cwl \
--read-only=true \
--user=1000 \
--rm \
--env=TMPDIR=/tmp \
--env=HOME=/var/spool/cwl \
welliton/trimgalore:v0.4.4 \
trim_galore \
-e \
0.1 \
/var/lib/cwl/stg7a303a48-36eb-4b40-9ebf-0bbe8fadcbb6/ERR127302_2_subset.fastq.gz \
--length \
18 \
--max_length \
30 \
--max_n \
2 \
--phred33 \
--quality \
20 \
--small_rna \
--stringency \
1
Maximum length cutoff set to >> 30 bp <<; sequences longer than this threshold will be removed (only advised for smallRNA-trimming!)

Path to Cutadapt set as: 'cutadapt' (default)
1.13
Cutadapt seems to be working fine (tested command 'cutadapt --version')
Writing report to 'ERR127302_2_subset.fastq.gz_trimming_report.txt'

SUMMARISING RUN PARAMETERS
==========================
Input filename: /var/lib/cwl/stg7a303a48-36eb-4b40-9ebf-0bbe8fadcbb6/ERR127302_2_subset.fastq.gz
Trimming mode: single-end
Trim Galore version: 0.4.4
Cutadapt version: 1.13
Quality Phred score cutoff: 20
Quality encoding type selected: ASCII+33
Adapter sequence: 'TGGAATTCTCGG' (Illumina small RNA adapter; user defined)
Maximum trimming error rate: 0.1 (default)
Maximum number of tolerated Ns: 2
Minimum required adapter overlap (stringency): 1 bp
Minimum required sequence length before a sequence gets removed: 18 bp
Maxiumum tolerated read length after trimming (for smallRNA trimming): 30 bp
Output file(s) will be GZIP compressed

Writing final adapter and quality trimmed output to ERR127302_2_subset_trimmed.fq.gz

>>> Now performing quality (cutoff 20) and adapter trimming in a single pass for the adapter sequence: 'TGGAATTCTCGG' from file /var/lib/cwl/stg7a303a48-36eb-4b40-9ebf-0bbe8fadcbb6/ERR127302_2_subset.fastq.gz <<<
This is cutadapt 1.13 with Python 2.7.9
Command line parameters: -f fastq -e 0.1 -q 20 -O 1 -a TGGAATTCTCGG /var/lib/cwl/stg7a303a48-36eb-4b40-9ebf-0bbe8fadcbb6/ERR127302_2_subset.fastq.gz
Trimming 1 adapter with at most 10.0% errors in single-end mode ...
Finished in 0.40 s (20 us/read; 3.02 M reads/minute).

=== Summary ===

Total reads processed: 20,000
Reads with adapters: 6,160 (30.8%)
Reads written (passing filters): 20,000 (100.0%)

Total basepairs processed: 1,440,000 bp
Quality-trimmed: 147,439 bp (10.2%)
Total written (filtered): 1,283,165 bp (89.1%)

=== Adapter 1 ===

Sequence: TGGAATTCTCGG; Type: regular 3'; Length: 12; Trimmed: 6160 times.

No. of allowed errors:
0-9 bp: 0; 10-12 bp: 1

Bases preceding removed adapters:
A: 20.2%
C: 30.2%
G: 24.0%
T: 25.6%
none/other: 0.0%

Overview of removed sequences
length count expect max.err error counts
1 3924 5000.0 0 3924
2 1476 1250.0 0 1476
3 652 312.5 0 652
4 74 78.1 0 74
5 25 19.5 0 25
6 4 4.9 0 4
7 2 1.2 0 2
8 1 0.3 0 1
42 1 0.0 1 0 1
55 1 0.0 1 0 1

RUN STATISTICS FOR INPUT FILE: /var/lib/cwl/stg7a303a48-36eb-4b40-9ebf-0bbe8fadcbb6/ERR127302_2_subset.fastq.gz
=============================================
20000 sequences processed in total
Sequences removed because they became shorter than the length cutoff of 18 bp: 743 (3.7%)
Sequences removed because they contained more Ns than the cutoff of 2: 19 (0.1%)
Sequences removed because after trimming they were longer than the maximum length cutoff of 30 bp: 18663 (93.3%)

[job trim] completed success
[step trim] start
[job trim_2] Output of job will be cached in /home/welliton/lab/cwl_workflow_cache/cache/b109df1b0737531e68773a3168e2e2a5
[job trim_2] /home/welliton/lab/cwl_workflow_cache/cache/b109df1b0737531e68773a3168e2e2a5$ docker \
run \
-i \
--volume=/home/welliton/lab/cwl_workflow_cache/cache/b109df1b0737531e68773a3168e2e2a5:/var/spool/cwl:rw \
--volume=/tmp/tmpdBu8NU:/tmp:rw \
--volume=/home/welliton/lab/cwl_workflow_cache/cache/58d8a9f042a3f2bac76bc2c7b6e72bf9/ERR127302_1_subset.fastq.gz:/var/lib/cwl/stg21338929-448a-4d46-aed2-7a62952e0cef/ERR127302_1_subset.fastq.gz:ro \
--workdir=/var/spool/cwl \
--read-only=true \
--user=1000 \
--rm \
--env=TMPDIR=/tmp \
--env=HOME=/var/spool/cwl \
welliton/trimgalore:v0.4.4 \
trim_galore \
-e \
0.1 \
/var/lib/cwl/stg21338929-448a-4d46-aed2-7a62952e0cef/ERR127302_1_subset.fastq.gz \
--length \
18 \
--max_length \
30 \
--max_n \
2 \
--phred33 \
--quality \
20 \
--small_rna \
--stringency \
1
Maximum length cutoff set to >> 30 bp <<; sequences longer than this threshold will be removed (only advised for smallRNA-trimming!)

Path to Cutadapt set as: 'cutadapt' (default)
1.13
Cutadapt seems to be working fine (tested command 'cutadapt --version')
Writing report to 'ERR127302_1_subset.fastq.gz_trimming_report.txt'

SUMMARISING RUN PARAMETERS
==========================
Input filename: /var/lib/cwl/stg21338929-448a-4d46-aed2-7a62952e0cef/ERR127302_1_subset.fastq.gz
Trimming mode: single-end
Trim Galore version: 0.4.4
Cutadapt version: 1.13
Quality Phred score cutoff: 20
Quality encoding type selected: ASCII+33
Adapter sequence: 'TGGAATTCTCGG' (Illumina small RNA adapter; user defined)
Maximum trimming error rate: 0.1 (default)
Maximum number of tolerated Ns: 2
Minimum required adapter overlap (stringency): 1 bp
Minimum required sequence length before a sequence gets removed: 18 bp
Maxiumum tolerated read length after trimming (for smallRNA trimming): 30 bp
Output file(s) will be GZIP compressed

Writing final adapter and quality trimmed output to ERR127302_1_subset_trimmed.fq.gz

>>> Now performing quality (cutoff 20) and adapter trimming in a single pass for the adapter sequence: 'TGGAATTCTCGG' from file /var/lib/cwl/stg21338929-448a-4d46-aed2-7a62952e0cef/ERR127302_1_subset.fastq.gz <<<
This is cutadapt 1.13 with Python 2.7.9
Command line parameters: -f fastq -e 0.1 -q 20 -O 1 -a TGGAATTCTCGG /var/lib/cwl/stg21338929-448a-4d46-aed2-7a62952e0cef/ERR127302_1_subset.fastq.gz
Trimming 1 adapter with at most 10.0% errors in single-end mode ...
Finished in 0.40 s (20 us/read; 2.98 M reads/minute).

=== Summary ===

Total reads processed: 20,000
Reads with adapters: 6,328 (31.6%)
Reads written (passing filters): 20,000 (100.0%)

Total basepairs processed: 1,440,000 bp
Quality-trimmed: 88,502 bp (6.1%)
Total written (filtered): 1,341,840 bp (93.2%)

=== Adapter 1 ===

Sequence: TGGAATTCTCGG; Type: regular 3'; Length: 12; Trimmed: 6328 times.

No. of allowed errors:
0-9 bp: 0; 10-12 bp: 1

Bases preceding removed adapters:
A: 20.0%
C: 31.0%
G: 24.7%
T: 24.3%
none/other: 0.0%

Overview of removed sequences
length count expect max.err error counts
1 4069 5000.0 0 4069
2 1513 1250.0 0 1513
3 624 312.5 0 624
4 86 78.1 0 86
5 23 19.5 0 23
6 7 4.9 0 7
7 2 1.2 0 2
15 1 0.0 1 0 1
38 1 0.0 1 0 1
52 1 0.0 1 0 1
71 1 0.0 1 0 1

RUN STATISTICS FOR INPUT FILE: /var/lib/cwl/stg21338929-448a-4d46-aed2-7a62952e0cef/ERR127302_1_subset.fastq.gz
=============================================
20000 sequences processed in total
Sequences removed because they became shorter than the length cutoff of 18 bp: 52 (0.3%)
Sequences removed because they contained more Ns than the cutoff of 2: 3 (0.0%)
Sequences removed because after trimming they were longer than the maximum length cutoff of 30 bp: 19497 (97.5%)

[job trim_2] completed success
[step trim] completed success
[step qa_trim] start
[job qa_trim] Output of job will be cached in /home/welliton/lab/cwl_workflow_cache/cache/75e0ab77de27f9757e5ce03a5594b503
[job qa_trim] /home/welliton/lab/cwl_workflow_cache/cache/75e0ab77de27f9757e5ce03a5594b503$ docker \
run \
-i \
--volume=/home/welliton/lab/cwl_workflow_cache/cache/75e0ab77de27f9757e5ce03a5594b503:/var/spool/cwl:rw \
--volume=/tmp/tmpXbWbhu:/tmp:rw \
--volume=/home/welliton/lab/cwl_workflow_cache/cache/b109df1b0737531e68773a3168e2e2a5/ERR127302_1_subset_trimmed.fq.gz:/var/lib/cwl/stg5247157f-11b7-4d16-9e8b-954a2e82aa90/ERR127302_1_subset_trimmed.fq.gz:ro \
--volume=/home/welliton/lab/cwl_workflow_cache/cache/694cbde9f5690336d246b18f673236d4/ERR127302_2_subset_trimmed.fq.gz:/var/lib/cwl/stgdd0af7f0-fca3-48c2-b92c-703fc9765d6f/ERR127302_2_subset_trimmed.fq.gz:ro \
--workdir=/var/spool/cwl \
--read-only=true \
--user=1000 \
--rm \
--env=TMPDIR=/tmp \
--env=HOME=/var/spool/cwl \
welliton/rqc \
Rqc \
--files \
/var/lib/cwl/stgdd0af7f0-fca3-48c2-b92c-703fc9765d6f/ERR127302_2_subset_trimmed.fq.gz,/var/lib/cwl/stg5247157f-11b7-4d16-9e8b-954a2e82aa90/ERR127302_1_subset_trimmed.fq.gz \
--groups \
None,None \
--sample \
--reads \
1000000 \
--workers \
2 \
--report_file \
qa_report_trim \
--rds_file \
rqc.rds
[1] "/var/spool/cwl/qa_report_trim.html"
Warning messages:
1: In exists(sym, ns, inherits = FALSE) :
closing unused connection 4 (/var/lib/cwl/stg5247157f-11b7-4d16-9e8b-954a2e82aa90/ERR127302_1_subset_trimmed.fq.gz)
2: In exists(sym, ns, inherits = FALSE) :
closing unused connection 3 (/var/lib/cwl/stgdd0af7f0-fca3-48c2-b92c-703fc9765d6f/ERR127302_2_subset_trimmed.fq.gz)
[job qa_trim] completed success
[step qa_trim] completed success
[workflow workflow_cache.cwl] completed success
Final process status is success
```

Second.

```
/usr/local/bin/cwl-runner 1.0.20170717120410
Resolved 'workflow_cache.cwl' to 'file:///home/welliton/lab/cwl_workflow_cache/workflow_cache.cwl'
/usr/local/lib/python2.7/dist-packages/urllib3/util/ssl_.py:335: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
SNIMissingWarning
/usr/local/lib/python2.7/dist-packages/urllib3/util/ssl_.py:133: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
InsecurePlatformWarning
[workflow workflow_cache.cwl] start
[step retrieve] start
[job retrieve] Using cached output in /home/welliton/lab/cwl_workflow_cache/cache/58d8a9f042a3f2bac76bc2c7b6e72bf9
[step retrieve] completed success
[step qa_raw] start
[job qa_raw] Using cached output in /home/welliton/lab/cwl_workflow_cache/cache/3e1def7ec528ae55597d12d7db3fcfa4
[step qa_raw] completed success
[step trim] start
[job trim] Using cached output in /home/welliton/lab/cwl_workflow_cache/cache/694cbde9f5690336d246b18f673236d4
[step trim] start
[job trim_2] Output of job will be cached in /home/welliton/lab/cwl_workflow_cache/cache/0fadca98c79fc96a7c534450b4e54e8e
[job trim_2] /home/welliton/lab/cwl_workflow_cache/cache/0fadca98c79fc96a7c534450b4e54e8e$ docker \
run \
-i \
--volume=/home/welliton/lab/cwl_workflow_cache/cache/0fadca98c79fc96a7c534450b4e54e8e:/var/spool/cwl:rw \
--volume=/tmp/tmpgDlHQG:/tmp:rw \
--volume=/home/welliton/lab/cwl_workflow_cache/cache/58d8a9f042a3f2bac76bc2c7b6e72bf9/ERR127302_1_subset.fastq.gz:/var/lib/cwl/stgf676c125-0535-4549-b1ea-40a69b5cba7b/ERR127302_1_subset.fastq.gz:ro \
--workdir=/var/spool/cwl \
--read-only=true \
--user=1000 \
--rm \
--env=TMPDIR=/tmp \
--env=HOME=/var/spool/cwl \
welliton/trimgalore:v0.4.4 \
trim_galore \
-e \
0.1 \
/var/lib/cwl/stgf676c125-0535-4549-b1ea-40a69b5cba7b/ERR127302_1_subset.fastq.gz \
--length \
18 \
--max_length \
30 \
--max_n \
2 \
--phred33 \
--quality \
20 \
--small_rna \
--stringency \
1
Maximum length cutoff set to >> 30 bp <<; sequences longer than this threshold will be removed (only advised for smallRNA-trimming!)

Path to Cutadapt set as: 'cutadapt' (default)
1.13
Cutadapt seems to be working fine (tested command 'cutadapt --version')
Writing report to 'ERR127302_1_subset.fastq.gz_trimming_report.txt'

SUMMARISING RUN PARAMETERS
==========================
Input filename: /var/lib/cwl/stgf676c125-0535-4549-b1ea-40a69b5cba7b/ERR127302_1_subset.fastq.gz
Trimming mode: single-end
Trim Galore version: 0.4.4
Cutadapt version: 1.13
Quality Phred score cutoff: 20
Quality encoding type selected: ASCII+33
Adapter sequence: 'TGGAATTCTCGG' (Illumina small RNA adapter; user defined)
Maximum trimming error rate: 0.1 (default)
Maximum number of tolerated Ns: 2
Minimum required adapter overlap (stringency): 1 bp
Minimum required sequence length before a sequence gets removed: 18 bp
Maxiumum tolerated read length after trimming (for smallRNA trimming): 30 bp
Output file(s) will be GZIP compressed

Writing final adapter and quality trimmed output to ERR127302_1_subset_trimmed.fq.gz

>>> Now performing quality (cutoff 20) and adapter trimming in a single pass for the adapter sequence: 'TGGAATTCTCGG' from file /var/lib/cwl/stgf676c125-0535-4549-b1ea-40a69b5cba7b/ERR127302_1_subset.fastq.gz <<<
This is cutadapt 1.13 with Python 2.7.9
Command line parameters: -f fastq -e 0.1 -q 20 -O 1 -a TGGAATTCTCGG /var/lib/cwl/stgf676c125-0535-4549-b1ea-40a69b5cba7b/ERR127302_1_subset.fastq.gz
Trimming 1 adapter with at most 10.0% errors in single-end mode ...
Finished in 0.39 s (20 us/read; 3.06 M reads/minute).

=== Summary ===

Total reads processed: 20,000
Reads with adapters: 6,328 (31.6%)
Reads written (passing filters): 20,000 (100.0%)

Total basepairs processed: 1,440,000 bp
Quality-trimmed: 88,502 bp (6.1%)
Total written (filtered): 1,341,840 bp (93.2%)

=== Adapter 1 ===

Sequence: TGGAATTCTCGG; Type: regular 3'; Length: 12; Trimmed: 6328 times.

No. of allowed errors:
0-9 bp: 0; 10-12 bp: 1

Bases preceding removed adapters:
A: 20.0%
C: 31.0%
G: 24.7%
T: 24.3%
none/other: 0.0%

Overview of removed sequences
length count expect max.err error counts
1 4069 5000.0 0 4069
2 1513 1250.0 0 1513
3 624 312.5 0 624
4 86 78.1 0 86
5 23 19.5 0 23
6 7 4.9 0 7
7 2 1.2 0 2
15 1 0.0 1 0 1
38 1 0.0 1 0 1
52 1 0.0 1 0 1
71 1 0.0 1 0 1

RUN STATISTICS FOR INPUT FILE: /var/lib/cwl/stgf676c125-0535-4549-b1ea-40a69b5cba7b/ERR127302_1_subset.fastq.gz
=============================================
20000 sequences processed in total
Sequences removed because they became shorter than the length cutoff of 18 bp: 52 (0.3%)
Sequences removed because they contained more Ns than the cutoff of 2: 3 (0.0%)
Sequences removed because after trimming they were longer than the maximum length cutoff of 30 bp: 19497 (97.5%)

[job trim_2] completed success
[step trim] completed success
[step qa_trim] start
[job qa_trim] Output of job will be cached in /home/welliton/lab/cwl_workflow_cache/cache/2e72b2c3dd00931c952f6bf5986a0ab8
[job qa_trim] /home/welliton/lab/cwl_workflow_cache/cache/2e72b2c3dd00931c952f6bf5986a0ab8$ docker \
run \
-i \
--volume=/home/welliton/lab/cwl_workflow_cache/cache/2e72b2c3dd00931c952f6bf5986a0ab8:/var/spool/cwl:rw \
--volume=/tmp/tmpyrrf77:/tmp:rw \
--volume=/home/welliton/lab/cwl_workflow_cache/cache/0fadca98c79fc96a7c534450b4e54e8e/ERR127302_1_subset_trimmed.fq.gz:/var/lib/cwl/stgd7174b07-cbea-4f36-9cdb-0cd487b6ecea/ERR127302_1_subset_trimmed.fq.gz:ro \
--volume=/home/welliton/lab/cwl_workflow_cache/cache/694cbde9f5690336d246b18f673236d4/ERR127302_2_subset_trimmed.fq.gz:/var/lib/cwl/stg8dbc3fad-bba7-4789-8bb0-044b9514a50f/ERR127302_2_subset_trimmed.fq.gz:ro \
--workdir=/var/spool/cwl \
--read-only=true \
--user=1000 \
--rm \
--env=TMPDIR=/tmp \
--env=HOME=/var/spool/cwl \
welliton/rqc \
Rqc \
--files \
/var/lib/cwl/stg8dbc3fad-bba7-4789-8bb0-044b9514a50f/ERR127302_2_subset_trimmed.fq.gz,/var/lib/cwl/stgd7174b07-cbea-4f36-9cdb-0cd487b6ecea/ERR127302_1_subset_trimmed.fq.gz \
--groups \
None,None \
--sample \
--reads \
1000000 \
--workers \
2 \
--report_file \
qa_report_trim \
--rds_file \
rqc.rds
[1] "/var/spool/cwl/qa_report_trim.html"
Warning messages:
1: In exists(sym, ns, inherits = FALSE) :
closing unused connection 4 (/var/lib/cwl/stgd7174b07-cbea-4f36-9cdb-0cd487b6ecea/ERR127302_1_subset_trimmed.fq.gz)
2: In exists(sym, ns, inherits = FALSE) :
closing unused connection 3 (/var/lib/cwl/stg8dbc3fad-bba7-4789-8bb0-044b9514a50f/ERR127302_2_subset_trimmed.fq.gz)
[job qa_trim] completed success
[step qa_trim] completed success
[workflow workflow_cache.cwl] completed success
Final process status is success
```

Third.

```
/usr/local/bin/cwl-runner 1.0.20170717120410
Resolved 'workflow_cache.cwl' to 'file:///home/welliton/lab/cwl_workflow_cache/workflow_cache.cwl'
/usr/local/lib/python2.7/dist-packages/urllib3/util/ssl_.py:335: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
SNIMissingWarning
/usr/local/lib/python2.7/dist-packages/urllib3/util/ssl_.py:133: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
InsecurePlatformWarning
[workflow workflow_cache.cwl] start
[step retrieve] start
[job retrieve] Using cached output in /home/welliton/lab/cwl_workflow_cache/cache/58d8a9f042a3f2bac76bc2c7b6e72bf9
[step retrieve] completed success
[step qa_raw] start
[job qa_raw] Using cached output in /home/welliton/lab/cwl_workflow_cache/cache/3e1def7ec528ae55597d12d7db3fcfa4
[step qa_raw] completed success
[step trim] start
[job trim] Using cached output in /home/welliton/lab/cwl_workflow_cache/cache/694cbde9f5690336d246b18f673236d4
[step trim] start
[job trim_2] Using cached output in /home/welliton/lab/cwl_workflow_cache/cache/0fadca98c79fc96a7c534450b4e54e8e
[step trim] completed success
[step qa_trim] start
[job qa_trim] Using cached output in /home/welliton/lab/cwl_workflow_cache/cache/2e72b2c3dd00931c952f6bf5986a0ab8
[step qa_trim] completed success
[workflow workflow_cache.cwl] completed success
Final process status is success
```