Optimizing for Multiple cutadapt calls/ guidance on complex regex

Hi,

So our team is using combinatorial sequencing with multistep barcode appends. Thus our read analysis pipeline uses pipe and multiple cutadapt calls. Thus, somewhere in the source code a for loop is running each time cutadapt is called which is suboptimal for us. Below is a sample shell script that is being used:

cutadapt -m 24 --nextseq-trim=15 -e 0.15 \
--pair-filter=first \
--cores=$core --report=minimal --trim-n \
-g file:$ligation_barcode \
--rename="{id}_{r1.adapter_name}" \
--discard-untrimmed \
--interleaved \
$input_folder/$sample.R1.fastq.gz $input_folder/$sample.R2.fastq.gz 2>> $output_folder/report_cutadapt.txt | \
  cutadapt -e 0.15 \
--pair-filter=any \
--cores=$core --report=minimal --trim-n \
--rename='{id}_{r1.adapter_name}_{r1.match_sequence}' \
-g file:$N_RT \
--discard-untrimmed \
--interleaved \
-o $output_folder/$sample.tagged.R1.fastq.gz -p $output_folder/$sample.tagged.R2.fastq.gz - >> $output_folder/report_cutadapt.txt 2>&1

As you can see, we first call cutadapt to trim 'ligation' barcode (read adapter 1) and add it to sample id in fastq file, then take the output of the previous step and trim 'rt' barcode (read adapter 2) and redirect the final output to an output folder. We want to do away with the need to create this intermediate fastq and get the final output with a single for loop. Now, I can create a bit more complex regex for Adapter but we want to control the error handling for those two barcodes individually and not aggregately for e.g. if the max error on adapter 1 and 2 is 0.15, the found sequence need to be capped at 0.15 each and not cap at 0.3 cumulatively.

We usually run in these in multiple cores if that is important.

Start with the sample shell pipeline and the two sequential cutadapt calls described in the issue. Investigate whether both barcode operations can be represented in one invocation while preserving separate error limits, renaming, paired-read handling, and multi-core behavior. Done means the intermediate FASTQ is unnecessary and each barcode's error threshold remains independent.