Low-complexity sequence filtering

Description of feature

Hi,
I have been working with some new datasets, and I see lots of reads with lots of G, and I wonder if the low-complexity filter option from fastP could be useful to discard such reads. It might make the assemblies a bit better.
The data was created on a Novaseq.

Why do I ask this?
I was checking the multiqc output of a MAG run and I saw lots of reads with polyG tails.
I then reran MAG with the fastP option for polyG tail trimming :TRUE.

In the new multiqc output, I see that the number of reads with PolyG dropped from 21.5 million to 600K reads. But it is not all removed. That made me curious and I decided to run this on my samples

SEQ="GGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGG"

find . -name "*.fastq.gz" | parallel -j 16 "echo -n '{}: '; zcat {} | grep -F -c '$SEQ'" > poly_G_counts.txt

the output from that showed this:

./TEI373_run0_phix_removed.unmapped_2.fastq.gz: 221649
./TEI374_run0_phix_removed.unmapped_2.fastq.gz: 151894
./TEI221_run0_phix_removed.unmapped_2.fastq.gz: 123335
./TEI248_run0_phix_removed.unmapped_2.fastq.gz: 122574
./TEI390_run0_phix_removed.unmapped_2.fastq.gz: 121840
./TEI476_92_run0_phix_removed.unmapped_2.fastq.gz: 110508
etcetera...

Which is a lot more than 600.000 reads. So that did not add up.

When checking the top fastq file, I then see that I actually have many reads like this:

@LH00534:182:22VG73LT4:5:1101:21961:1490 2:N:0:TATCGCAC+ACACTAAG
GGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGAGGGGGGGT

@LH00534:182:22VG73LT4:5:1234:28595:1042 2:N:0:TNTCGCAC+ANACTAAG
GGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTT

so not a real polyG tail, but it does not have a lot of information.

I also check with some reads the quality scores and I get something like this:

GGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGTGGTTTGGTAATACGAGGTCGCCGAGGAGATGTT  
+  
IIIIII*9IIIIIII9IIIIIIIIIIIIIIIIIIII9IIIII9IIIIIIIIIIIIIIIIIIIIIIII9IIIIIIIIIIIII9IIIIIIIIIII99II*I*IIIII9II9I99*9***************9**9**99*99**9**9*9*9*

Which shows that the end of the read has low quality bases (e.g. phred score 24 or lower).

What is funky, is that it is mainly in the reverse reads files. The worst Forward dataset only had 888 reads with a lot of Gs.

So I am wondering how to deal with this issue.

I ran MAG with the default fastp settings, and I will first try increasing the minimum quality to 25. That should remove a lot of the bad tails, and maybe also remove reads with long stretches of Gs.

But I would be interested if it was possible to add the low-complexity option from fastP in the pipeline. That might also help, and the min quality could then possibly be lower than 25.

Start by tracing where the pipeline invokes fastp and how its filtering options are exposed; the issue does not name a file, test, or entry point. Compare the requested low-complexity behavior with fastp's documented option, then confirm what pipeline output or filtering result would demonstrate that the option is applied.