Combine steps with pipes to reduce disk footprint and make the pipeline faster

[muffato]

Description of feature

The pipeline has a massive disk footprint (approx. 1.5 TB per Gbp of sequence) that is mostly caused by each step storing its output file, e.g. in the samtools sub-workflows.

We can often combine consecutive steps with by piping the output of one step into the next one, thus removing the need to write a (often large) file on disk. This not only reduces the disk footprint, it can also make the pipeline much faster, especially when the filesystem is being heavily used.

[priyanka-surana]

It would involve going to something like nf-core/modules#3310. In fact this would be my step 1.

[muffato]

I know you know the command already, but I just found it explained on the samtools website too: https://www.htslib.org/algorithms/duplicate.html#workflow

[muffato]

"samtools sormadup" is being implemented in #82 but to close this ticket we should review the whole pipeline.