seqkit rmdup killed

[JieWu2012]

Hi,

When I run seqkit rmdup "seqkit rmdup -j 20 -s my.fastq > my_rmdup.fastq"
The Seqkit got killed and no other error message is ouput. My fastq file is 35G, and I have 126G Mem in the server. When I run other data with smaller than 30G by setting threads to higher, it works. But for the big data, no matter how many threads I set, it didn't work.

[shenwei356]

Increasing the number of threads does not help.

Please paste the result of seqkit version and seqkit stats my.fastq.

[JieWu2012]

I realized if I don't set "-D". It can run without being killed. What do you think is the reason?

my seqkit is v2.8.2.
Here is the output of seqkit stats my.fastq

file      format  type     num_seqs        sum_len  min_len  avg_len  max_len
my.fastq  FASTQ   DNA   290,523,523  9,917,166,757       10     34.1      101