update fastp and fix linting and testing issues #5813
Conversation
|
Even our test data had this bug, I included a test that should fail. |
| @@ -25,19 +25,19 @@ | |||
| #set $in2_name = re.sub('[^\w\-\s]', '_', str("%s_%s" % ($single_paired.paired_input.name, "R2"))) + $ext | |||
| #set out1 = $output_paired_coll.forward | |||
| #set out2 = $output_paired_coll.reverse | |||
| ln -s '$in1' '$in1_name' && | |||
| ln -s '$in2' '$in2_name' && | |||
| ln -sf '$in1' '$in1_name' && | |||
There was a problem hiding this comment.
Is that the fix to your issue, did you need the force for resubmitted jobs ? That seems like it would need a framework level fix, it doesn't seem like something we should fix within tools.
There was a problem hiding this comment.
Right, just to be sure, resubmitted jobs are supposed to get a clean working dir, or?
Then the -f could only "help" with name collisions of the inputs (for which it might be better to fail).
There was a problem hiding this comment.
Nope, this was not the fix.
Resubmission from Galaxy gets a clean job_working_dir ... I think / hope / have seen it :) ... but job schedulers can also resubmit jobs. In this case we do not get a clean job dir I think.
Either way, this was not the fix, the update was the fix. The latest version is supposed to fail when the fastq files do not match up. I recommend updating this tool. For us, a few fastp jobs are blocking slots for days...
There was a problem hiding this comment.
Right, eu uses htcondor retry/resubmit
but job schedulers can also resubmit jobs. In this case we do not get a clean job dir I think.
most likely not.
Maybe it would be cleaner to add a rm -rf working/* to the script referred to by environment_setup_file? The environment_setup_file is sourced in the job script (galaxy_xyz.sh), i.e. it would run with every submission.
fixes: #5812
FOR CONTRIBUTOR: