Fix critical bugs and integrate new features #184
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This pull request contains a bunch of improvements.
Critical bugs in PUMI were found and solved
Not explicitly specifying MultiProc could lead to resource problems
If the plugin argument was not explicitly specified, then the CLI argument parser did not pass the nipype runtime arguments to nipype, resulting in the usage of nearly all recourses.
Bug in bbr-argument
Another critical bug was in the parsing of the bbr argument, which lead to the situation, that bbr was never used, even though it should be used per default.
CLI help argument did not work
It was not possible to build our Dockerfile(s)
FSL 6.0.1 installation problems as well as conda related problems lead to the Dockerfile(s) build crashing.
These problems were fixed.
Requirement related problems
A couple of python package requirement related problems were fixed.
The pyproject.toml was missing a couple of requirements as well as the requirements.txt.
Could not install PUMI with Python 3.11
Now, it's possible to install PUMI with Python 3.11
Adapt RPN model for scikit-learn 1.3.2
We used scikit-learn 1.1.2, but this version won't work with upcoming Python versions.
Because of this reason, I upgraded from scikit-learn 1.1.2 to 1.3.2.
Unfortunately, the RPN model did not work with 1.3.2, a slight modification of the model needed to be done.
Some new features were also introduced
Tool versions are now also sinked
It's important to be able to identify which PUMI version was used to create the given derivatives, since pipelines might have changed over time. It's also interesting to know which tool version was used (e.g., FSL version).
Per default, these versions will now be sinked into the file called 'software_versions.txt' (for the currently existing pipelines).
cluster-pipeline-executor.sh
The new script allows you to run PUMI pipelines on the cluster efficiently. It creates one job per subject and is scalable across many cluster nodes, also due to a strategy that reduces the chance of network congestion as much as possible.
Some additional things that were done:
Rework CLI argument parser
Since the CLI argument parser code was not well readable, I reworked the code and added more documentation.
Update Python version in Docker image from 3.8 to 3.11
Update FSL version inside of Docker image from 6.0.1 to 6.0.5
Add missing scripts to pyproject.toml
Issues that were fixed:
close #182
close #181
close #180
close #178
close #177
close #176
close #175
close #170
close #156