Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

RF: Move to fit/apply workflow #360

Merged
merged 142 commits into from
Aug 1, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
142 commits
Select commit Hold shift + click to select a range
97aef93
MAINT: Update dependencies, move to ruff
mgxd Mar 27, 2024
89c3198
MAINT: Update dependencies
mgxd Mar 28, 2024
0024278
CHORE: Add Thomas to contributors
mgxd Mar 28, 2024
d7f8866
RF[WIP]: Start fit-apply
mgxd Mar 28, 2024
4c50c8d
RF: Clean up segmentation workflow
mgxd Apr 2, 2024
44f02dd
STY: Single connect formatting, add log message
mgxd Apr 3, 2024
fbdd679
ENH[WIP]: More fitting
mgxd Apr 3, 2024
1a147de
ENH: Add segmentation fit step
mgxd Apr 3, 2024
7ea329a
ENH[WIP]: Add template normalization, imports, type annots
mgxd Apr 4, 2024
c1e19ed
ENH: Add surface recon logic, move towards single anatomical reference
mgxd Apr 10, 2024
a0dbf7c
ENH: Add mask refinement, rework T1w mask logic
mgxd Apr 10, 2024
9138398
ENH: Handle surfaces/morphometrics/fsLR registration
mgxd Apr 11, 2024
96b3f8a
ENH: Simplify mcribs/infantfs workflows
mgxd Apr 12, 2024
f14e816
ENH: Add symbolic copy of sphere.reg to make FS integration easier
mgxd Apr 15, 2024
0e2c2a2
FIX: Simplify dhcp reg workflow, remove crud
mgxd Apr 15, 2024
b93349e
FIX: Insert new parameters, update mcribs -> fslr workflow
mgxd Apr 15, 2024
9452907
RF: Contrast -> image type
mgxd Apr 15, 2024
f3e4204
RF: Move fit to anatomical workflows module
mgxd Apr 15, 2024
bd851ce
FIX: T2w brainmask, move precomputed to dict
mgxd Apr 18, 2024
502cca2
ENH: Add top-level anat workflow
mgxd Apr 18, 2024
57f6053
RF: Drop derivatives class for dict approach
mgxd Apr 18, 2024
b6ca0ab
ADD: Expected anatomical files
mgxd Apr 18, 2024
cdbfbd2
RF: Make midthickness creation standalone workflow
mgxd Apr 19, 2024
c8b8f24
FIX: Anat IO, brain extraction inputs
mgxd Apr 19, 2024
1eaf43e
RF: Brain extraction parameters, typing
mgxd Apr 19, 2024
3168621
RF: Branch fsLR resampling if using mcribs recon
mgxd Apr 23, 2024
9b455af
ADD: Midthickness resampling workflow for MCRIBS
mgxd Apr 23, 2024
5ac10ed
RF: Determine reference anatomical prior to fit workflow
mgxd Apr 23, 2024
8f56476
ENH: Add fingerprint file to quickly determine MCRIB runs
mgxd Apr 24, 2024
161e66a
ENH: Beef up anatomical summary
mgxd Apr 24, 2024
034620b
ENH: Add new parser options
mgxd Apr 25, 2024
5cb53e0
ENH: Reflect new options within configuration
mgxd Apr 25, 2024
d67ad10
FIX: Add session label when fetching derivatives
mgxd Apr 25, 2024
224f6a1
FIX: Proper typing, parameters
mgxd Apr 25, 2024
cbc4cb3
ENH: Add anatomical processing
mgxd Apr 25, 2024
a3d44b9
PIN: smriprep working branch
mgxd Apr 25, 2024
fb7255c
CHORE: Update build requirements
mgxd Apr 25, 2024
6fc8509
DOCKER: Update micromamba step
mgxd Apr 25, 2024
23a0d24
FIX: Traits version
mgxd Apr 25, 2024
a94f627
FIX: Determine which surface recon method to use, by default
mgxd Apr 25, 2024
f2e685e
PIN: niworkflows branch
mgxd Apr 25, 2024
8e0becf
FIX: Remove eager imports
mgxd Apr 25, 2024
d5066aa
FIX: Connections, missed parameters
mgxd Apr 25, 2024
665309a
FIX: Use template iterator in anat reports
mgxd Apr 25, 2024
fcf201f
ENH: output both forward and reverse coreg transforms
mgxd Apr 25, 2024
7b0098a
FIX: fstring + quotes
mgxd Apr 26, 2024
0fc5d04
ENH: Add named derivatives support
mgxd Apr 26, 2024
40eb1de
FIX: Fetch other anatomical files
mgxd Apr 26, 2024
27e5c90
more workflow fixes
mgxd Apr 26, 2024
c74ca50
DOCKER: Debug build
mgxd Apr 26, 2024
f717888
DKR: Add step to install traits, /src/nibabies->/src
mgxd Apr 28, 2024
dcfcfcd
MAINT: Update new --patch path
mgxd Apr 28, 2024
3483498
FIX: Connections
mgxd Apr 28, 2024
cbc1826
ENH: Add pdb debug mode
mgxd Apr 29, 2024
567e007
FIX: Misfired connections
mgxd Apr 29, 2024
cb13b5b
FIX: aaand more missing/incorrect connections
mgxd Apr 29, 2024
32d9be5
RF: Add "apply" workflow to facilitate fit caching
mgxd Apr 29, 2024
5e1f064
FIX: Remove mcribs templates hack now that they are on templateflow
mgxd May 1, 2024
58a9c10
FIX: Ensure midthickness is saved to subjects_dir
mgxd May 1, 2024
c5d6365
ENH: Clean up dual anat workflow, port logic to single anat
mgxd May 1, 2024
2d94ad1
ENH: Finalize single anat workflow
mgxd May 1, 2024
1a0be93
RF: Specify T1w/T2w space in derivatives
mgxd May 2, 2024
2ec91e7
FIX: Specify space on outputs that may be in T1w/T2w space
mgxd May 2, 2024
bdcd452
DKR: Cache new templates
mgxd May 2, 2024
67feb16
FIX: Handle case of multiple T1w files
mgxd May 2, 2024
4e2b238
FIX: Catch errors with basic, no derivatives, processing
mgxd May 2, 2024
8d40176
FIX: Add entrypoint for single anat workflow, prioritize recon_method…
mgxd May 2, 2024
c10d842
FIX: Remove superfluous denoise
mgxd May 2, 2024
1c4d3c8
FIX: Single anat problems (T2-only)
mgxd May 6, 2024
2b04820
FIX: Port bold2anat feature into BOLD workflow
mgxd May 7, 2024
0c2fce3
FIX: Handle some derivatives but not preproc
mgxd May 10, 2024
69fa824
WIP: BOLD registration fit/apply
mgxd May 9, 2024
4d8af2f
DOCS: Start outlining outputs
mgxd May 9, 2024
e8153c7
DOCS: Flesh out outputs from fmriprep
mgxd May 15, 2024
aedfca9
FIX: Actually fetch templates
mgxd May 15, 2024
a26470e
WIP: Port over bold processing helpers
mgxd May 15, 2024
2778b41
WIP: Functional integration
mgxd May 15, 2024
1743edf
WIP/STY: Ruff bold workflows
mgxd May 15, 2024
feaaea7
WIP: functional - anatomical coregistration
mgxd May 15, 2024
03b9f27
WIP: Modernize resampling
mgxd May 16, 2024
65ffaac
WIP: Update BOLD to use reference anatomical
mgxd May 16, 2024
ddde0ba
RF: Consider changing FAST inputs based on T1w/T2w image
mgxd May 16, 2024
23a4184
ENH: Add BIDSURI interface (should move to niworkflows)
mgxd May 16, 2024
3bb6727
WIP: Port updated confounds workflow
mgxd May 16, 2024
81665b8
FIX: T1w->anat inputs
mgxd May 16, 2024
6454ac9
WIP: Add apply workflow
mgxd May 16, 2024
bd34f11
WIP: Add resampling tools
mgxd May 16, 2024
cc71158
WIP: HMC workflow
mgxd May 16, 2024
5f2ba33
WIP: Update confound interfaces
mgxd May 16, 2024
c581805
ENH: Add functional derivatives crawler
mgxd May 16, 2024
7726042
FIX: Functional workflow connections
mgxd May 16, 2024
62228f5
ENH: Alternative boldref workflow
mgxd May 16, 2024
ee35efe
FIX: Imports, default dismiss entities
mgxd May 16, 2024
38330d9
FIX: fmriprep->nibabies
mgxd May 16, 2024
7780e82
ENH: Modernize config
mgxd May 16, 2024
a0229ac
FIX: Update FunctionalSummary with new bold2anat_init options
mgxd May 16, 2024
2b30bb6
ENH: Add Sources
mgxd May 16, 2024
5e04b3f
RF: more fmriprep->nibabies
mgxd May 16, 2024
073b705
FIX: Minimal
mgxd May 17, 2024
aa7670f
ENH: Drop early BOLD frames from boldref calculation
mgxd May 17, 2024
e7838ca
FIX: Avoid carpetplot unless CIFTI outputs
mgxd May 24, 2024
821a0a7
RF: Move anatomical apply into separate module
mgxd May 24, 2024
68c6dfb
FIX: Update base workflows to reflect previous changes
mgxd May 24, 2024
d9c6c1c
FIX: Reverse size params, remove pesky warning
mgxd May 30, 2024
2e7e583
DKR: Cache MNI templates to avoid cache invalidation
mgxd May 30, 2024
57a0b73
ENH: Re-add subcortical ROI alignment for dtseries output
mgxd May 30, 2024
2dfc73f
RF: Split segmentation datasink into single, reusable wf
mgxd Jun 3, 2024
0dc07d6
FIX: Segmentation indexes
mgxd Jun 3, 2024
392f6f0
RF: Func io spec
mgxd Jun 3, 2024
c473a1c
DOC: Fix reporting
mgxd Jun 3, 2024
7a9b850
RF: Do not connect from `ds_` node to avoid cache invalidation
mgxd Jun 3, 2024
4eb6520
FIX: Move select node out of loop
mgxd Jun 4, 2024
604179f
RF: Reorganize warp transpose
mgxd Jun 4, 2024
36b5562
RM: Template files now available in templateflow
mgxd Jun 4, 2024
01368c1
CI/STY: Fix spelling errors, update style check
mgxd Jun 4, 2024
997b9b4
FIX: Slicetiming
mgxd Jun 7, 2024
e557609
FIX: Ensure libitk matches ANTs expectation
mgxd Jun 10, 2024
ff3f815
chore: bump niworkflows to master
mgxd Jun 10, 2024
4da4a3d
FIX: Pass anat2std xfm to BOLD workflow
mgxd Jul 10, 2024
4885bc5
ADD: Option to ensure output directory is fully populated
mgxd Jul 11, 2024
15327e1
ENH: Bump patched MCRIBS
mgxd Jul 11, 2024
f086706
FIX: Update FS->MCRIBS label mappings
mgxd Jul 11, 2024
d7abbef
FIX: Do not use FAST if precomputed aseg is available
mgxd Jul 15, 2024
fb3345f
FIX: Single anat template name
mgxd Jul 18, 2024
5ad69eb
FIX: Template anatomical reporting
mgxd Jul 18, 2024
ae53389
FIX: Boilerplate file as string
mgxd Jul 18, 2024
69d3e6e
ENH/FIX: Make BOLD HMC start frame configurable, do not clip BOLD tim…
mgxd Jul 22, 2024
1ecd1f1
FIX: Stringify output directory
mgxd Jul 24, 2024
c480f78
MAINT: Bump preps to pre-release master branches
mgxd Jul 29, 2024
1b6070e
FIX: Remove libitk hack
mgxd Jul 29, 2024
bf7c009
MAINT: Update dependency pins
mgxd Jul 30, 2024
d861911
CHORE: Remove crud
mgxd Jul 30, 2024
6821e4b
Remove ICA AROMA
mgxd Jul 30, 2024
6bd037c
TST: Update config test
mgxd Jul 31, 2024
bee1558
FIX: Clean up pyproject
mgxd Jul 31, 2024
7e3a895
FIX: Data intake
mgxd Jul 31, 2024
a663533
TST: Add acres, test derivatives intake
mgxd Jul 31, 2024
8ebd45b
TST: Create new derivatives test
mgxd Jul 31, 2024
edbd316
MAINT: Exclude non-source files from checks
mgxd Jul 31, 2024
1b6b537
STY: Mass ruff-ication
mgxd Jul 31, 2024
bcb6b0b
chore: pin preps to latest releases
mgxd Aug 1, 2024
3881973
FIX: Revert wrapper to PY2 compatibility
mgxd Aug 1, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions .codespellrc
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
[codespell]
skip = .git,*.pdf,*.svg,*.html,dataset_description.json,*.bib
# te - TE
# Weill - name
# reson - Reson. abbreviation in citation
ignore-words-list = te,weill,reson
39 changes: 18 additions & 21 deletions .github/workflows/style.yml
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
name: Style check
name: Style + spelling check

on:
push:
Expand All @@ -11,30 +11,27 @@ defaults:
run:
shell: bash

concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true

permissions:
contents: read

jobs:
build:
if: "!contains(github.event.head_commit.message, '[skip ci]')"
style:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- run: pipx run ruff check .
- run: pipx run ruff format --diff .

codespell:
name: Check for spelling errors
runs-on: ubuntu-latest
strategy:
matrix:
python-version: ['3.11']

steps:
- uses: actions/checkout@v4
with:
fetch-depth: 1 # Only fetch the latest commit
- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v5
with:
python-version: ${{ matrix.python-version }}
- name: Install dependencies
run: |
pip install black isort
- name: black code formatting check
run: |
black --check nibabies/
- name: isort formatting check
run: isort nibabies/ --check-only
- name: Checkout
uses: actions/checkout@v4
- name: Codespell
uses: codespell-project/actions-codespell@v2
5 changes: 5 additions & 0 deletions .maint/contributors.json
Original file line number Diff line number Diff line change
@@ -1,4 +1,9 @@
[
{
"affiliation": "Department of Pediatrics, University of Minnesota, MN, USA",
"name": "Madison, Thomas",
"orcid": "0000-0003-3030-6580"
},
{
"affiliation": "Montreal Neurological Institute, McGill University",
"name": "Huberty, Scott",
Expand Down
30 changes: 15 additions & 15 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
@@ -1,18 +1,18 @@
exclude: ".*/data/.*"
repos:
- repo: https://github.com/pre-commit/pre-commit-hooks
rev: v2.4.0
- repo: https://github.com/pre-commit/pre-commit-hooks
rev: v4.4.0
hooks:
- id: trailing-whitespace
- id: end-of-file-fixer
- id: check-yaml
- id: check-added-large-files
- repo: https://github.com/psf/black
rev: 22.3.0
- id: trailing-whitespace
exclude: '.*\.svg'
- id: end-of-file-fixer
exclude: '.*\.svg'
- id: check-yaml
- id: check-json
- id: check-toml
- id: check-added-large-files
- repo: https://github.com/astral-sh/ruff-pre-commit
rev: v0.2.0
hooks:
- id: black
files: ^nibabies/
- repo: https://github.com/pycqa/isort
rev: 5.10.1
hooks:
- id: isort
files: ^nibabies/
- id: ruff
- id: ruff-format
4 changes: 3 additions & 1 deletion .readthedocs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ version: 2
build:
os: ubuntu-22.04
tools:
python: "3.10"
python: "3.11"
jobs:
post_checkout:
- git fetch --unshallow
Expand All @@ -15,3 +15,5 @@ python:
path: .
extra_requirements:
- doc
- method: pip
path: wrapper/
4 changes: 2 additions & 2 deletions CHANGES.md
Original file line number Diff line number Diff line change
Expand Up @@ -36,9 +36,9 @@ Previous, *NiBabies* expected input from the `--derivatives` flag to be in T1w s
* DKR: Add M-CRIB-S to Docker container (#283)
* DKR: Update dependencies, split into multi-stage build
* ENH: Add option to exclude projecting high variance voxels to surface (#278)
* ENH: Resample morphometrics to fsLR CIFTI-2 files when outputing CIFTIs (#279)
* ENH: Resample morphometrics to fsLR CIFTI-2 files when outputting CIFTIs (#279)
* ENH: Add MCRIBReconAll as alternative surface reconstruction method (#283)
* ENH: Reorder anatomical processsing, run ANTs DenoiseImage on anatomicals (#286)
* ENH: Reorder anatomical processing, run ANTs DenoiseImage on anatomicals (#286)
* ENH: Extract participant ages from BIDS sources, deprecate `--age-months` (#287)
* ENH: Dilate BOLD mask by 2 voxels to prevent over-aggressive masking degrading T2star map estimation (#296)
* ENH: Allow precomputed derivatives in T1w or T2w space (#305)
Expand Down
27 changes: 17 additions & 10 deletions Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -23,15 +23,15 @@
# SOFTWARE.

# Ubuntu 22.04 LTS - Jammy
ARG BASE_IMAGE=ubuntu:jammy-20230605
ARG BASE_IMAGE=ubuntu:jammy-20240405

# NiBabies wheel
FROM python:slim AS src
RUN pip install build
RUN apt-get update && \
apt-get install -y --no-install-recommends git
COPY . /src/nibabies
RUN python -m build /src/nibabies
COPY . /src
RUN python -m build /src

# Older Python to support legacy MCRIBS
FROM python:3.6.15-slim as pyenv
Expand Down Expand Up @@ -90,19 +90,26 @@ RUN mkdir /opt/workbench && \

# Micromamba
FROM downloader as micromamba

# Install a C compiler to build extensions when needed.
# traits<6.4 wheels are not available for Python 3.11+, but build easily.
RUN apt-get update && \
apt-get install -y --no-install-recommends build-essential && \
apt-get clean && rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/*

WORKDIR /
# Bump the date to current to force update micromamba
RUN echo "2023.06.29"
RUN curl -Ls https://micro.mamba.pm/api/micromamba/linux-64/latest | tar -xvj bin/micromamba
RUN echo "2024.04.25" && curl -Ls https://micro.mamba.pm/api/micromamba/linux-64/latest | tar -xvj bin/micromamba
ENV MAMBA_ROOT_PREFIX="/opt/conda"
COPY env.yml /tmp/env.yml
COPY requirements.txt /tmp/requirements.txt
WORKDIR /tmp
RUN micromamba create -y -f /tmp/env.yml && \
micromamba clean -y -a

ENV PATH="/opt/conda/envs/nibabies/bin:$PATH"
RUN /opt/conda/envs/nibabies/bin/npm install -g svgo@^2.8 bids-validator@1.11.0 && \
RUN npm install -g svgo@^3.2.0 bids-validator@^1.14.0 && \
rm -r ~/.npm
COPY requirements.txt /tmp/requirements.txt
RUN /opt/conda/envs/nibabies/bin/pip install --no-cache-dir -r /tmp/requirements.txt

# Main container
FROM ${BASE_IMAGE} as nibabies
Expand Down Expand Up @@ -207,7 +214,7 @@ ENV PERL5LIB="$MINC_LIB_DIR/perl5/5.8.5" \
PATH="$FREESURFER_HOME/bin:$FREESURFER_HOME/tktools:$MINC_BIN_DIR:$PATH"

# MCRIBS (required legacy python)
COPY --from=nipreps/mcribs@sha256:6c7a8dedd61d0ead8c7c4a57ab158928c1c1d787d87dae33ab7ee43226fb1e0f /opt/MCRIBS/ /opt/MCRIBS
COPY --from=nipreps/mcribs@sha256:d5ea7616402cbc0b80f32634832dce540a32bb1d5e1e61dfdd2cc795dba8944a /opt/MCRIBS/ /opt/MCRIBS
COPY --from=pyenv /usr/local/lib/ /usr/local/lib/
ENV PATH="/opt/MCRIBS/bin:/opt/MCRIBS/MIRTK/MIRTK-install/bin:/opt/MCRIBS/MIRTK/MIRTK-install/lib/tools:${PATH}" \
LD_LIBRARY_PATH="/opt/MCRIBS/lib:/opt/MCRIBS/ITK/ITK-install/lib:/opt/MCRIBS/VTK/VTK-install/lib:/opt/MCRIBS/MIRTK/MIRTK-install/lib:/usr/local/lib:${LD_LIBRARY_PATH}" \
Expand Down Expand Up @@ -257,7 +264,7 @@ RUN ${CONDA_PYTHON} -m pip install --no-cache-dir --upgrade templateflow && \
find $HOME/.cache/templateflow -type f -exec chmod go=u {} +

# Install pre-built wheel
COPY --from=src /src/nibabies/dist/*.whl .
COPY --from=src /src/dist/*.whl .
RUN ${CONDA_PYTHON} -m pip install --no-cache-dir $( ls *.whl )[telemetry,test]

# Facilitate Apptainer use
Expand Down
2 changes: 1 addition & 1 deletion docs/community.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
## NiPreps Community

Check out the [official NiPreps community page](https://www.nipreps.org/community/), where topics such as contributing, code of coduct, and licensing are outlined.
Check out the [official NiPreps community page](https://www.nipreps.org/community/), where topics such as contributing, code of conduct, and licensing are outlined.


## NiBabies Coding Style
Expand Down
Loading
Loading