Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

⚡ Improve Engine Performance and Implementation #578

Draft
wants to merge 121 commits into
base: develop
Choose a base branch
from

Commits on Mar 24, 2023

  1. 🔧 Use pyproject.toml for bdist_wheel configuration

    - Use `pyproject.toml` for `bdist_wheel` configuration
    shaneahmed committed Mar 24, 2023
    Configuration menu
    Copy the full SHA
    ef55e95 View commit details
    Browse the repository at this point in the history

Commits on Mar 31, 2023

  1. Configuration menu
    Copy the full SHA
    49a0624 View commit details
    Browse the repository at this point in the history
  2. ⚡ Improve Engines performance and implementation

    - Improve `Engines` performance and implementation
    shaneahmed committed Mar 31, 2023
    Configuration menu
    Copy the full SHA
    8ba6def View commit details
    Browse the repository at this point in the history

Commits on Apr 9, 2023

  1. Configuration menu
    Copy the full SHA
    5cbcfcf View commit details
    Browse the repository at this point in the history

Commits on Apr 28, 2023

  1. ♻️ Refactor engines_abc.py

    - Refactor engines_abc.py
    shaneahmed committed Apr 28, 2023
    Configuration menu
    Copy the full SHA
    fac1000 View commit details
    Browse the repository at this point in the history

Commits on May 5, 2023

  1. Configuration menu
    Copy the full SHA
    a72d9ba View commit details
    Browse the repository at this point in the history

Commits on May 12, 2023

  1. Configuration menu
    Copy the full SHA
    57ea44a View commit details
    Browse the repository at this point in the history

Commits on Jun 2, 2023

  1. Configuration menu
    Copy the full SHA
    6618161 View commit details
    Browse the repository at this point in the history

Commits on Jun 16, 2023

  1. Configuration menu
    Copy the full SHA
    6996764 View commit details
    Browse the repository at this point in the history

Commits on Jul 6, 2023

  1. Configuration menu
    Copy the full SHA
    3584f6c View commit details
    Browse the repository at this point in the history

Commits on Jul 12, 2023

  1. Configuration menu
    Copy the full SHA
    eada692 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    77f1992 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    a477d32 View commit details
    Browse the repository at this point in the history

Commits on Jul 14, 2023

  1. Configuration menu
    Copy the full SHA
    f3e33b9 View commit details
    Browse the repository at this point in the history

Commits on Jul 21, 2023

  1. Configuration menu
    Copy the full SHA
    7d35285 View commit details
    Browse the repository at this point in the history

Commits on Jul 22, 2023

  1. Configuration menu
    Copy the full SHA
    7bad284 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    36fd629 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    443141c View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    b9d8c38 View commit details
    Browse the repository at this point in the history
  5. Configuration menu
    Copy the full SHA
    e608f7b View commit details
    Browse the repository at this point in the history
  6. Configuration menu
    Copy the full SHA
    1d7f5c0 View commit details
    Browse the repository at this point in the history
  7. Configuration menu
    Copy the full SHA
    b956bf5 View commit details
    Browse the repository at this point in the history

Commits on Jul 27, 2023

  1. Configuration menu
    Copy the full SHA
    e163f38 View commit details
    Browse the repository at this point in the history

Commits on Jul 31, 2023

  1. Configuration menu
    Copy the full SHA
    6de8020 View commit details
    Browse the repository at this point in the history
  2. ♻️ Refactor base code from IOSegmentorConfig to ModelIOConfigABC (#…

    …618)
    
    - Moved all ioconfigs to a single file.
    - Used dataclass to define ioconfig.
    - Refactor base code from `IOSegmentorConfig` to `ModelIOConfigABC`
    - Use `ModelIOConfigABC` for `PatchPredictor` instead of `IOSegmentorConfig`
    
    ---------
    
    Signed-off-by: Shan E Ahmed Raza <[email protected]>
    Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
    Co-authored-by: John Pocock <[email protected]>
    Co-authored-by: Mark Eastwood <[email protected]>
    Co-authored-by: Mostafa Jahanifar <[email protected]>
    Co-authored-by: Adam Shephard <[email protected]>
    6 people authored Jul 31, 2023
    Configuration menu
    Copy the full SHA
    d49fd0b View commit details
    Browse the repository at this point in the history
  3. ♻️ Move Dataset Classes to dataset_abc.py (#637)

    - Move `WSIStreamDataset`, `PatchDatasetABC` and `WSIPatchDataset` to `dataset_abc.py`
    
    ---------
    
    Signed-off-by: Shan E Ahmed Raza <[email protected]>
    Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
    shaneahmed and pre-commit-ci[bot] authored Jul 31, 2023
    Configuration menu
    Copy the full SHA
    1172187 View commit details
    Browse the repository at this point in the history

Commits on Aug 1, 2023

  1. Configuration menu
    Copy the full SHA
    8c7e244 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    8aa7120 View commit details
    Browse the repository at this point in the history

Commits on Aug 8, 2023

  1. Configuration menu
    Copy the full SHA
    5d83ce7 View commit details
    Browse the repository at this point in the history

Commits on Aug 10, 2023

  1. Configuration menu
    Copy the full SHA
    f1e64d8 View commit details
    Browse the repository at this point in the history

Commits on Aug 11, 2023

  1. Configuration menu
    Copy the full SHA
    a252b1b View commit details
    Browse the repository at this point in the history

Commits on Aug 14, 2023

  1. Configuration menu
    Copy the full SHA
    21e4584 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    476e1a6 View commit details
    Browse the repository at this point in the history

Commits on Aug 21, 2023

  1. Configuration menu
    Copy the full SHA
    218429d View commit details
    Browse the repository at this point in the history

Commits on Aug 23, 2023

  1. Configuration menu
    Copy the full SHA
    ad078a7 View commit details
    Browse the repository at this point in the history

Commits on Aug 24, 2023

  1. Configuration menu
    Copy the full SHA
    fb88150 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    613b5cf View commit details
    Browse the repository at this point in the history

Commits on Aug 29, 2023

  1. ♻️ Refactor Minor Changes to Keep #635 Clean (#705)

    * ♻️ Refactor minor changes to keep #635 clean
    shaneahmed authored Aug 29, 2023
    Configuration menu
    Copy the full SHA
    59a3553 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    f413775 View commit details
    Browse the repository at this point in the history

Commits on Aug 30, 2023

  1. Configuration menu
    Copy the full SHA
    07935a1 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    112d2b4 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    994fba0 View commit details
    Browse the repository at this point in the history

Commits on Aug 31, 2023

  1. ♻️ Move DataSet tests to test_dataset.py (#708)

    -  Move `DataSet` tests to `test_dataset.py`
    shaneahmed authored Aug 31, 2023
    Configuration menu
    Copy the full SHA
    5bfdcb1 View commit details
    Browse the repository at this point in the history

Commits on Sep 13, 2023

  1. Configuration menu
    Copy the full SHA
    08ed121 View commit details
    Browse the repository at this point in the history

Commits on Oct 6, 2023

  1. Configuration menu
    Copy the full SHA
    53c2145 View commit details
    Browse the repository at this point in the history

Commits on Oct 10, 2023

  1. Configuration menu
    Copy the full SHA
    8b5c902 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    bd134b0 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    02115bf View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    77921c9 View commit details
    Browse the repository at this point in the history

Commits on Oct 13, 2023

  1. Configuration menu
    Copy the full SHA
    d91518a View commit details
    Browse the repository at this point in the history

Commits on Oct 26, 2023

  1. Configuration menu
    Copy the full SHA
    7dd2909 View commit details
    Browse the repository at this point in the history

Commits on Oct 31, 2023

  1. 🎨 New EngineABC Design for Patches (#635)

    - New `EngineABC` design
    - Update `PatchPredictor` based on the new design
    - Move `model_to` from `utils.misc` to `model_abc`
    - Define `load_torch_model` in `model_abc`
    
    ---------
    
    Signed-off-by: Shan E Ahmed Raza <[email protected]>
    Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
    Co-authored-by: measty <[email protected]>
    Co-authored-by: abishekrajvg <[email protected]>
    4 people authored Oct 31, 2023
    Configuration menu
    Copy the full SHA
    a6cd508 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    aff520b View commit details
    Browse the repository at this point in the history

Commits on Nov 15, 2023

  1. 🔀 Merge develop into dev-define-engines-abc

    Signed-off-by: Shan E Ahmed Raza <[email protected]>
    shaneahmed committed Nov 15, 2023
    Configuration menu
    Copy the full SHA
    b218e87 View commit details
    Browse the repository at this point in the history
  2. 🥅 Fix merge errors in classification.py

    Signed-off-by: Shan E Ahmed Raza <[email protected]>
    shaneahmed committed Nov 15, 2023
    Configuration menu
    Copy the full SHA
    1cbf618 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    92f0d50 View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    d1b0d82 View commit details
    Browse the repository at this point in the history
  5. Configuration menu
    Copy the full SHA
    4b357f5 View commit details
    Browse the repository at this point in the history

Commits on Nov 22, 2023

  1. Configuration menu
    Copy the full SHA
    19036cf View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    814711a View commit details
    Browse the repository at this point in the history

Commits on Nov 30, 2023

  1. Configuration menu
    Copy the full SHA
    c9a3420 View commit details
    Browse the repository at this point in the history

Commits on Dec 15, 2023

  1. Configuration menu
    Copy the full SHA
    985e618 View commit details
    Browse the repository at this point in the history

Commits on Dec 18, 2023

  1. Configuration menu
    Copy the full SHA
    12bf6be View commit details
    Browse the repository at this point in the history

Commits on Jan 12, 2024

  1. Configuration menu
    Copy the full SHA
    badf7f0 View commit details
    Browse the repository at this point in the history

Commits on Jan 19, 2024

  1. Configuration menu
    Copy the full SHA
    330c12e View commit details
    Browse the repository at this point in the history

Commits on Jan 26, 2024

  1. Configuration menu
    Copy the full SHA
    67dde29 View commit details
    Browse the repository at this point in the history

Commits on Feb 2, 2024

  1. Configuration menu
    Copy the full SHA
    c812932 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    aaff1f8 View commit details
    Browse the repository at this point in the history

Commits on Feb 12, 2024

  1. Configuration menu
    Copy the full SHA
    af05d15 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    17581f5 View commit details
    Browse the repository at this point in the history

Commits on Feb 14, 2024

  1. Configuration menu
    Copy the full SHA
    2ee5723 View commit details
    Browse the repository at this point in the history

Commits on Feb 20, 2024

  1. Configuration menu
    Copy the full SHA
    f883e81 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    90f396c View commit details
    Browse the repository at this point in the history

Commits on Mar 13, 2024

  1. Configuration menu
    Copy the full SHA
    f60f69c View commit details
    Browse the repository at this point in the history

Commits on Mar 15, 2024

  1. Configuration menu
    Copy the full SHA
    7d3b530 View commit details
    Browse the repository at this point in the history

Commits on Mar 19, 2024

  1. Configuration menu
    Copy the full SHA
    2e522be View commit details
    Browse the repository at this point in the history

Commits on Mar 20, 2024

  1. Configuration menu
    Copy the full SHA
    3a0c884 View commit details
    Browse the repository at this point in the history

Commits on Apr 8, 2024

  1. Configuration menu
    Copy the full SHA
    fe3c697 View commit details
    Browse the repository at this point in the history

Commits on Apr 16, 2024

  1. Configuration menu
    Copy the full SHA
    5b24ba2 View commit details
    Browse the repository at this point in the history

Commits on Apr 23, 2024

  1. Configuration menu
    Copy the full SHA
    0bcf27d View commit details
    Browse the repository at this point in the history

Commits on Apr 26, 2024

  1. ✨ Add WSI processing to EngineABC (#737)

    Implements cache mode for processing using EngineABC. Cache_mode saves intermediate results to zarr file which can be converted to AnnotationStore. Full WSI pipeline needs to be implemented for Engines inheriting the base class.
    
    The output of models should be a dictionary according to the new Engine design.
    
    ---------
    
    Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
    Co-authored-by: Shan E Ahmed Raza <[email protected]>
    3 people authored Apr 26, 2024
    Configuration menu
    Copy the full SHA
    466d005 View commit details
    Browse the repository at this point in the history

Commits on Apr 29, 2024

  1. Configuration menu
    Copy the full SHA
    8ba4bca View commit details
    Browse the repository at this point in the history

Commits on May 13, 2024

  1. Configuration menu
    Copy the full SHA
    3252633 View commit details
    Browse the repository at this point in the history

Commits on May 14, 2024

  1. Configuration menu
    Copy the full SHA
    06af594 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    3dd2c7b View commit details
    Browse the repository at this point in the history

Commits on Jun 3, 2024

  1. Configuration menu
    Copy the full SHA
    7af1ac4 View commit details
    Browse the repository at this point in the history

Commits on Jun 7, 2024

  1. Configuration menu
    Copy the full SHA
    468aeb8 View commit details
    Browse the repository at this point in the history

Commits on Jun 12, 2024

  1. Configuration menu
    Copy the full SHA
    47a81f1 View commit details
    Browse the repository at this point in the history

Commits on Jun 18, 2024

  1. Configuration menu
    Copy the full SHA
    01e2262 View commit details
    Browse the repository at this point in the history

Commits on Jun 25, 2024

  1. Configuration menu
    Copy the full SHA
    557bda5 View commit details
    Browse the repository at this point in the history

Commits on Jul 5, 2024

  1. Configuration menu
    Copy the full SHA
    6495099 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    b1b74ff View commit details
    Browse the repository at this point in the history

Commits on Jul 8, 2024

  1. Configuration menu
    Copy the full SHA
    ae11939 View commit details
    Browse the repository at this point in the history

Commits on Jul 10, 2024

  1. Configuration menu
    Copy the full SHA
    3d04400 View commit details
    Browse the repository at this point in the history

Commits on Jul 17, 2024

  1. Configuration menu
    Copy the full SHA
    f85c6a6 View commit details
    Browse the repository at this point in the history

Commits on Aug 9, 2024

  1. Configuration menu
    Copy the full SHA
    ca294dd View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    c9dfba2 View commit details
    Browse the repository at this point in the history

Commits on Aug 15, 2024

  1. Configuration menu
    Copy the full SHA
    cc28a12 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    d8919d7 View commit details
    Browse the repository at this point in the history

Commits on Aug 20, 2024

  1. Configuration menu
    Copy the full SHA
    209c50a View commit details
    Browse the repository at this point in the history

Commits on Aug 23, 2024

  1. Configuration menu
    Copy the full SHA
    ffcc9aa View commit details
    Browse the repository at this point in the history

Commits on Aug 28, 2024

  1. Configuration menu
    Copy the full SHA
    371a6f3 View commit details
    Browse the repository at this point in the history

Commits on Sep 18, 2024

  1. Configuration menu
    Copy the full SHA
    ca355aa View commit details
    Browse the repository at this point in the history

Commits on Sep 19, 2024

  1. Configuration menu
    Copy the full SHA
    05b1b1d View commit details
    Browse the repository at this point in the history

Commits on Sep 20, 2024

  1. 🧑‍💻 Define PatchPredictor (#783)

    - Redesigns PatchPredictor engine using the new EngineABC base class.
    - The WSIs are now processed using the same code as for the processing the patches using WSI based dataloader.
    - The intermediate output is saved as zarr for the WSIs to resolve memory issues.
    - The output of model architectures should now be a dictionary.
    - The output can be specified as AnnotationStore for visualisation using TIAViz.
    
    ---------
    
    Co-authored-by: abishekrajvg <[email protected]>
    Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
    3 people authored Sep 20, 2024
    Configuration menu
    Copy the full SHA
    81e575d View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    bc73c5b View commit details
    Browse the repository at this point in the history
  3. 🐛 Fix mypy Type Checks for cli/common.py (#864)

    - Fix `mypy` Type Checks for `cli/common.py`
    shaneahmed authored Sep 20, 2024
    Configuration menu
    Copy the full SHA
    8c2f50b View commit details
    Browse the repository at this point in the history

Commits on Oct 2, 2024

  1. Configuration menu
    Copy the full SHA
    e80b77b View commit details
    Browse the repository at this point in the history

Commits on Oct 18, 2024

  1. Configuration menu
    Copy the full SHA
    843f1a6 View commit details
    Browse the repository at this point in the history

Commits on Oct 21, 2024

  1. Configuration menu
    Copy the full SHA
    d93fd94 View commit details
    Browse the repository at this point in the history

Commits on Oct 24, 2024

  1. Configuration menu
    Copy the full SHA
    02aa0e5 View commit details
    Browse the repository at this point in the history

Commits on Nov 15, 2024

  1. Configuration menu
    Copy the full SHA
    43ce16c View commit details
    Browse the repository at this point in the history

Commits on Nov 17, 2024

  1. 🐛 Fix model_to import

    shaneahmed committed Nov 17, 2024
    Configuration menu
    Copy the full SHA
    43afaf7 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    7b4f496 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    dba269c View commit details
    Browse the repository at this point in the history

Commits on Nov 20, 2024

  1. ✨ Add PatchPredictor Engine (#865)

    -  Add `PatchPredictor` Engine based on `EngineABC`
    - Add `return_probabilities` option to Params
    - Removes `merge_predictions` option in `PatchPredictor` engine.
    - Defines `post_process_cache_mode` which allows running the algorithm on `WSI`
    - Add `infer_wsi` for WSI inference
    - Removes `save_wsi_output` as this is not required after post processing.
    - Removes `merge_predictions` and fixes docstring in EngineABCRunParams
    - `compile_model` is now moved to EngineABC init
    - Fixes bug with `_calculate_scale_factor`
    - Fixes a bug in `class_dict` definition.
    - `_get_zarr_array` is now a public function `get_zarr_array` in `misc`
    - `patch_predictions_as_annotations` runs the loop on `patch_coords` instead of `class_probs`
    
    ---------
    
    Co-authored-by: Mark Eastwood <[email protected]>
    Co-authored-by: Mostafa Jahanifar <[email protected]>
    Co-authored-by: Adam Shephard <[email protected]>
    Co-authored-by: Jiaqi-Lv <[email protected]>
    5 people authored Nov 20, 2024
    Configuration menu
    Copy the full SHA
    1fccf15 View commit details
    Browse the repository at this point in the history

Commits on Nov 21, 2024

  1. Configuration menu
    Copy the full SHA
    0d68ad1 View commit details
    Browse the repository at this point in the history

Commits on Nov 22, 2024

  1. Configuration menu
    Copy the full SHA
    819e138 View commit details
    Browse the repository at this point in the history

Commits on Nov 29, 2024

  1. Configuration menu
    Copy the full SHA
    f6ba41f View commit details
    Browse the repository at this point in the history

Commits on Dec 3, 2024

  1. Configuration menu
    Copy the full SHA
    8d94094 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    9237368 View commit details
    Browse the repository at this point in the history