This napari plugin can be used to segment objects or structures in images based on a few brush strokes providing examples of the classes. Based on the same idea as other tools like ilastik, its main strength is that it can use features from pretrained neural networks like VGG16 or DINOV2, enabling the segmentation of more complex images.
Find more information and tutorials in the docs or read the preprint.
You can install napari-convpaint via pip
pip install napari-convpaint
To install latest development version :
pip install git+https://github.com/guiwitz/napari-convpaint.git
These are the scribble annotations provided for training:
And this is the resulting Convpaint segmentation:
segmentation_shark.mp4
Check out the documentation or the paper for more usecases!
Distributed under the terms of the BSD-3 license, "napari-convpaint" is free and open source software
Contributions are very welcome. Tests can be run with tox, please ensure the coverage at least stays the same before you submit a pull request.
If you encounter any problems, please file an issue along with a detailed description.
The idea behind this napari plugin was first developed by Lucien Hinderling in the group of Olivier Pertz, at the Institute of Cell Biology, University of Bern. The code has first been shared as open source resource in form of a Jupyter Notebook. With the desire to make this resource accessible to a broader public in the scientific community, the Pertz lab obtained a CZI napari plugin development grant with the title "Democratizing Image Analysis with an Easy-to-Train Classifier" which supported the adaptation of the initial concept as a napari plugin called napari-convpaint. The plugin has been developed by Guillaume Witz, Mykhailo Vladymyrov and Ana Stojiljkovic at the Data Science Lab, University of Bern, in tight collaboration with the Pertz lab (Lucien Hinderling, Roman Schwob, Benjamin Gräedel, Maciej Dobrzyński).
If you find Convpaint useful in your research, please consider citing:
@article {Hinderling2024.09.12.610926,
author = {Hinderling, Lucien and Witz, Guillaume and Schwob, Roman and Stojiljkovic, Ana and Dobrzynski, Maciej and Vladymyrov, Mykhailo and Frei, Joel and Graedel, Benjamin and Frismantiene, Agne and Pertz, Olivier},
title = {Convpaint - Universal framework for interactive pixel classification using pretrained neural networks},
year = {2024},
doi = {10.1101/2024.09.12.610926},
URL = {https://www.biorxiv.org/content/early/2024/09/14/2024.09.12.610926},
journal = {bioRxiv}
}