You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy config/example.py to this directory and modify the config file so that base_dir: ./
`run_topostats -c ./example.py
Expected behavior
Would expect it to run to completion.
Output
It doesn't, it crashes with...
[Thu, 16 Jun 2022 15:14:03] [INFO ] [topostats] [20220601_NDP52_FL_10ng_PLO_HEPES_20mM_NaCl_50mM.0_00021] Threshold : 7.502686318963248
[Thu, 16 Jun 2022 15:14:03] [INFO ] [topostats] [20220601_NDP52_FL_10ng_PLO_HEPES_20mM_NaCl_50mM.0_00021] : Applying Gaussian filter (mode : nearest; Gaussian blur (nm) : 2.0).
[Thu, 16 Jun 2022 15:14:03] [INFO ] [topostats] [20220601_NDP52_FL_10ng_PLO_HEPES_20mM_NaCl_50mM.0_00021] : Deriving mask.
[Thu, 16 Jun 2022 15:14:03] [INFO ] [topostats] [20220601_NDP52_FL_10ng_PLO_HEPES_20mM_NaCl_50mM.0_00021] : Created boolean image
[Thu, 16 Jun 2022 15:14:03] [INFO ] [topostats] [20220601_NDP52_FL_10ng_PLO_HEPES_20mM_NaCl_50mM.0_00021] : Tidying borders
[Thu, 16 Jun 2022 15:14:03] [INFO ] [topostats] [20220601_NDP52_FL_10ng_PLO_HEPES_20mM_NaCl_50mM.0_00021] : Labelling Regions
Minimum Grain Size : None
[Thu, 16 Jun 2022 15:14:03] [INFO ] [topostats] [20220601_NDP52_FL_10ng_PLO_HEPES_20mM_NaCl_50mM.0_00021] : Region properties calculated
/home/neil/.virtualenvs/topostats/lib/python3.10/site-packages/numpy/core/fromnumeric.py:3474: RuntimeWarning: Mean of empty slice.
return _methods._mean(a, axis=axis, dtype=dtype,
/home/neil/.virtualenvs/topostats/lib/python3.10/site-packages/numpy/core/_methods.py:189: RuntimeWarning: invalid value encountered in double_scalars
ret = ret.dtype.type(ret / rcount)
[Thu, 16 Jun 2022 15:14:03] [INFO ] [topostats] Processing : /home/neil/work/projects/topostats/TopoStats/tmp/problem_images/20220601_NDP52_FL_10ng_PLO_HEPES_20mM_NaCl_50mM.0_00020.spm
[Thu, 16 Jun 2022 15:14:03] [INFO ] [topostats] Extracting filename from : /home/neil/work/projects/topostats/TopoStats/tmp/problem_images/20220601_NDP52_FL_10ng_PLO_HEPES_20mM_NaCl_50mM.0_00020.spm
[Thu, 16 Jun 2022 15:14:03] [INFO ] [topostats] Filename : 20220601_NDP52_FL_10ng_PLO_HEPES_20mM_NaCl_50mM.0_00020
[Thu, 16 Jun 2022 15:14:03] [INFO ] [topostats] Loading image from : /home/neil/work/projects/topostats/TopoStats/tmp/problem_images/20220601_NDP52_FL_10ng_PLO_HEPES_20mM_NaCl_50mM.0_00020.spm
Processing images from /home/neil/work/projects/topostats/TopoStats/tmp/problem_images, results are under output: 0%| | 0/21 [00:00<?, ?it/s]
[Thu, 16 Jun 2022 15:14:03] [INFO ] [topostats] [20220601_NDP52_FL_10ng_PLO_HEPES_20mM_NaCl_50mM.0_00020] : Loaded image from : /home/neil/work/projects/topostats/TopoStats/tmp/problem_images/20220601_NDP52_FL_10ng_PLO_HEPES_20mM_NaCl_50mM.0_00020.spm
[Thu, 16 Jun 2022 15:14:03] [INFO ] [topostats] Extracting filename from : /home/neil/work/projects/topostats/TopoStats/tmp/problem_images/20220601_NDP52_FL_10ng_PLO_HEPES_20mM_NaCl_50mM.0_00020.spm
[Thu, 16 Jun 2022 15:14:03] [INFO ] [topostats] Loading image from : /home/neil/work/projects/topostats/TopoStats/tmp/problem_images/20220601_NDP52_FL_10ng_PLO_HEPES_20mM_NaCl_50mM.0_00020.spm
multiprocessing.pool.RemoteTraceback:
"""
Traceback (most recent call last):
File "/usr/lib/python3.10/multiprocessing/pool.py", line 125, in worker
result = (True, func(*args, **kwds))
File "/home/neil/work/projects/topostats/TopoStats/topostats/run_topostats.py", line 161, in process_scan
grains.find_grains()
File "/home/neil/work/projects/topostats/TopoStats/topostats/grains.py", line 199, in find_grains
self.calc_minimum_grain_size(image=self.images["labelled_regions"])
File "/home/neil/work/projects/topostats/TopoStats/topostats/grains.py", line 132, in calc_minimum_grain_size
1.5 * (np.quantile(grain_areas, 0.75) - np.quantile(grain_areas, 0.25))
File "<__array_function__ internals>", line 180, in quantile
File "/home/neil/.virtualenvs/topostats/lib/python3.10/site-packages/numpy/lib/function_base.py", line 4371, in quantile
return _quantile_unchecked(
File "/home/neil/.virtualenvs/topostats/lib/python3.10/site-packages/numpy/lib/function_base.py", line 4383, in _quantile_unchecked
r, k = _ureduce(a,
File "/home/neil/.virtualenvs/topostats/lib/python3.10/site-packages/numpy/lib/function_base.py", line 3702, in _ureduce
r = func(a, **kwargs)
File "/home/neil/.virtualenvs/topostats/lib/python3.10/site-packages/numpy/lib/function_base.py", line 4552, in _quantile_ureduce_func
result = _quantile(arr,
File "/home/neil/.virtualenvs/topostats/lib/python3.10/site-packages/numpy/lib/function_base.py", line 4663, in _quantile
previous = np.take(arr, previous_indexes, axis=DATA_AXIS)
File "<__array_function__ internals>", line 180, in take
File "/home/neil/.virtualenvs/topostats/lib/python3.10/site-packages/numpy/core/fromnumeric.py", line 190, in take
return _wrapfunc(a, 'take', indices, axis=axis, out=out, mode=mode)
File "/home/neil/.virtualenvs/topostats/lib/python3.10/site-packages/numpy/core/fromnumeric.py", line 57, in _wrapfunc
return bound(*args, **kwds)
IndexError: cannot do a non-empty take from an empty axes.
"""
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/home/neil/.virtualenvs/topostats/bin/run_topostats", line 33, in <module>
sys.exit(load_entry_point('topostats', 'console_scripts', 'run_topostats')())
File "/home/neil/work/projects/topostats/TopoStats/topostats/run_topostats.py", line 272, in main
for img, result in pool.imap_unordered(processing_function, img_files):
File "/usr/lib/python3.10/multiprocessing/pool.py", line 870, in next
raise value
IndexError: cannot do a non-empty take from an empty axes.
If applicable please include the output error, this can be a copy and paste of the output (preferable) or a screenshot.
If applicable, add screenshots to help explain your problem.
Output files can be attached to this bug report.
(Not required if using Docker) Your computer configuration (please complete the following information):
The problem arises when calling the self.calc_minimum_grain_size(image=self.images["labelled_regions"]) to calculate the minimum grain size. The method looks like this...
defcalc_minimum_grain_size(self, image: np.array) ->float:
"""Calculate the minimum grain size. Very small objects are first removed via thresholding before calculating the lower extreme. Parameters ---------- image : np.array Numpy array of regions of interest after labelling. Returns ------- float Threshold for minimum grain size. """self.get_region_properties()
grain_areas=np.array([grain.areaforgraininself.region_properties])
grain_areas=grain_areas[grain_areas>threshold(grain_areas, method=self.threshold_method)]
self.minimum_grain_size=np.median(grain_areas) - (
1.5* (np.quantile(grain_areas, 0.75) -np.quantile(grain_areas, 0.25))
)
LOGGER.info(f"Minimum Grain Size calculated : {self.minimum_grain_size}")
grain_areas is initially a single value of 234, that is there is a single grain in self.region_properties with that area.
Then after thresholding which currently uses the otsu method to exclude small areas of dirt/noise the value of grain_areas is None since there are no grains that exceed the threshold calculated by this method.
In turn the self.minimum_grain_size can not be calculated and no more calculations can be done.
Images from the Filter stage can be found here (Restricted Access).
@derollins has advised that a better image is 20220601_NDP52_FL_10ng_PLO_HEPES_20mM_NaCl_50mM.0_00021.spm and so I will investigate further but these sorts of errors will need handling sanely and cleanly and shouldn't prevent other images in a given directory from being processed.
Judicious use of try: ... except: throughout at stages where failures can occur would be sensible to allow processing of all images to be attempted.
The text was updated successfully, but these errors were encountered:
Description
Some of the images @derollins tried to processed failed to complete. One such sample is
20220601_NDP52_FL_10ng_PLO_HEPES_20mM_NaCl_50mM.0_00021.spmTo Reproduce
Running off of
devbranch (commit f671bf4)devbranch in a virtual environment.config/example.pyto this directory and modify the config file so thatbase_dir: ./Expected behavior
Would expect it to run to completion.
Output
It doesn't, it crashes with...
If applicable please include the output error, this can be a copy and paste of the output (preferable) or a screenshot.
If applicable, add screenshots to help explain your problem.
Output files can be attached to this bug report.
(Not required if using Docker) Your computer configuration (please complete the following information):
Additional context
The problem arises when calling the
self.calc_minimum_grain_size(image=self.images["labelled_regions"])to calculate the minimum grain size. The method looks like this...grain_areasis initially a single value of234, that is there is a single grain inself.region_propertieswith that area.Then after thresholding which currently uses the
otsumethod to exclude small areas of dirt/noise the value ofgrain_areasisNonesince there are no grains that exceed the threshold calculated by this method.In turn the
self.minimum_grain_sizecan not be calculated and no more calculations can be done.Images from the Filter stage can be found here (Restricted Access).
@derollins has advised that a better image is
20220601_NDP52_FL_10ng_PLO_HEPES_20mM_NaCl_50mM.0_00021.spmand so I will investigate further but these sorts of errors will need handling sanely and cleanly and shouldn't prevent other images in a given directory from being processed.Judicious use of
try: ... except:throughout at stages where failures can occur would be sensible to allow processing of all images to be attempted.The text was updated successfully, but these errors were encountered: