running error on MissingInputException in rule resolve_cellpose for merscope data

[KunHHE]

Hi,
Nice development on the sopa that we can use. I am a new user and trying to see how sopa cool on spatial data analysis.

I installed sopa in conda (pip install sopa[snakemake,cellpose,baysor,tangram]) and installed Baysor in julia.
So tried to run using config file (baysor_cellpose) for merscope datasets.

But met error looks like "MissingInputException in rule resolve_cellpose" step.

Any ideas where my issue is? Thansk very much!!!

(sopa) C:\Users\hekun\sopa>snakemake --config data_path=C:/Users/hekun/Downloads/S3R1 --configfile=C:/Users/hekun/sopa/workflow/config/merscope/baysor_cellpose.yaml --cores 1 --use-conda
SpatialData object path set to default: C:\Users\hekun\Downloads\S3R1.zarr
To change this behavior, provide --config sdata_path=... when running the snakemake pipeline
Building DAG of jobs...
Provided cores: 1 (use --cores to define parallelism)
Rules claiming more threads will be scaled down.
Job stats:
job count

---

aggregate 1
all 1
explorer 1
image_write 1
patchify_baysor 1
patchify_cellpose 1
report 1
resolve_baysor 1
resolve_cellpose 1
to_spatialdata 1
total 10

Select jobs to execute...

[Tue Nov 26 22:47:24 2024]
rule to_spatialdata:
input: C:/Users/hekun/Downloads/S3R1
output: C:/Users/hekun/Downloads/S3R1.zarr/.zgroup
jobid: 4
reason: Missing output files: C:/Users/hekun/Downloads/S3R1.zarr/.zgroup
resources: tmpdir=C:\Users\hekun\AppData\Local\Temp, mem_mb=128000, mem_mib=122071

Activating conda environment: sopa
C:\Users\hekun\miniconda3\envs\sopa\lib\site-packages\numba\core\decorators.py:246: RuntimeWarning: nopython is set for njit and is ignored
warnings.warn('nopython is set for njit and is ignored', RuntimeWarning)
C:\Users\hekun\miniconda3\envs\sopa\lib\functools.py:938: UserWarning: The index of the dataframe is not monotonic increasing. It is recommended to sort the data to adjust the order of the index before calling .parse() (or call parse(sort=True)) to avoid possible problems due to unknown divisions.
return method.get(obj, cls)(*args, **kwargs)
INFO The column "global_x" has now been renamed to "x"; the column "x" was already present in the dataframe, and
will be dropped.
INFO The column "global_y" has now been renamed to "y"; the column "y" was already present in the dataframe, and
will be dropped.
[INFO] (sopa.io.standardize) Writing the following spatialdata object to C:\Users\hekun\Downloads\S3R1.zarr:
SpatialData object
├── Images
│ └── 'Downloads_S3R1_z3': DataTree[cyx] (4, 39211, 35709), (4, 19605, 17854), (4, 9802, 8927), (4, 4901, 4463), (4, 2450, 2231)
└── Points
└── 'Downloads_S3R1_transcripts': DataFrame with shape: (, 9) (2D points)
with coordinate systems:
▸ 'global', with elements:
Downloads_S3R1_z3 (Images), Downloads_S3R1_transcripts (Points)
INFO The Zarr backing store has been changed from None the new file path: C:\Users\hekun\Downloads\S3R1.zarr
[Tue Nov 26 22:48:43 2024]
Finished job 4.
1 of 10 steps (10%) done
Select jobs to execute...

[Tue Nov 26 22:48:43 2024]
rule image_write:
input: C:/Users/hekun/Downloads/S3R1.zarr/.zgroup
output: C:/Users/hekun/Downloads/S3R1.explorer/morphology.ome.tif
jobid: 8
reason: Missing output files: C:/Users/hekun/Downloads/S3R1.explorer/morphology.ome.tif; Input files updated by another job: C:/Users/hekun/Downloads/S3R1.zarr/.zgroup
resources: tmpdir=C:\Users\hekun\AppData\Local\Temp, mem_mb=64000, mem_mib=61036, partition=longq

Activating conda environment: sopa
C:\Users\hekun\miniconda3\envs\sopa\lib\site-packages\numba\core\decorators.py:246: RuntimeWarning: nopython is set for njit and is ignored
warnings.warn('nopython is set for njit and is ignored', RuntimeWarning)
[WARNING] (sopa._sdata) sdata object has no valid segmentation boundary. Consider running Sopa segmentation first.
[INFO] (sopa.io.explorer.images) Writing multiscale image with procedure=semi-lazy (load in memory when possible)
Writing tiles: 0%| | 0/5460 [00:00<?, ?it/s][INFO] (sopa.io.explorer.images) > Image of shape (4, 39211, 35709)
Writing tiles: 100%|██████████████████████████████████████████████████████████████▉| 5459/5460 [01:51<00:00, 48.97it/s]
[INFO] (sopa.io.explorer.images) (Loading image of shape (4, 19605, 17854)) in memory
[INFO] (sopa.io.explorer.images) > Image of shape (4, 19605, 17854)
[INFO] (sopa.io.explorer.images) > Image of shape (4, 9802, 8927)
[INFO] (sopa.io.explorer.images) > Image of shape (4, 4901, 4463)
[INFO] (sopa.io.explorer.images) > Image of shape (4, 2450, 2231)
[INFO] (sopa.io.explorer.images) > Image of shape (4, 1225, 1115)
[INFO] (sopa.io.explorer.converter) Saved files in the following directory: C:\Users\hekun\Downloads\S3R1.explorer
[INFO] (sopa.io.explorer.converter) You can open the experiment with 'open C:\Users\hekun\Downloads\S3R1.explorer\experiment.xenium'
[Tue Nov 26 22:52:00 2024]
Finished job 8.
2 of 10 steps (20%) done
Select jobs to execute...

[Tue Nov 26 22:52:00 2024]
checkpoint patchify_cellpose:
input: C:/Users/hekun/Downloads/S3R1.zarr/.zgroup
output: C:/Users/hekun/Downloads/S3R1.zarr/.sopa_cache/patches_file_image, C:/Users/hekun/Downloads/S3R1.zarr/.sopa_cache/patches
jobid: 6
reason: Missing output files: C:/Users/hekun/Downloads/S3R1.zarr/.sopa_cache/patches_file_image; Input files updated by another job: C:/Users/hekun/Downloads/S3R1.zarr/.zgroup
resources: tmpdir=C:\Users\hekun\AppData\Local\Temp
DAG of jobs will be updated after completion.

Activating conda environment: sopa
C:\Users\hekun\miniconda3\envs\sopa\lib\site-packages\numba\core\decorators.py:246: RuntimeWarning: nopython is set for njit and is ignored
warnings.warn('nopython is set for njit and is ignored', RuntimeWarning)
[INFO] (sopa.patches.patches) 49 patches were saved in sdata['sopa_patches']
Touching output file C:/Users/hekun/Downloads/S3R1.zarr/.sopa_cache/patches.
[Tue Nov 26 22:52:05 2024]
Finished job 6.
3 of 10 steps (30%) done
MissingInputException in rule resolve_cellpose in file C:\Users\hekun\sopa\workflow\Snakefile, line 159:
Missing input files for rule resolve_cellpose:
output: C:/Users/hekun/Downloads/S3R1.zarr/.sopa_cache/cellpose_boundaries_done
affected files:
C:\Users\hekun\Downloads\S3R1.zarr.sopa_cache\cellpose_boundaries\15.parquet
C:\Users\hekun\Downloads\S3R1.zarr.sopa_cache\cellpose_boundaries\5.parquet
C:\Users\hekun\Downloads\S3R1.zarr.sopa_cache\cellpose_boundaries\29.parquet
C:\Users\hekun\Downloads\S3R1.zarr.sopa_cache\cellpose_boundaries\3.parquet
C:\Users\hekun\Downloads\S3R1.zarr.sopa_cache\cellpose_boundaries\26.parquet
C:\Users\hekun\Downloads\S3R1.zarr.sopa_cache\cellpose_boundaries\25.parquet
C:\Users\hekun\Downloads\S3R1.zarr.sopa_cache\cellpose_boundaries\4.parquet
C:\Users\hekun\Downloads\S3R1.zarr.sopa_cache\cellpose_boundaries\22.parquet
C:\Users\hekun\Downloads\S3R1.zarr.sopa_cache\cellpose_boundaries\13.parquet
C:\Users\hekun\Downloads\S3R1.zarr.sopa_cache\cellpose_boundaries\44.parquet
C:\Users\hekun\Downloads\S3R1.zarr.sopa_cache\cellpose_boundaries\2.parquet
C:\Users\hekun\Downloads\S3R1.zarr.sopa_cache\cellpose_boundaries\24.parquet
C:\Users\hekun\Downloads\S3R1.zarr.sopa_cache\cellpose_boundaries\45.parquet
C:\Users\hekun\Downloads\S3R1.zarr.sopa_cache\cellpose_boundaries\47.parquet
C:\Users\hekun\Downloads\S3R1.zarr.sopa_cache\cellpose_boundaries\37.parquet
C:\Users\hekun\Downloads\S3R1.zarr.sopa_cache\cellpose_boundaries\46.parquet
C:\Users\hekun\Downloads\S3R1.zarr.sopa_cache\cellpose_boundaries\23.parquet
C:\Users\hekun\Downloads\S3R1.zarr.sopa_cache\cellpose_boundaries\41.parquet
C:\Users\hekun\Downloads\S3R1.zarr.sopa_cache\cellpose_boundaries\43.parquet
C:\Users\hekun\Downloads\S3R1.zarr.sopa_cache\cellpose_boundaries\42.parquet
C:\Users\hekun\Downloads\S3R1.zarr.sopa_cache\cellpose_boundaries\30.parquet
C:\Users\hekun\Downloads\S3R1.zarr.sopa_cache\cellpose_boundaries\33.parquet
C:\Users\hekun\Downloads\S3R1.zarr.sopa_cache\cellpose_boundaries\34.parquet
C:\Users\hekun\Downloads\S3R1.zarr.sopa_cache\cellpose_boundaries\39.parquet
C:\Users\hekun\Downloads\S3R1.zarr.sopa_cache\cellpose_boundaries\28.parquet
C:\Users\hekun\Downloads\S3R1.zarr.sopa_cache\cellpose_boundaries\9.parquet
C:\Users\hekun\Downloads\S3R1.zarr.sopa_cache\cellpose_boundaries\14.parquet
C:\Users\hekun\Downloads\S3R1.zarr.sopa_cache\cellpose_boundaries\10.parquet
C:\Users\hekun\Downloads\S3R1.zarr.sopa_cache\cellpose_boundaries\16.parquet
C:\Users\hekun\Downloads\S3R1.zarr.sopa_cache\cellpose_boundaries\7.parquet
C:\Users\hekun\Downloads\S3R1.zarr.sopa_cache\cellpose_boundaries\8.parquet
C:\Users\hekun\Downloads\S3R1.zarr.sopa_cache\cellpose_boundaries\31.parquet
C:\Users\hekun\Downloads\S3R1.zarr.sopa_cache\cellpose_boundaries\12.parquet
C:\Users\hekun\Downloads\S3R1.zarr.sopa_cache\cellpose_boundaries\27.parquet
C:\Users\hekun\Downloads\S3R1.zarr.sopa_cache\cellpose_boundaries\1.parquet
C:\Users\hekun\Downloads\S3R1.zarr.sopa_cache\cellpose_boundaries\18.parquet
C:\Users\hekun\Downloads\S3R1.zarr.sopa_cache\cellpose_boundaries\19.parquet
C:\Users\hekun\Downloads\S3R1.zarr.sopa_cache\cellpose_boundaries\17.parquet
C:\Users\hekun\Downloads\S3R1.zarr.sopa_cache\cellpose_boundaries\21.parquet
C:\Users\hekun\Downloads\S3R1.zarr.sopa_cache\cellpose_boundaries\32.parquet
C:\Users\hekun\Downloads\S3R1.zarr.sopa_cache\cellpose_boundaries\0.parquet
C:\Users\hekun\Downloads\S3R1.zarr.sopa_cache\cellpose_boundaries\48.parquet
C:\Users\hekun\Downloads\S3R1.zarr.sopa_cache\cellpose_boundaries\6.parquet
C:\Users\hekun\Downloads\S3R1.zarr.sopa_cache\cellpose_boundaries\35.parquet
C:\Users\hekun\Downloads\S3R1.zarr.sopa_cache\cellpose_boundaries\20.parquet
C:\Users\hekun\Downloads\S3R1.zarr.sopa_cache\cellpose_boundaries\11.parquet
C:\Users\hekun\Downloads\S3R1.zarr.sopa_cache\cellpose_boundaries\36.parquet
C:\Users\hekun\Downloads\S3R1.zarr.sopa_cache\cellpose_boundaries\38.parquet
C:\Users\hekun\Downloads\S3R1.zarr.sopa_cache\cellpose_boundaries\40.parquet

[quentinblampey]

Hi @KunHHE,
I think this is related to the issue #64, which is a windows-specific issue (probably related to snakemake).

By any chance, do you have access to a linux or macOS machine? If yes, could you let me know if it works?
Of course, I would still need to fix it on windows, but it's just to make sure it's the same issue

[KunHHE]

Thanks @quentinblampey, trying to run it in CLI model. It looks working now at least finished --patchify and running --segmentation for patches. Thanks!