Memory issue when running the cnmfe and estimates.F_dff

[zhouyi0812]

Hello!

My computer C drive currently have 600G free space and 128 memory. But I have different memory error.

First, when I run cnmfe without downsampling the memory error says require 200-900 GB. But it used to work well without downsampling

Second, when I downsampling my tsub=2, things work fine. But when I run estimates.F_dff, it has another memory error:

Here are the steps I have tried from the similar question:

1. Enabling Swapping: after restart it seems doesn't work
   

2. Adjust the n_process: Seems doesn't work either as I change from 10, 15 to none (24)

3. caimanmanager diagnose
   

Thank you very much for your help!

[EricThomson]

Hi it looks like you are not using the most recent version of Caiman (1.10.4). Could you please update and try again?

[zhouyi0812]

may I ask how should I update if I am in the mescore environment?
Thank you!

[EricThomson]

Within your environment, conda update caiman -c conda-forge

[zhouyi0812]

Thank you!!
I updated with the code, but it still memory error

[EricThomson]

Looks like you should probably downsample (as you originally tried), and if the results suffer then go to online algorithm.

[EricThomson]

I added more detailed discussion of handling memory in the demo notebook (in PR #1300 ). It gives a more clear roadmap of what you should do, and I added a discussion of subsampling. I'm closing this for now, but @zhouyi0812 feel free to open if this isn't enough info.