-
Notifications
You must be signed in to change notification settings - Fork 4
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Benchmarking #129
Comments
Hi Tischi, great! Maybe two or three scenarios are enough. I would definitely love to see some numbers comparing local SSD versus remove fileserver. I'm also curious what RAM your laptop uses. Does it makes sense? Does RAM-generation matter much? If you like, you can have some small tests on different computers and then you do two extreme tests ans talk about it in the manuscript. Cheers, |
Sure thing!
Good point!
|
Great, I have started making a table for this and adding supplementary text. |
@nornil Thanks! I added more measurements:
Could you put the table somewhere where it is accessible from here and paste a link to it please? |
Tischi's MacHardware
Measurements
Testing different output file types:
|
I have written it into our SI google doc as it need to go there eventually |
We can copy it there later! |
Tischi's Mac / Data on network shareHardware
Measurements
@haesleinhuepf |
Windows VMVirtual machine in the data centre numCores = 4
|
@nornil |
Cluster node
All 10 timepoints on same cluster node:
|
I think I have most of it now. Missing:
Anything else? |
Home office with mediocre internet connection
(*) I think we can say that this is on the edge of being bearable as research shows that humans typically have a short term attention span of 5 seconds. For sure this is not useful to actually process the data, but, if one has no other choice, it may be just OK to use this mode to record a processing macro to be later batch executed, e.g. on a computer cluster in the data centre. However, recommendation for home office scenario is to use a VM in the data centre. |
@nornil
|
Read Tif / Write Tif: I/O threads: 1 ; process threads 4 (Tischi's Mac)
|
@haesleinhuepf @nornil
I think I have everything in place now for some benchmarking. The issue is that there is an exponential number of parameter combinations and we thus have to limit ourselves.
Dataset
For the initial benchmarking I suggest to only use one data set where the xyz dimensions are typical for light-sheet data.
Readouts
LPV
: Loading and processing of a single xyz volume [ ms ]SV
: Saving of a single xyz volume [ ms ]M
: maximal memory usage during processing [GB]To estimate the total processing and saving (
TPS
) time of a whole data set on a single computer one would have to use the following formula:TPS = nC * nT * (LPV + SV)
Processing
In order to assess the processing we added a single processing step, namely a 3x3 binning in x&y.
This is a very typical processing step and should be the computationally most expensive one of the currently available.
Note that this significantly reduces the saving times as the data to be saved is 3x3=9 times smaller than the data to be loaded.
My plan would be to measure these numbers for a few different computer and data storage scenarios.
What do you think?
The text was updated successfully, but these errors were encountered: