Example of how to write data files as series of submodels.

[nrnhines]

Closes neuronsimulator/nrn#943

[pramodk]

@nrnhines : can we get version as property or via some method? We have same problem of hard coding it in neurodamus and I wonder if it could be done in better way.

[nrnhines]

I was thinking the same thing. The first thing that comes to mind is yet another ParallelContext method such as pc.nrncore_data_version() or perhaps a more generic thing such as pc.nrncore_property('property_name')
since I would also like to know if coreneuron is available, the gpu is available, etc. Also the documentation for pc.nrnbbcore_write(...) is missing the first line data version requirement for files.dat.

[pramodk]

Or, as an end user, I don't want to manage combining these models like this. What I want to say is take these subdir_1 subdir_2 subdir_3 and simulate it. Internally we can do necessary symlinks and group files.dat.

Another possibility: coreneuron can be also updated to accept multiple data directories! (I am now thinking more about this possibility!)

[nrnhines]

We should discuss by zoom. I'm not seeing the value of multiple data directories for this problem since I can't envision the filenames not being unique regardless of the number of submodels. You are right that files.dat can easily be handled by nrnbbcore_write but need to work out some api details. Maybe the user does not have to be aware of the gidgroups array of Vector either.

[nrnhines]

I propose extending nrnbbcore_write with a new signature

pc.nrnbbcore_write([path, [i_submodel, n_submodel])

which indicates how many submodels are involved and at which point in the sequence this call is embedded. (when i_submodel == (nsubmodel - 1), the files.dat file is written at the end. Other possibilities are to count down from a beginning n_submodel to 0 and when 0 is reached, emit the files.dat file. In any event we would like for this to work on an mpi cluster and/or threads. It would also be nice to support the submodel strategy for the case of separate launch of NEURON for groups of submodels. That would require that the accumulating groupgid information persist across launches which would mean writing some kind of intermediate file or else a files.dat to which nrnbbcore_write appends further gidgroup information as well as update the second line of the file (ngroup). If this latter is implemented then the signature could be further simplified to

pc.nrnbbcore_write([path], [bool append] )

which means that files.dat is to be updated at the end of this call and so is always valid when the call exits.
Lastly, I could imagine that the optional bool append could be eliminated and default is always append and it would be
up to the user to clear out the "outdat" folder or start a new one if wanting to start from the beginning.

[nrnhines]

And is this a good time to allow the synonym, pc.nrncore_write([path], [bool append])?
Note, in test_submodel.py this would change the statement to

pc.nrncore_write('./coredat', isubmodel != 0)

and would eliminate write_files_dat(...)

[pramodk]

You are right that files.dat can easily be handled by nrnbbcore_write but need to work out some api details.

Ah yes! we don't need multiple directories!