Skip to content

Commit

Permalink
Review comments: nrnmpi_comm and docs.
Browse files Browse the repository at this point in the history
  • Loading branch information
1uc committed Aug 9, 2023
1 parent c5bd73e commit 8c05a93
Show file tree
Hide file tree
Showing 2 changed files with 14 additions and 7 deletions.
19 changes: 13 additions & 6 deletions docs/dev/data-structures.rst
Original file line number Diff line number Diff line change
Expand Up @@ -932,18 +932,25 @@ Most likely, the best approach is to uniformly handle ``area`` and ``diam`` in t
voltages, both in terms of the underlying data structure and how they are accessed in the generated
code.

Report memory usage statistics, including the bookkeeping overhead
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Provide a useful tool for reporting how much memory is consumed in the model data structures (those
reachable via ``neuron::model()``).
This should provide some kind of breakdown between the actual data, the "active" bookkeeping costs
Analyze the bookkeeping overhead
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
There a crude printf-based tool to access memory usage of the datastructures
``print_local_memory_usage`` and ``print_memory_stats``.
This provides some kind of breakdown between the actual data, the "active" bookkeeping costs
(the currently-used index columns, as explained above), and also the "wasted" overhead of values
that have their deletion deferred in order to avoid leaving any data handles "in the wild" from
accidentally dereferencing freed pointers.

This "wasted" storage could, in principle, be recovered after a full traversal of all data
The need to "leak" the stable identifiers could be avoided by replacing the
"raw pointer to integer" idea with a reference counted integer, with bitpacking
and all.

Alternatively, this "wasted" storage could, be recovered after a full traversal of all data
structures that hold ``data_handle<T>`` or ``generic_data_handle`` that collapses handles that are
in previously-valid-but-not-any-more (once valid?) state into "null" (never valid?) state.

Reporting and monitoring the scale of this "waste" is much easier than recovering it, which should
only be done **if** this is **shown** to be a real problem.

Measurements at BBP have shown that under certain conditions the amount of
"leaked" stable identifiers adds up.
2 changes: 1 addition & 1 deletion src/nrnmpi/memory_usage.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,7 @@ void nrnmpi_memory_stats(neuron::container::MemoryStats& stats,
MPI_Type_commit(&memory_usage_mpitype);

MPI_Allreduce(
&local_memory_usage, &stats.total, 1, memory_usage_mpitype, op, nrnmpi_world_comm);
&local_memory_usage, &stats.total, 1, memory_usage_mpitype, op, nrnmpi_comm);

MPI_Op_free(&op);
MPI_Type_free(&memory_usage_mpitype);
Expand Down

0 comments on commit 8c05a93

Please sign in to comment.