Automatic memory management?

[dcoracle]

Hello,

we are close to pushing a Microstream solution into production for cache-based (microservice) workloads. One thing that we are encountering as we are building this out is to properly tune the data structures so that it is efficient to store in RAM otherwise we are encountering OOM exceptions. Since this is an MVP, we kept it as simple as possible for developers and did not go into the lazy-loading and on-demand features to control this behaviour. Essentially, it is all eagerly loading the cache into RAM.

Having the Lazy option is good, but I feel this requires developers to have quite a bit of low level (and imperative) tweaking if that makes any sense. In other words, this adds complexity to what is aimed to be simple.

My ask is if there are any plans to somehow change the behaviour so that Microstream can eagerly load as much into RAM as possible, and if it comes close to an OOM situation, to intelligently move some data back to disk based off of some rules. Basically abstracting this detail similar to how operating systems use memory and swap space?

[fh-ms]

The only way MicroStream can achieve partial loading and unloading is with the intermediary Lazy type. It doesn't use any bytecode manipulation or other 'magic' for the sole reason of not interfering with the user object model.
If you place the lazy references well in strategic places of your object design, the lazy reference manager will do the magic for you. It will unload lazy loaded subgraphs automatically when the memory is about to run out. With the default settings, the first lazies to be unloaded will be the ones which are rarely used. So the 'hot' data will stay in memory.
For more information, see the explanation in the docs.

[dcoracle]

Thanks. So suppose we have a root instance with the following instance var: Lazy<ArrayList> allEntities; and if it contains 20 million entities, does the lazy reference manager load and unload 20 million (to and from RAM) at a time?

I understand traditionally, this should be the case, but I am not sure if Microstream has some added 'magic' to have this kind of feature to partially load this into heap.

The basis of my question comes from the fact that we have RAM constraints due to our quota and want to understand when best to use Microstream and when not to. If we have millions of entities containing document-based data (i.e. html/js/css-based cache for each employee), would it be better to use some other embedded persistence store like H2/SQLite/etc so that it does not hog all the RAM.

[fh-ms]

Yes, the complete sub-graph, which is lazy-referenced, will be loaded and unloaded. In that case, the whole list.
A different approach would be to segment the data. For example, a Map<Key, Lazy<List<Value>>.
We are currently developing lazy collections, which do the segmented lazy loading internally. They should be ready in Q3.

And, of course, it is possible to combine different storage solutions with MicroStream. Keep documents or file data outside of your RAM, and just store references, like names of keys, in your object-graph.

[Sandared]

I just stumbled upon this discussion. Regarding those Lazy Collections: Are they already available (maybe in a beta version)?
We are currently investigating such lazy collection possibilities and it would be nice to have a reference on how you would approach such a collection :)

[fh-ms]

They will be available in 8.0-EA2 early next year.