Deep recursion in VM code fails with java StackOverflowError unless -Xss is significant

[mkalinin]

Reproduced in ImportLighTest.deepRecursionTest(). It starts to work properly if -Xss parameter is set at least to 5m.

An invocation in VM breeds its own repository track to run the code on it. Each inner invocation creates track on top of a previous one. Thus when code accesses repository it have to pass all tracks and once the level of recursive invocation becomes big enough it turns with StackOverflowError. It's a rare case, but anyway. There is no straightforward solution, requires deeper investigation.

[Nashatyrev]

Here we have 2 recusrions:

• VM call recursion
• Storage recursion

Normally both recursions can be flattened, but this requires some extra efforts