Core Substeps

[PavelBlend]

@OmidGhotbi I removed my old code. I just wrote the code in this PR. But for some reason it fails.

[OmidGhotbi]

ok is saw the Simulation.c code that you changed, one thing I noticed is that you added a for loop to calculate substeps in C code then in Python code, you did not change the substeps behavior so the frame in Python is not the same frame in C code, in your case every single subframe in the blender will be split to the number of substeps again. for example if you have 16 substeps in blender it will be multiplied in C code, unless I missed out the python modification this behavior will cause miscalculation.
plus, double down the performance.

[OmidGhotbi]

Second, in gravity, calculation you multiply deltatime and gravity and once more you multiply it with velocity, it's basically deltatime**2 . I'm not sure if you should use deltatime like this in this part. i will check it.
I'll get back to you as soon as I reach home and be able to run a few tests.

[OmidGhotbi]

About the steps if you want to do all steps in C code you need to remove substeps from python and add it to C, (kind of replace it)
in python

blender-molecular/blender/ops.py

Line 316 in 8f66898

frame_float = self.step / mol_substep

        mol_substep = scene.mol.substep

        frame_float = self.step / mol_substep
        frame_current = int(frame_float)
        subframe = frame_float - frame_current

And finally in this line :

blender-molecular/blender/ops.py

Line 401 in 8f66898

scene.frame_set(frame=frame_current, subframe=subframe)

scene.frame_set(frame=frame_current, subframe=subframe)
you get the new substep and apply it as a subframe, if you do that in C you do not need it here,

[PavelBlend]

@OmidGhotbi I removed the previous code and added a new one. The simulation turns out to be unstable.

[OmidGhotbi]

Let me check it and i get back to you, just one thing the new code has 3 conflicts with my code do I need to keep them or resolve them by replacing them?
Can I help you with the code, and push the code to the new branch?

[PavelBlend]

I'm sorry. I just noticed that I didn't add all the code. For some reason I didn't add the python code from the addon. I accidentally deleted it. You won't be able to run my code. I didn't plan to merge this code. I wanted to do a test, and if it succeeds, then rewrite the code in a clean version. One of these days I’ll try to write the code again and if I succeed, I’ll add it here.

[PavelBlend]

Can you tell me the formula for adding gravity and moving particles? Does moving look like this?

delta_time = 1 / (fps * substeps)
new_pos = old_pos + delta_time * velocity

[OmidGhotbi]

First of all in ops python line 401 you need this change if you are about to calculate substeps in C.

            self.check_write_uv_cache(context)
            # scene.frame_set(frame=frame_current, subframe=subframe) // Replace 
            scene.frame_set(frame=frame_current)

with this change, you will not get a double substep calculation.

please share the source of Eq that you are using for new_pos = old_pos + delta_time * velocity with me so I can see why they used it.
In general, about delta_time we need to talk a little bit. dt is a variable that we normally use it formulate to simulate the time in actual code is not that relevant in general because the changing time is dt itself so normally we add gravity and velocity etc in every frame or subframe and with changing every frame or subframe the dt is implied by nature of change.
so in physics formula, because we can not change time on paper we add it and inside calculation use it, but when we really have time and frames we do not need it because we already have it.

imagine this simple eq,
x = (x + vel ) * dt
so if you have it on paper you need to add a range for dt so it can show the change of x in time. now imagine the dt is frame change in the blender. you will need to add x to itself instead.
x = x + vel
they will give you the same results one in the paper on in time (that we recreate in frame change in blender) because with adding every frame old x position will be the sum of x plus the amount of velocity etc. i hope I have explained it correctly,
remember in code if you are not using the frame for repeating the code in any part you will be forced to use dt instead again. by the way, to make affect smaller and whole process slower you can multiply by dt so every x+vel get smaller and smaller depends on your dt and substeps and control the speed of movement.

the way you can better understand it is dt is for calculate the frame progress.

float frameTime = (float)(_currentFrameDeltaTimeRemaining + _currentFrameTimeStep);
float frameProgress = 1.0f - frameTime / (float)_currentFrameDeltaTime;

or

    int estimatedNumFrameSubsteps = std::max((int)std::ceil(dt / timeStep), 1);
    timeStep = dt / estimatedNumFrameSubsteps;

one point I need to mention last night I rewrote the part you changed and associated Python codes and noticed one thing when the collider objects move fast the calculation will not be correct because C code doesn't know about the small changes that happened in the scene, for example in Trap cup I created in my test scene for TAngra it will start good but because all the movements are happening in two frames and I have over 70 substeps the continuation of the movement doesn't pass to C code and by nature of movement the results altho is good it's not correct. Cause the C code is not aware of the change in 70 substeps frames.
so I can say in the current state of this code you will get performance and in the other hand, you will lose precision.

[OmidGhotbi]

About the repo i understand it's a test branch I think maybe it is faster if I can modify the code in my forked repo modify this branch and push it so you can see ass well. it's just a suggestion that's all.

[PavelBlend]

Is it possible to make two types of substeps? One for blender and the other for core. For example, blender will have 16 substeps, and core will have 4 substeps. As a result, each substep from blender will be divided into susbteps from core:

blender_substeps = 16
core_substeps = 4
final_core_substeps = blender_substeps * core_substeps

This will prevent you from losing so much accuracy. The addon will handle collision every 4 steps.

[PavelBlend]

please share the source of Eq that

I found it a long time ago. Now I don’t remember where.

I think I need to remove the current code and rewrite it again.

[OmidGhotbi]

Is it possible to make two types of substeps? One for blender and the other for core. For example, blender will have 16 substeps, and core will have 4 substeps. As a result, each substep from blender will be divided into susbteps from core:

blender_substeps = 16
core_substeps = 4
final_core_substeps = blender_substeps * core_substeps

This will prevent you from losing so much accuracy. The addon will handle collision every 4 steps.

I was thinking about that a few days ago too, something about making 2 or 4 substeps in the blender and others in C code as much as I like it it could scarify the precision but it's more balance, the question will be how much performance we can gain with it. if it's 25% it's not worth it because we can do that just by using vectors instead of arrays in existing code. I'm not sure about that. if you can gain more than 50% of performance okay we can work with it.

[OmidGhotbi]

you can make 3 different presets:
1 - Full Python substeps for precision = best for accuracy slow movment
2 - Halp precision by a combination of Python + C balance precision = best for normal scenes and medium movement
3 - Full C Low precision = best for slow movement

Another choice is to try to find another type of relative solver or granular solver that is already better.

[OmidGhotbi]

Based on the movement we have in the current code you can calculate the Velocity like this:
dt = timeStep / numSubsteps
vel = (pos - prevPos) / dt

As you can see the (position + velocity) means new position
newPos = currentPos + vel

if you have gravity you will get the te new position the same way
newPos = currentPos + (gravity * dt)

you can combine all of them in one function or do in separate functions that is preferred.

of course, you need the equation to update particle position and this is a general answer.

Something like that:

while (simulate) {
    for (int i = 0; i < particles.length; i++)
    {
        Vel [ ị ] = Vel [ ị ] + dt * gravity
        Pos [ ị ] = Xpos [ ị ]
        Xpos [ ị ] = Xpos [ ị ] + dt * Vel [ ị ]

       foreach C in constraints
            solve(C, dt)

       for all particles i
            Vel [ ị ] < (Xpos [ ị ] - Pos [ ị ]) / dt
    }
}    

This can change from equation to equation and this is a general approach so you need the math for it or try as many as math to get what you are looking for.
this approach is based on position and can work in many situations.
Remember dt is just here to make change smaller for example if old pos is 0 and the movement is 16 cm for 1 frame and your substeps is 16 that means (Xpos + NewPos) * dt = (0+16) * 1/16 = 16/16 = 1 so for every subframe 1cm of movement. dt is here to reduce the movement base of the number of subframes so movement in 16 sub frames is the same as movement in 1 frame.

[OmidGhotbi]

i tested the new version and as we talked I made a few changes, first I divided substeps into two parts one subsample from the blender and the second internal iteration directly in C code. In this manner, first, I divided the frames by substeps. Then inside the C code divide every step by iteration number, so if we have 10 substeps from blender and 4 internal iterations it will be 40 subsamples total for every frame.
The result was better depending on how many iterations we had. at least it is more accurate with the same time for simulation or faster with the same substeps.

by the way, I converted the code into C++ as well and I'm trying to use nvc++ to compile it for GPU so it can be faster by a factor of 14 to 20 times of course the limitation of blender itself will be applied.
here is some resource if you need to read more about it.
https://developer.nvidia.com/blog/accelerating-python-on-gpus-with-nvc-and-cython/

let me know about it.

[OmidGhotbi]

Remember for substeps from blender we need dt but for internal iteration, we do need to apply dt / iteration otherwise it will explode.