Python ML and SOFA simulation integration

[snsvsn]

I have been working on an RL problem for quite sometime and i have been looking online for environment simulator. I recently stumbled upon SOFA framework and found the Timishenko Beam plugin to be very close to what i want as an environment simulator. I am requesting for help in integrating the Timoshenko Beam: catheters plugin into python environment . Basically i will be using SOFA framework as an environment simulation. My RL training would look like

1)Start RL python code in anaconda
2)Get information from SOFA framework ( current state)
3)Use the information in RL prediction
4)Update the SOFA framework based on RL prediction (next state)
5) determine reward and train RL in python
6) Repeat step 2 - 5 until end goal achieved

My questions are

1. Is it possible to perform step by step simulation in SOFA environment. More specifically can i reload the position/orientation of the test object (such as a catheter in vessel), from the last iteration and run the simulation with new inputs in the current iteration. Or do i have to restart everything every iteration.
2. what installation steps do i need to follow to be able to call SOFA framework from my python code .

PS: I am not a prolific programmer and my knowledge in software is very limited to being able to write simple python scripts. Exchanging information between two softwares or processes is not my domain, but i can learn.

Best,
SVSN

[hugtalbot]

Hi @snsvsn

thanks a lot for your post and your interest in SOFA!
We do have an open-source plugin for beams following the Timoshenko theory. Note that we also have a plugin mainly developed by @younesssss on beams following the Cosserat rod theory.

To reply your points, some work already performed step by step simulation in the context of simulation of RL. @p-shg should be able to enlighten us!
Pierre, are the SofaGym codes now available?

Best.

[damienmarchal]

Hi @snsvsn

It is possible to create a sofa scene and run a step by a step simulation from a pure python3 interpreter. At each simulation step it is also possible to get the simulation state.

Restarting a simulation from an existing state is much harder in the general case as it depend on how are implemented the components used in the scene. Not all of sofa component allowed this "interruptile and warm start" behavior. I have no clear view on the current compatibility state on this topic.

The solution used in SofaGym was to fork the process to create the different branches of the simulation. I let @p-shg to speak about that.

[p-shg]

Hi

From what I understand of your message, what you are trying to develop is quite close to my thesis subject. I can probably help a little.
First I would very strongly advise against using anaconda, anaconda and SOFA do not work well together in my experience. I would recommend installing libraries with pip and running the code in another interpreter.

I don't understand what you mean by "Use the information in RL prediction to update the SOFA scene". The SOFA scene is the RL environment, I don't get what you want to predict.

Anyways to answer your questions:
It is indeed possible to start and stop a SOFA simulation from a python script and have it behave as an RL environment. It is not possible to reload the position/orientation and then start simulating again. The workaround we found was to fork the process, which duplicates the simulation in RAM.
You need at least SOFA and SofaPython3 compiled.

We have been working on a SofaGym software library. It will be published probably at the end of the month on the SofaDefrost github. We are hoping to include SOFA binaries to make the code easier to run.
Once SofaGym is public you will want to have a look at the ConcentricTubeRobot example which is a CTR modeled with Timoshenko beams navigating a nasal cavity. If you want to simulate a guidewire and concentric catheter you should be able to do it easily from this scene. If you want to work with actuated catheters we do not have those available at the moment but you can model them using the SoftRobots plugin and integrate your scene in SofaGym.

Hope this helps. If you need further help once the code is released please contact me or Etienne Ménager (https://team.inria.fr/defrost/team-members/etienne-menager/)
Cheers

[hugtalbot]

In addition, to be noted : the recent release of DeepPhysX allowing to couple SOFA with ML libraries but I doubt the platform already supports RL examples

[hugtalbot]

SofaGym has been released here: https://github.com/SofaDefrost/SofaGym/

[snsvsn]

@p-shg : Thank you for your response. I am definitely looking forward to woking with SofaGym. For my RL problem, i am using keras and tesnor flow to build and train the neural network. I intend to use SofaGym as an environment simulator. I would like to simulate motion of devices in SOFA, send the motion commands from tensorflow code, have Sofa run the scene with the command and send the result of the scene back to tensorflow code. This would go on until the training is completed. From your description it seems like SofaGym could be the right fit.
@p-shg @hugtalbot @damienmarchal : I would like to sincerely thank you for your help. We are not a programming group per say. So concepts related to running and integrating softwares are not simple for us. But it has to be done to do the actual work. Your efforts in making these concepts more accessible and friendlier to non -programmers is critical. Thank you. I will be trying SofaGym in a few weeks to see how it does. Will keep in touch.
Kudos

[hugtalbot]

thanks a lot for your feedback @snsvsn
It's great to read the progress of your work. Please keep us updated, and do not hesitate to join us for the next 2022 SOFA Week to present your work! Indeed, let's keep in touch 🙌