Processing Images Using pose_world_landmarks in MediaPipe to Extract 3D Coordinates and Map Them to Unity

[Omaryehya21]

I am currently working on a project where I need to process images to extract 3D coordinates of a human pose using MediaPipe's pose_world_landmarks function. I would appreciate guidance on the following:

Is it possible to extract the 3D coordinates from images using pose_world_landmarks in MediaPipe? If so, could you please provide any examples or references on how to do this?

Once the 3D coordinates are extracted, what is the recommended method to map these coordinates to Unity for further use? Are there any specific transformations or tools I should be aware of when integrating this data into Unity?

[kuaashish]

Hi @Omaryehya21,

MediaPipe does not offer any official support/library/module for mapping the pose landmarks to Unity as such. However,
there are plugin from the community is quite up-to-date which might help in this context But We don't have a plan to support Unity yet and does not provide support in how to use them.

Thank you!!

[Omaryehya21]

Hi @kuaashish,

I greatly appreciate you taking the time to share this information. I am aware of the existing plugin; however, I have decided not to utilize it, as my project is focused on mobile devices. To ensure optimal performance and minimize the computational load on these devices, I am implementing MediaPipe on a server instead.

[kuaashish]

Hi @Omaryehya21,

Thank you for providing additional details. Currently, we do not have plans to support Unity in the near future. Due to limited bandwidth, we believe we may not be able to assist with this at the moment. However, Please allow us some time to confirm with our team, and we will let you know if there is anything further we can assist with.