convert_2d_to_3d produces Value Errors

[nliedmey]

Hello,

I'm using the convert_2d_to_3d function to bring my 2D-Coordinates and the belonging depth to the 3D-Coordinate system of the camera. I am taking the depth value from looking up the coordinate's depth value in the capture.transformed_depth image.

Now when throwing the coordinates and their depth values into convert_2d_to_3d function, i get many valid 3D coordinates, but also "some" Value Errors. For 208 Frames, with each having 33 Landmark-coordinates, i get 6864 valid and 352 non-valid 3D-Coordinates.

Is this a normal error count or is there a way to optimize this? What is the best color resolution & (WFOV) depth mode combination for this purpose or is this not important for this function?

I looked up the SDK documentation below but this was not very helpful for me:
https://microsoft.github.io/Azure-Kinect-Sensor-SDK/master/group___functions_ga664602bdb48dab38117a6c1d14b880de.html#ga664602bdb48dab38117a6c1d14b880de

Thank you for your helping me out :-)
Greets

[nliedmey]

Ok i have found out that missing depth values, which return a value of 0mm, are the reason for ~80% of the non-valid 3D-coordinates. I'll try to minimize them though interpolation.
But what could be the reason for the other non-valid 3D-coordinates?

[JJLimmm]

Hi @nliedmey , i am having some trouble in using the API convert_2d_to_3d(). Could you show me how did you write the code and input arguments for using this API?

[lpasselin]

Hi, I am not sure what else causes invalid values.
Are you able to repeat the errors using the point cloud viewer example?
https://github.com/etiennedub/pyk4a/blob/master/example/viewer_point_cloud.py

[JJLimmm]

Hi @lpasselin , Invalid values are caused by the SDK as mentioned in the online documentation.
Are you able to show a snippet of your code that calls this API? convert2d_to_3d()

[nliedmey]

Sorry for late response.

I don't know why but i got many depth measurements with value of 120mm that seem to produce 3D calculation error through this SDK..
120mm is outside the operating range of WFOV_Unbinned mode and no realistic values for my use case anyway.

Is this a known issue or has someone else the same problem?

[lpasselin]

Sorry I have not seen this before.

The only time I have seen weird values is when my device was sitting on a table, not a tripod. It was sending back weird values due to the obstruction of the table which was in the field of view of the camera!