How to do the triangulation

[victkid]

Hi,

I'm trying to reconstruct the 3D position from uv. I first equation (1) from the paper ((u, v, 1)^T = P(X, Y, Z, 1)^T) to get (X, Y, Z)_2 and (X, Y, Z)_3 for camera 2 and 3. Then I don't know how to use the camera position to get the final X, Y, Z location.

[Adamink]

Hi, @enrico-c, I am also trying to reconstruct 3D position from uv. However, I can't get the correct 3D position. I am wondering if you could point out what I have done wrong or release the code of triangulation.

def triangulation_test(self):
    poses, p_mats = self.triangulation_getitem(0)
    cam_num = len(poses) # 2
    # poses: (2, 2, 13)
    # p_mats: (2, 3, 4)
    b = np.empty((cam_num * 3, self.num_joints)) #(6, 13)
    for cam in range(cam_num):
        b[cam * 3] = poses[cam][0]
        b[cam * 3 + 1] = self.image_h - poses[cam][1] # flip
        b[cam * 3 + 2] = np.ones((1, self.num_joints))
    A = np.concatenate(p_mats) #(6, 4)
    # x = (A'A)^(-1)A'b, (4, 13), least square method
    x = np.matmul(np.matmul(np.linalg.inv(np.matmul(A.T, A)), A.T), b)   
    pred_skeleton = x[:3]
    return pred_skeleton

[ShallowWill]

Hi @enrico-c @tobidelbruck ,
Could you please provide some help here?
It would be much better if you can release the corresponding code.

[ShallowWill]

Hi @Adamink , did you solve the problem of reconstructing 3D position from uv?

[enrico-c]

This is the algorithm we used to triangulate from two cameras:

a) Back project each 2D point to 3D space using the pseudo-inverse of the camera projection matrix [1].
b) For each camera, draw the ray that passes through the projected point and the camera center [1].
c) Find the intersection of the two rays using least squares (due to non idealities, the lines do not actually intersect in space) [2].

References:
[1] - Book "Multiple View Geometry in Computer Vision" by Hartley and Zisserman. See chapter 6.2.2 for an explanation of back projection.
[2] - https://stackoverflow.com/questions/52088966/nearest-intersection-point-to-many-lines-in-python/52102743

Another useful resource:
https://www.cc.gatech.edu/~hays/compvision/proj3/

Please have a look at the notebook Eval_2D_triangulation_and_3D_tutorial.ipynb for the code details.