Virtual Camera

[crunchwrap89]

The problem

When i was trying to add the Transformation Controls from the Three.js utils and attach it to a 3D object i stumbled upon some issues.

Attaching the TransformationControls to the 3D Object works fine, i can see the Controls and the position and axis is correct after the latest major update to the package. However.. interacting with the TransformationControls does not work. Nothing happens when clicking/dragging the controls.

When dragging the controls it should translate the objects position on the map.
This currently doesnt work due to how the current camera is positioned.

Feature request

After discussing this with Martin we concluded that the solution to this issue would be to add a virtual camera, that "sees" the same as the actual maps viewport sees. Then that virual camera can be passed to the TransformControls and then the internal raycaster of the Transform Controls should work as intended.

There are more use cases for adding a virtual camera that "sees" the same as the actual maps viewport.
Here is a list of some that i can come up with:

Usecases

• Ability to frustum cull objects outside of the cameras viewport. (A Very great performance improvement, that will allow building a significantly larger 3D world with higher frame rates.)
• Ability to perform Animations on objects when they enter or leave the Viewport.
• Ability to trigger any type of function or world event when the a specific object enter / leaves Viewport.
• Ability to use object.lookAt(virtualCamera.position) which itself has alot of use cases, such as letting floating player names and texts always focus on the virtual camera no matter which rotation, tilt or zoom is being used.
• Ability to use the controls that Three.js has such as TransformationControls, there are many of them.

Here is a suggested calculation for setting the position of the Virtual Camera on each frame update.
It Almost works, there is one small factor missing but i could really use your guys help to figure out what piece of the puzzle is missing.

function updateVCam() {
    if (!_camera || !_scene || !_renderer)
      return;

    _vCamMvpMatrix.copy(_camera.projectionMatrix)
    _vCamInverseMvpMatrix.copy(_vCamMvpMatrix).invert()

    _v4.set(0, 0, 1, 0)
    _v4.applyMatrix4(_vCamInverseMvpMatrix)
    _v4.multiplyScalar(1 / _v4.w)
    const cameraPosition = new Vector3()
    cameraPosition.set(_v4.x / _v4.w, _v4.y / _v4.w, _v4.z / _v4.w)

    const c = new Vector3(0, 0, 0).applyMatrix4(_vCamInverseMvpMatrix)
    const x = new Vector3(1, 0, 0).applyMatrix4(_vCamInverseMvpMatrix).sub(c).normalize()
    const y = new Vector3(0, 1, 0).applyMatrix4(_vCamInverseMvpMatrix).sub(c).normalize()
    const z = new Vector3(0, 0, -1).applyMatrix4(_vCamInverseMvpMatrix).sub(c).normalize()

    const near = new Vector3(0, 0, -1).applyMatrix4(_vCamInverseMvpMatrix).sub(cameraPosition).length()
    const far = new Vector3(0, 0, 1).applyMatrix4(_vCamInverseMvpMatrix).sub(cameraPosition).length()

    _m4.makeBasis(x, y, z)

    _vCam.position.copy(cameraPosition)
    _vCam.quaternion.setFromRotationMatrix(_m4)
    _vCam.updateMatrixWorld()

    _vCam.matrixWorldInverse.copy(_vCam.matrixWorld).invert()

    const projectionMatrix = new Matrix4().multiplyMatrices(_vCamMvpMatrix, _vCam.matrixWorldInverse)

    const fov = (2 * Math.atan(1 / projectionMatrix.elements[5]) * 180) / Math.PI
    const aspectRatio = projectionMatrix.elements[5] / projectionMatrix.elements[0]

    _vCam.fov = fov
    _vCam.aspect = aspectRatio
    _vCam.near = near
    _vCam.far = far

    _vCam.updateProjectionMatrix()

    _renderer.render(_scene, _camera)
    _renderer.resetState()
  }

[crunchwrap89]

Hi,
I made an update to this issue description, added some useCases for the suggested Virtual Camera and also added an "Almost" working code example of how it might work that was calculated by @usefulthink

The above code example works fine for utilizing object.lookAt(_vCam.position), but there is some small factor missing.
Hopefully you can help and this might be added to the package. :)

Love this package btw! <3

[crunchwrap89]

Any chance this will be made? :) Im still looking for ways to implement tranform controls on the webgloverlayview.

Br
Marcus

[MstrD]

I'm pinging this thread because I am also needing some sort of virtual camera attached to my project. In my specific case, I am trying to work with CSS3DRenderer, but I can't seem to place my instances of CSS3DObject in the correct place; despite I already found how to get the virtual camera position, like @crunchwrap89 reported, there's a small factor missing in the calculations, and I can't seem to find it.
Are you already trying to implement this feature, @usefulthink ?

Any sort of feedback would be appreciated.
Thanks in advance!