Point cloud splats normal direction always pointing up

[jk-ethz]

Hi,

First off, thanks for your great work and research on the field of point cloud visualization in UE4! Vielen Dank :)

I am currently working on a robotic framework in UE4 as my master thesis which should be open sourced later on.
As a part of it, point clouds of RealSense cameras are rendered, at which I could successfully integrate your great plugin.
Unfortunately though, the normals of the points always face in positive z direction (independet of the rotation of the GPU point cloud renderer), which makes part of the cloud seem "transparent" if the splat size is sufficiently small (see image below). I have read in your paper and seen in the DynPCMat that you render the points using a stack on the positive z axis with some well-thought id generation for depth ordering.

Is it intended that the point normals are always facing up? I have seen you use the "Camera Position" in "camera to point dir", so I guess the points should be oriented towards the camera?! The recent fixes have not made any difference for me on this.

The point cloud gets set using GPUPointCloudRenderer->SetInput(Positions, Colors);

I am running the master branch of your repository on UE 4.26.1.
Possibly related to #2.

[jk-ethz]

Interestingly, with larger splat size, the points start to follow the camera orientation by a bit.
But not entirely, as the points get oval (meaning they are not pointing fully towards the player), whereas when viewed from the top or bottom, they are perfectly round.

[ValentinKraft]

Hey, thanks for your message!
yep, this is a well-known problem for me. In fact, I already tried to fix the problem for several hours but could not find the bug. It has to be some small problem but due to the complexity of the whole system it is quite hard to identify the problem.. Feel free to search for the bug yourself! If you want to, I'd recommend to have a look at the "DynPCMat" material. This is the material where the points are taken to their destined position and should be rotated to face the camera. If needed, I could also give some more details about it. Would be nice if you could help solving the problem!

[jk-ethz]

OK, good to know this.
I already experimented with the DynPCMat for quite some time without any success so far. I'll let you know if I find something.
Do you know if this is a problem of the recent UE4 release? Or was this issue present in all UE4 versions?

[ValentinKraft]

No, this problem was present in all UE4 Versions

[roemertimon]

Hey, thanks for your message! yep, this is a well-known problem for me. In fact, I already tried to fix the problem for several hours but could not find the bug. It has to be some small problem but due to the complexity of the whole system it is quite hard to identify the problem.. Feel free to search for the bug yourself! If you want to, I'd recommend to have a look at the "DynPCMat" material. This is the material where the points are taken to their destined position and should be rotated to face the camera. If needed, I could also give some more details about it. Would be nice if you could help solving the problem!

Hi Valentin,

first of all, thanks for your great work!
I'm facing the same problem with point cloud splats where the normals point upwards. I've observed that the points at camera height fully face upwards, while those further away from the camera height are more correctly rotated.

Is there any update on this issue? I've also experimented with the material settings, particularly in the "Hack for Orientation bug" section, and changed a few settings. It seems this section was intended to counter the issue. Could you explain why you add/subtract 0.8 to the points in that section? Any insights or updates would be greatly appreciated.

Thank you!

[ValentinKraft]

Hey Timon, Sorry but I’m too busy with other projects at the moment. You can, however, hire me for a consultation based on my regular charges (80€/h). Let me know if this sounds interesting to you. Best, Valentin Von: Timon ***@***.***> Gesendet: Dienstag, 2. Juli 2024 13:19 An: ValentinKraft/UE4_GPUPointCloudRenderer ***@***.***> Cc: Valentin Kraft ***@***.***>; Comment ***@***.***> Betreff: Re: [ValentinKraft/UE4_GPUPointCloudRenderer] Point cloud splats normal direction always pointing up (#7) Hey, thanks for your message! yep, this is a well-known problem for me. In fact, I already tried to fix the problem for several hours but could not find the bug. It has to be some small problem but due to the complexity of the whole system it is quite hard to identify the problem.. Feel free to search for the bug yourself! If you want to, I'd recommend to have a look at the "DynPCMat" material. This is the material where the points are taken to their destined position and should be rotated to face the camera. If needed, I could also give some more details about it. Would be nice if you could help solving the problem! Hi Valentin, first of all, thanks for your great work! I'm facing the same problem with point cloud splats where the normals point upwards. I've observed that the points at camera height fully face upwards, while those further away from the camera height are more correctly rotated. Is there any update on this issue? I've also experimented with the material settings, particularly in the "Hack for Orientation bug" section, and changed a few settings. It seems this section was intended to counter the issue. Could you explain why you add/subtract 0.8 to the points in that section? Any insights or updates would be greatly appreciated. Thank you! — Reply to this email directly, view it on GitHub <#7 (comment)> , or unsubscribe <https://github.com/notifications/unsubscribe-auth/ADFSMG4WWKUZY22UTNPFPWLZKKEBBAVCNFSM6AAAAABKHKUND2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDEMBSHAYTQNJTGE> . You are receiving this because you commented. <https://github.com/notifications/beacon/ADFSMGYBF6Z6OMAD5GIAWHTZKKEBBA5CNFSM6AAAAABKHKUND2WGG33NNVSW45C7OR4XAZNMJFZXG5LFINXW23LFNZ2KUY3PNVWWK3TUL5UWJTUDJRL6G.gif> Message ID: ***@***.*** ***@***.***> >