BlockifyVR (Virtual Reality)

[Brackets-Coder]

This is a very-much-still-work-in-progress Virtual Reality extension I've been working on since January of 2023.

The end goal is to have:

• Little to no performance overhead with the extension.
• Cross-platform compatibility for major 6DOF headsets supported by A-frame (Oculus, Vive, and Windows Mixed reality)
• An easy to use system that's coherent and efficient
• Works well with any 3D engine

Right now, it meets most of these standards, but there's always room for improvement.

Here are some things to note:

• It includes a local copy of the A-frame JS library, version 1.6.0, MIT License. This is for VR Management, rendering, and ease of development as I'd have to write the entire thing from scratch using WebXR, WebGL, and the Gamepad API otherwise. Setting up the VR boilerplate and stereoscopy and other things would be a pain without it.
• Gallery thumbnail, documentation, and sample projects are not currently available and will be worked on as I get nearer towards release
• The rendering uses a plane fixed to the camera which directly mirrors the scratch stage with an unshaded texture each frame. This is relatively performant and works well, but there could be better solutions to look into. The plane is placed 1 meter away from the camera and A-frame uses stereoscopic rendering to reduce eye strain. Note that the plane automatically scales to the size of the camera (display) while attempting to maintain the ratio of the stage. I use this instead of drawing directly to the canvas because it won't let me and I would have to re-write the renderer that's already tailored for VR.
• Sorry the code's a bit messy

Upcoming release plan:

v1.0-pre-alpha: Earliest version, showcased in the video above. Minimal functionality, proof-of-concept version. Not available here. Not open source.

v1.0-alpha: work-in-progress. Expect significant bugs and poor performance. Should not be used.

v1.0-beta: Current version. Cross-platform compatibility, major bug fixes, and optimizations have been added.

v1.0-release-candidate: Feature complete. Meets standards of full release with minimal issues, includes preparations for full release. This means gallery thumbnail, sample project, documentation, etc.

v1.0: First release. Feature complete and meets all standards of great performance, very few issues, cross-platform compatibility, and intuitive design.

v1.5: Bugfixes and optimizations. Possibly a small feature or change, such as controller vibrations.

v2.0: Big update. Most likely something like hand tracking support for some platforms. Also will include some minor bugfixes and optimizations.

v2.2: Bugfixes and optimizations.

Active Development

v1.0-beta

• One more bug to fix before moving into v1.0-release-candidate
• Prep for release candidate

[hammouda101010]

that looks cool

[Brackets-Coder]

that looks cool

It looks cool on the flat screen right now, but it's not so great when you try it. You'd probably get motion sickness. I'm still working on it.

[hammouda101010]

that looks cool

It looks cool on the flat screen right now, but it's not so great when you try it. You'd probably get motion sickness. I'm still working on it.

sadly i dont think that's possible to stop this.

[Xeltalliv]

Does this show the same scratch 3D perspective to both eyes?

[Brackets-Coder]

Does this show the same scratch 3D perspective to both eyes?

A-frame has stereoscopic rendering built in and I've enabled it in the rendering settings, so no. This fixes some issues with visualizing depth, so the eye offset allows your brain to process the image with more clarity. Even if the matrices aren't used in the renderer, it should properly use the correct eye offset. Why do you ask, just out of curiosity? If you'd like I could add a block that disables or enables this feature, in case the Scratcher wants to build that framework themselves, which I'd doubt.

Keep in mind this is still a work in progress and most of the code is many months old so it's not the most efficient way to do VR.

[Xeltalliv]

Why do you ask, just out of curiosity?

Yes. At one point I tried to extend my AR extension to support AR headsets and VR as well, but the multiple eye rendering has been the main thing holding me back. I was just curious to know how you solved it.

[Brackets-Coder]

Why do you ask, just out of curiosity?

Yes. At one point I tried to extend my AR extension to support AR headsets and VR as well, but the multiple eye rendering has been the main thing holding me back. I was just curious to know how you solved it.

I believe that there can be two XRViewerPose matrices for each eye if available, according to Mozilla Docs. You might want to look into that again. It'd be great if Augmented Reality could support AR headsets such as the Oculus Quest 3 in addition to just mobile touchscreen devices. There should be a views array that can do this. An example can be seen here. Just a suggestion though. I simply think having AR headsets would be a nice update.

[Brackets-Coder]

@Xeltalliv Your extensions are good quality and you seem to know what you're doing with WebXR.

I've looked at the code like 100 times. Do you mind taking a peek at the left-controller-manager and right-controller-manager A-frame components? Pressing any button significantly drops the FPS even without calling the event hat block. Additionally, pressing any button on the right controller returns its left controller counterparts except for X, Y, A, and B on Oculus Quest.

Just asking for another pair of eyes, that's all. I've asked other developers I know and I've asked ChatGPT and I've referenced the A-frame docs and everything looks fine to me, I just can't narrow down the issue.

[Xeltalliv]

I'll take a look when I have time. Thought it will probably be hard because the only kind of VR I have access to is phone-based 3DOF no controller VR.

Also, do you mind giving some links, so that I can faster figure out where to look?

[Brackets-Coder]

I'll take a look when I have time. Thought it will probably be hard because the only kinf of VR I have access to is phone-based 3DOF no controller VR.

Also, do you mind giving some links, so that I can faster figure out where to look?

Ok, no problem! I wasn't asking you to test the extension, and Google Cardboard-like devices aren't supported by it anyway, so you're fine, don't worry about it.

As for the links, do you mean the XRViewerPose Mozilla Developer Docs links or the links to my controller code? Because as of right now the latest version of my extension is only available on private GitHub repo accessible only to me. You could always just view the added BlockifyVR.js commit here and scroll down to the [hand]-controller-component blocks of code. Just a note though, it does include the entire minified A-frame library, so it might lag your browser a bit.

[Brackets-Coder]

I'll take a look when I have time. Thought it will probably be hard because the only kinf of VR I have access to is phone-based 3DOF no controller VR.

Also, do you mind giving some links, so that I can faster figure out where to look?

As for VR testing, you wouldn't really need a VR headset. I've used Meta Quest's Immersive Web Emulator extension for testing when I didn't care to load up the entire headset.

[Drago-Cuven]

ah yes, I remember seeing a trailer for this on scratch. I wanted to help, but now I can't

[Drago-Cuven]

nvm I think I can
i'm Martinelplayz on scratch btw

[Thebloxers998]

could i try the extension?

Hi siskka7 it's me Thebloxers998

[siskka7]

Thebloxers998

hmm https://github.com/TurboWarp/extensions/blob/2837a7adfbfc2e3aba49108003686affa1cb7beb/extensions/MasterMath/BlockifyVR.js

[Brackets-Coder]

On the meta quest controlers are 3 lines button. Some games are using it.

Not a supported event listener by A-frame, but I could look into adding it. Might be a bit more difficult though. Other platforms support a similar button, but Quest doesn't.

Right now I'm kind of focused on fixing bugs and working on a release candidate rather than adding new features, but that doesn't stop small features like this from making it into beta.

[Thebloxers998]

How do people get the extension

[Brackets-Coder]

How do people get the extension

I'll try to post a link after every commit for the latest file. You can see the commit above here.

Alternatively, you can also just use this Simple3D Engine VR.zip file for testing as it can be imported into Turbowarp like a normal scratch project. It has the commit above included, but it will not be automatically updated as I push more commits here.

[Brackets-Coder]

Known bugs that need fixing before v1.0-release-candidate:

• Changing any Turbowarp setting (language, block color, light/dark mode, etc.) after the extension is loaded breaks the editor
• Attempting to create a new my block after the extension is loaded crashes the editor (creating variables and lists still works fine)

The above issues both return TypeError: Argument 1 ('child') to Node.removeChild must be an instance of Node.

• Texture resolution scaling. Pressing Oculus button in immersive experience causes texture to zoom in, lower resolution, and perspective projection to break. I'll have to narrow down whether this is a demo project issue or an actual JavaScript issue with the extension.

I have no idea how to fix these.

These issues will take some time, so I expect Beta to last at least a few more weeks before they can be fixed.

[Thebloxers998]

Known bugs that need fixing before v1.0-release-candidate:

• Changing any Turbowarp setting (language, block color, light/dark mode, etc.) after the extension is loaded breaks the editor
• Attempting to create a new my block after the extension is loaded crashes the editor (creating variables and lists still works fine)

The above issues both return TypeError: Argument 1 ('child') to Node.removeChild must be an instance of Node.

• Texture resolution scaling. Pressing Oculus button in immersive experience causes texture to zoom in, lower resolution, and perspective projection to break. I'll have to narrow down whether this is a demo project issue or an actual JavaScript issue with the extension.

I have no idea how to fix these.

These issues will take some time, so I expect Beta to last at least a few more weeks before they can be fixed.

It says "must be an instance of Node" I think you should make it an instance of node

[Brackets-Coder]

Known bugs that need fixing before v1.0-release-candidate:

• Changing any Turbowarp setting (language, block color, light/dark mode, etc.) after the extension is loaded breaks the editor
• Attempting to create a new my block after the extension is loaded crashes the editor (creating variables and lists still works fine)

The above issues both return TypeError: Argument 1 ('child') to Node.removeChild must be an instance of Node.

• Texture resolution scaling. Pressing Oculus button in immersive experience causes texture to zoom in, lower resolution, and perspective projection to break. I'll have to narrow down whether this is a demo project issue or an actual JavaScript issue with the extension.

I have no idea how to fix these.
These issues will take some time, so I expect Beta to last at least a few more weeks before they can be fixed.

It says "must be an instance of Node" I think you should make it an instance of node

I think I figured that part out

[Thebloxers998]

Known bugs that need fixing before v1.0-release-candidate:

• Changing any Turbowarp setting (language, block color, light/dark mode, etc.) after the extension is loaded breaks the editor
• Attempting to create a new my block after the extension is loaded crashes the editor (creating variables and lists still works fine)

The above issues both return TypeError: Argument 1 ('child') to Node.removeChild must be an instance of Node.

• Texture resolution scaling. Pressing Oculus button in immersive experience causes texture to zoom in, lower resolution, and perspective projection to break. I'll have to narrow down whether this is a demo project issue or an actual JavaScript issue with the extension.

I have no idea how to fix these.
These issues will take some time, so I expect Beta to last at least a few more weeks before they can be fixed.

It says "must be an instance of Node" I think you should make it an instance of node

I think I figured that part out

Ok

[Brackets-Coder]

I'm committed to keeping you guys updated, so here we go again:
I suspected that the editor-crashing issues mentioned above were due to how I was injecting the A-frame scene HTML into the DOM as TW doesn't generally like HTML/CSS/JS evaluation. However, with the help of ChatGPT (because I couldn't find the prepend() function on W3Schools), I've managed to change one line and inject the A-frame scene code into the top the <body> instead of the bottom of the <head>. This seems to fix the editor-crashing issues, but I'll have to do more testing to see if the rest of the extension's functionality remains intact before pushing another commit here.

[Thebloxers998]

I'm committed to keeping you guys updated, so here we go again: I suspected that the editor-crashing issues mentioned above were due to how I was injecting the A-frame scene HTML into the DOM as TW doesn't generally like HTML/CSS/JS evaluation. However, with the help of ChatGPT (because I couldn't find the prepend() function on W3Schools), I've managed to change one line and inject the A-frame scene code into the top the <body> instead of the bottom of the <head>. This seems to fix the editor-crashing issues, but I'll have to do more testing to see if the rest of the extension's functionality remains intact before pushing another commit here.

Nice 👍

[Thebloxers998]

Ideas I have
Event: when [object1] touches [object2]
What it does: Triggers an event when two VR objects collide.

Block: animate [object] with [animation]
What it does: Applies an animation to a VR object (e.g., bounce, spin).

[Thebloxers998]

Ideas I have Event: when [object1] touches [object2] What it does: Triggers an event when two VR objects collide.

Block: animate [object] with [animation] What it does: Applies an animation to a VR object (e.g., bounce, spin).

You could use keyframes

[Thebloxers998]

Ideas I have Event: when [object1] touches [object2] What it does: Triggers an event when two VR objects collide.
Block: animate [object] with [animation] What it does: Applies an animation to a VR object (e.g., bounce, spin).

You could use keyframes

Just remember, it will take a long time!

[Brackets-Coder]

Those ideas will not be added. This extension isn't a 3D engine, it's a framework for integrating VR functionality into other already existing 3D engines. Because of this, collision detection and animations should be managed by the project, not the extension.

[siskka7]

I think you can add key simulation to controlers

[siskka7]

And if vr is in standby?

[Thebloxers998]

And if vr is in standby?

???

[Brackets-Coder]

Let's try to avoid minuscule feature updates while in beta unless they improve the core functionality of the extension, like button touch detection. For now I'm focused on bug fixing, optimizations, and preparing for release.

[siskka7]

Change display resolution that could make something faster idk or turbomode mode

[siskka7]

And if vr is in standby?

???

power button

[Brackets-Coder]

Progress Update

I know you’re patiently waiting, but I’m taking longer than expected to complete my job.

I’m finalizing some things and working on a major issue before moving into release candidate mode. I hope to do this before the end of the year, but no promises. I’ve added blocks for managing rendering settings, including high refresh rate, stereoscopy, foveation level, and exposure. The last exposure option mitigates an issue where the immersive experience stage texture was brighter than the original source image. I’m still testing, so there’s no commit yet.

It’ll likely be several more weeks (or months) before we open this PR for review.

The plan is to prepare this release as soon as possible without losing my sanity. Once ready, the PR will be opened for review. During the review period, the version will be v1.0-release-candidate. Once approved, it’ll be v1.0 (the first public release).

To the reviewers:
Due to the extension’s size and complexity, I’d like as many available and willing reviewers as possible. This will likely take longer than usual, so more reviewers are welcome. I’d prefer a timely release, but not rushed. Some code may be outdated or written by me two years ago. Any recommendations, suggestions, or changes to preserve best practices are welcome. I may also request specific reviewers, like Xeltalliv, who has experience with WebXR. Thanks for understanding.

TL;DR: I'm still working on it and making updates, but it’ll take a bit longer than expected.

[Brackets-Coder]

I think I may have found a theoretically plausible solution to the display scaling issues, but I'll have to check it out tomorrow. More likely it won't work than it will, but it's worth a shot.

[siskka7]

i have a simple idea from Static Dropdown to Circle Dropdown (from square to circle dropdown ).for easier coding.in circele dropdown i can put variables in it.

[Brackets-Coder]

i have a simple idea from Static Dropdown to Circle Dropdown (from square to circle dropdown ).for easier coding.in circele dropdown i can put variables in it.

Some blocks have these, other blocks do not. For example, button blocks do, but enter/exit VR does not. This really depends on the kinds of functions the blocks do and when a variable would be most needed. If you have specific blocks that need reporter input capabilities, let me know, I'll add it if I deem it necessary.

[Brackets-Coder]

I'm still working on attempting to resolve this issue. I've already made several different tests of probable solutions and none of them worked. I'm back to the drawing board and will need to take a short break before I put my brain cells to work again.
probably just a few days

[siskka7]

tell when you will start work

[Brackets-Coder]

tell when you will start work

Probably after Christmas break. I just need a week or two to clear the fog in my brain and then I'll get back to it.
Maybe I might even work here and there during Christmas break