Facets, Containers and Workspaces

[Frooxius]

Hello everyone!

I'm currently designing the facet system, which will be the bread and butter of the new Radiant UI and I wanted to gather some feedback and thoughts from you, the community.

---

Currently there are three main concepts in this system:

• Facets
  -- Essentially contained pieces of UI. Think of a little box that can be slotted into different places.
  -- It can contain anything, typically standard UI, but 3D objects can be included as well
  -- They can be anything as simple as FPS counter, to complex ones like inventory views, chat windows and inspector views
  -- Since you can include arbitrary 3D objects in them as well, you can for example build things like a private mirror/camera that toggled on a press of a button. The facet is only a way of "wrapping" what you build into something that fits into the whole UI system

• Containers
  -- Structures that hold one or more facets. This can be something like a grid, allowing multiple facets to be arranged in a grid pattern, partitioned canvases or even some wonky layouts (like arranging them in a circle and whatnot) - essentially they just define how are facets arranged/sized within them
  -- Importantly, containers themselves can be facets too, allowing for nesting. E.g. creating a facet that has a bunch of tabs to switch between "screens" that can contain other facets. This will allow for high flexibility for organization of the UI
  -- There will be some key points where you can find containers by default - the new dash, your viewpoint, playspace and your hands (more on this later)
  -- There will be a few default types of containers provided, but I want to add ability to build your own as well. It will be a good way to expand the system into the future

• Workspaces / Loadouts
  -- A workspace is a specific "loadout" of facets, both in your private UI and in public one. This includes which facets are there, any container facets and their positioning/layout
  -- You'll be able to switch between different loadouts of your private UI. This way you can have loadouts optimized for different situations, like social, working and so on.
  -- Private UI loadout can also be tied to settings (e.g. default laser status or item shelf visibility) as suggested by @DeliriousJax before. This would replace the old "physical mode", allowing you to make an "immersion" loadout
  -- Each tool you equip in the world will come with its own workspace/loadout. This includes facets giving you detailed control over its functions - e.g. editing curves and properties for brushes, controlling modes and triggering actions with the developer tooltip and so on. You can rearrange those facets and customize your workspace for each tool, which will be saved and loaded when you equip it again
  -- This can apply to other things in the world as well - e.g. if you're building a game, you can build facets that show things like health, ammo or swap weapons and such and let users customize the workspace
  -- Workspaces aren't mutually compatible though - you can only have one at the time. However the worldspace (tools, games...) and private UI workspaces are independent of each other.
  -- If you open your private UI workspace (this would be bound to the same button as the world switcher / voice controls are on now) the worldspace one will be temporarily hidden.
  -- You'll be able to make as many workspaces as you like and clone them

---

Facets can exist as standalone pieces in the world too or in containers that are in the world, so they can be used for world/UI. However for private UI, there will be several anchor points, where you can place the facets and container/faces to customize your private UI:

• The new Dash
  -- It will have the same/similar key binding as the the current dash - single button to open/close.
  -- However instead of just being a few buttons, it's going to be a curved screen, with the buttons at the bottom switching between views (think of virtual desktops).
  -- Each view is a grid layout where facets can be placed. Switching to different views (e.g. Friends, Inventory, Worlds, Settings...) will move the screen to them, rather than popping out a new window, preventing new dialogs from appearing everywhere (however you'll be able to "pop out" the dialog into a window if you want.
  -- You will be able to add/remove those views, to create custom screens (e.g. you could make one has facets that show the latest news from the web, weather and such)
  -- It'll likely include a few containers at the top and sides that will be independent of the screen switches, for things like volume meter/switch, FPS counter, NCR counter and so on
  -- By default this will be the only place where all essential facets will be pre-arranged. This will keep it simple for new users, as they only need 1 button to access all the essentials (switching voice, changing audio, finding sessions, friends and so on)
• Hands
  -- By default two containers - one on the front of the palm and one on the back. You can resize those and adjust their positioning
• In front of the hand
  -- Positions itself in front of your hand when opened, but stays there, just like the context menu does. Useful for very quick access to common options.
  -- Should it dismiss itself if you move too far away? E.g. to interact with a different facet
  -- Should there be two for each hand, with both opening at the same time?
• Viewpoint
  -- Moves with your head. Useful for small facets, like FPS counter, unread messages count and such (although technically you could place anything there)
• Playspace
  -- This one is independent of your avatar and stays in place as you move around. There are a few options though - it can auto-align if you move too far, the way the current dash does or it can essentially stay pinned to your room, meaning you can walk far away from this UI or put it behind you. Which would you prefer? Or both?

One of the anchor points that was considered were also elbows/forearms. However this is tricky, as most users don't have elbow tracking, so it would need IK, which doesn't exist in userspace and it's specific to a particular avatar skeleton. I don't see too much use in this right now, as you can just make the hand anchor bigger. This is something we could save for later in the future if there's a demand for it.

For world-space UI you could put containers anywhere though, there would be no restrictions there. This only concerns the private UI anchors.

However you'll be able to define the offsets for the private UI workspace on each avatar, so it better fits the particular avatar that you're wearing. As interesting possibility some avatars could even have unique anchors, which will show private UI facets that are unique to that particular avatar (or group of avatars). With addition of some features like sending impulses from private UI (userspace) to the current world avatar this will let you make controls for your avatar that only you can see and that won't affect other users in the world.

---

There are a few other points and design ideas:

• By default, you won't be able to move facets. To do that you'll have to trigger the facet editing mode. This could be done by pressing both "system" buttons at once (the one that opens/closes dash and one that shows/hides private workspace). It's quick enough to not be too annoying and also not too complicated or accidentally triggered
• When in editing mode, containers can be resized too. This will include the dash, so you can adjust the curvature, range, height, width and so on
• Facets can be sized. E.g. you can grab a facet and drag it over area of a container to place it there at specific size. This will let you utilize the space better, or make some UI pieces more compact if you're already familiar with them (e.g. making it more compact will hide button descriptions and labels and show only icons)

---

Overall, there's a lot more to this system, but these should be the main points summed up, hopefully in a lot more coherent manner than in the notes. Here are some of my notes from the design process upon which this is based on, it covers a lot of other stuff that I haven't talked about yet.

I'm going to leave it here now, as this post is already getting pretty long. Anyway let me know what you think so far or if you have any questions and I'll include extra information or discuss possible options!

[shiftyscales]

Will it be possible for users to share their loadouts/workspaces with each other?

[Frooxius]

Yes, I want that to be a feature. Including for the new dash.

[ghost]

Agree with the concept. Will look forward to it.

[TehTurk]

So there was one thing I wanted to ask, Will this open up local userspaces for users to put logix systems, or tools, or even all kinds of things in there? Or would it be only accessible from edit mode in terms of Facets? Currently you can access and see things in there, but I feel like access to another root, like userspace would open alot of possibilities :) Because even certain game systems, or even user data/etc would be really handy if this was a thing or there were ways to allow communicating between them.

[alexderpyfox]

For the viewpoint facets can it be so it's smoothed movement like with the lasers? So when it follows the head it doesn't feel like it's stuck to it but smoothed so it bounces a little when you turn your head else it will be constantly moving around/vibrate by micromovements of the body so smoothing for it would be really handy 😋

[Frooxius]

@TehTurk Yes! Well in some way. The way to bring facets into userspace will be more controlled. You'll be able to build them as any other object in the world, and then transfer them to the userspace to place in your private UI.

For the tools, you'll be able to do things like put a "mini-inventory" on your hand for example and equip tools from there into the world (assuming you're allowed to spawn things out).

Keep in mind that the userspace is actually a separate world, so you can't access things in the world you are in directly from there.

@alexderpyfox Yeah that might be a good idea, added a smoothed anchor point.

[TehTurk]

Hmm okay, I was aware from most of those concepts from some of the earlier changes and discussions. I thought maybe we'd have more Userspace to MainRoot interactions on top due to the boon it could provide. Plus in alot of aspects considering some Logix might not run too well in Userspace vs MainRoot. And Yes! I understand that part, but I know the layer like aspect of it is really handy. How would say the interaction events between the windows and the tools end up being :0, or would most tools have placeholder objects underneath them similar to the pose system.

[KacperLa]

Playspace
-- This one is independent of your avatar and stays in place as you move around. There are a few options though - it can auto-align if you move too far, the way the current dash does or it can essentially stay pinned to your room, meaning you can walk far away from this UI or put it behind you. Which would you prefer? Or both?

Both! I can imagine both being useful depending on the situation. Having a toggle would be ideal in my opinion.

Thanks for letting us put in our two cents!
Great Work!!!

[shiftyscales]

Running through the list of questions proposed by @Frooxius:

Regarding hand-menus:
Should it dismiss itself if you move too far away?

The context menus are a good template for how to handle containers in front of the hand. Have them dismiss when the hand is lowered/moved away from them- also close them when the button is pressed again like the context menus.

Should there be two for each hand, with both opening at the same time?

By default I think both hands should correspond to the same menus/there should only be one open at the same time- again, like the context menu. But perhaps there could be a way to define handedness? E.g. primary hand/off-hand based on the handedness value in the settings?

Regarding Playspace:
Which would you prefer? Or both?

Both is good, and both are needed. If we use software like XSOverlay, and OVRToolkit as examples- people want the ability to pin windows to their hands, head, and playspace. E.g. place a Discord window, or Twitch chat, or web browser somewhere within your physical playspace.

As you move away it could either stay visible or fade away based on distance.

In addition to fading based on distance, you can have them fade in/out based on if you are looking at them- particularly useful for forearm/wrist-mounted windows/UI. Visible when you need it, but not a distraction when you don't.

General head direction can be used for most HMDs, but for eye-tracking capable HMDs, precise eye data can be used to determine gaze for when to make UIs visible (this could even be useful for the 'always visible' containers. Use eye gaze to activate them, then fade away when they're not looked at.

[TehTurk]

Ah! I didn't see those, but you've hit most of the answers on the tip of the nose @shiftyscales :). But I think the behaviors should be adjustable as much as possible :0, so auto-dismissal or no dismissing would be a plus, giving more tools for people to play, preference, create, and make things is always a plus in my book.

In Terms of Anchor Points, something of a Chest Target? I feel like if we defined a slot or a bodynode slot of some kind that would be applicable too.

[shiftyscales]

I think that would be taken care of by anchoring to the playspace/user-root (as was noted by Frooxius, sort of like the current dash.) Stays in your geneal vicinity, but won't move until you get past a certain threshold away from it. Having a chest target wouldn't work for the same reason he outlined with elbows/other targets- most people don't have full body trackers, and so it would rely on the IK system, which could make it feel strange when it doesn't correspond to your actual body movements.

And I agree, options are good, hence 'default'. Make the specified behaviors default behaviors, but open the options up to allow things like auto-dismissal, etc. for general flexibility/user-preference.

[BlazeVR]

This sounds very great, thank you for the transparency and hard work as always!

I'm curious if there are security measures planned against malicious users when we have more access later on the new UI system.

Can we mark specific parts as critical stuff so it's only accessible from the owner (maybe blurred out for everyone else if it's part of a bigger custom UI and brought to worldspace)? It's useful to show your own UI to guide users of something but keep your sensitive stuff out you might have forgotten to hide/close.
Or for example against people forcing (via logix or third-party tools) to see my custom UI containing sensitive stuff like chats, profile email etc. Sorry if this has been answered already in a way before!

[Frooxius]

@TehTurk I'm not quite sure I understand what do you mean by "Userspace to MainRoot interactions", can you elaborate on that? What exactly is "MainRoot" in this context?

---

Regarding the fading, this would be a good general addition, like something you can add to each anchor. However it would be most likely be based on scale (so it would scale up and down, the way the world switcher does), because it's not possible to easily do alpha fade for arbitrary objects.

In general that would be a good approach - add general behaviors to anchors. E.g. one that fades it in/out based on where you're looking or pointing. That way the whole system is extensible and customizable like everything else in Neos.

---

@TehTurk Hmm not sure about Chest target. It has the same issues as Elbows, most people don't have a chest tracker, so it would need to be somehow driven with IK and that wouldn't actually match your chest. Maybe in the future, but probably not now.

Although we could do the one that's based on your player position in general, rather than chest specifically, so it moves with you.

---

Also more on the anchor in front of hand. There's also one issue to consider - how it it invoked? We only have one dedicated button we can allocate to triggering the private UI workspace. This would trigger all of it at once, meaning the container would appear in front of either hand when you press it, along with all others.

This means that even if you want to just interact with one on your palm, the in front of hand one would appear as well. You might just simply not click anything on it (or even move your hand away, which will dismiss it), but it might be weird that it pops up every time.

Alternatively we could provide some gestures to deal with this - e.g. holding grip while triggering the private workspace would differentiate.

What do you think?

---

@BlazeVR Security is a bit separate topic to this, but related too. For now, there will be a controlled way to bring facets into your userspace, so you should bring only ones you trust.

I do want to build a similar process, where the facets will have to ask for certain privileges, based on which functions they contain - e.g. any sort of networking would be a red flag for something that doesn't need them.

Later on, a more robust system would restrict their access only within themselves and ensure they can't interact with other facets and parts of the private hierarchy, except for a few well defined messaging mechanisms.

[shiftyscales]

With the current UI, some of the most common interactions for the context menus are for undo/redo, and adjusting locomotion- all of which are going to become facets. Perhaps the context menu could be its own facet container?

That would address the nature of menus that open at the hand- just make them as extensions for the context menu instead, perhaps?

Alternatively, pressing the menu button could instead cycle through hand-bound menus. E.g. instead of open/close, it would be context menu > custom container(s) > close.

[Frooxius]

Context menu can't be a facet container, that's not really what it is and what it's designed to be. Context menu's are also world based, while the facet UI will be mostly in private UI.

[Abysmal2134]

Would you be able to show your private UI to other people (only visually so that they can't interact with colliders)?

I'm mainly asking because currently when you meet new people you can show your Inventory or File Browser and explain what each button does and how to navigate visually.

Being able to enter some kind of UI share mode so that you can explain how each facet works would be nice, especially further down the line when you want to show off/explain custom facets.

[TehTurk]

@Frooxius Ah! Sorry Tried to use terminology that'd be easier for you to click, UserSpace Root would be where Facets are normally, and MainRoot would be the World/Map itself.

Ah ok on the Anchor Point!, It's using the base heads and hands right? So I was trying to think of ways people would add uis or do certain things(someone looks at their foot and sees the time something silly like that)

For default interactions I think one to open them all would be good at the start, but expanded in terms of assigning certain gestures we have currently and such. In terms of the hand interactions that could also conflict with certain toys/tools too no?(For grip shortcut) But I'm also trying to considering in terms of how the Facets will be on the hands and how'd they conflict because default I don't think there'd be any there no? Sometimes hands stuff can be case by case depending on specifically "What" your doing. Can you give any scenarios? I'm having trouble visualizing it. Because in the future being able to assign some of the gesture controls to certain menus opening or interacting I think would be better.

[Reiko9]

I wish focus was on first making a streamlined and consistent GUI for when new users log in, before adding more flexibility.
As of the moment I keep having to apologize to new users for some of the weird extra steps needed for basic functionality. (Like directly moving an item into a folder)

[Frooxius]

@Abysmal2134 You could bring any facets into the world, so that would let you show them in some way. However I'll have to work out some details on security with this, so it's done in a controlled manner.

@TehTurk Ah. The facets wouldn't be actually under the root in the userspace, but typically deeper in the hierarchy. I'm not still quite sure if I understand about what you mean in the interactions.

And yes, the containers will be anchored to hands, head, playspace and so on, as described in my initial post.

I'm not sure I understand the point about the conflict? There's no reason for the facets to conflict unless you put a bunch of them in one spot.

@Reiko9 As describe in my initial post, that is the goal of this new UI. We are not adding customizability to the existing UI, but building a completely new system that's inherently customizable by design.

It is designed to start easy and streamlined for new users and then allow them to customize it as they go. It's not something that can be "added" later on, that would require rebuilding the whole system again from the grounds up.

[TehTurk]

Ah interactions I mean in Userspace to RootSpace movement/public/private or accessing stuff in UserSpace and bringing it to RootSpace. Interactions like those.

Yup! That was me agreeing/remarking but answering your question/statement, and looking for extra options that could be added.

Ah for conflict I was considering the access to them, the original current gestures, and splitting that between controller inputs/space, and having them be shared. You mentioned the conflict yourself so was trying to address it. I think having initial open be tied to one controller, and make individual actions be assignable would be better in the longer run as it would benefit the customizability of facets.

[DeliriousJax]

Addition request:
For workspace/interaction setting. "GrabClosestOne" toggle. When disabled, grabbing is the same as it works now. When Enabled, the grab sphere will take priority over the laser, and only grab the closest grabbable/slider/joint, etc to the hand itself.

Perhaps also having settings for snapping said object to the center of the grab sphere or to a close proximity to the hand.

Also possibly allow settings for grabbing nearest object within a certain distance and snaps said object to the hand even if its not within the grab sphere itself, and the laser is also not aiming at any other grabbables.

These settings would be able to be adjusted ahead of time in the workspace presets designer.

[sirkitree]

Any tips or tutorials on starting to use this now?

I'm specifically interested in what you show in some of your videos where you dynamically resize facets in a container with what looks like a drag and drop operation - is that something we can do in VR? I've tried every which way.

[shiftyscales]

The system isn't fully finished/developed yet, @sirkitree. As per the patch notes released when the new dash was implemented:
- Editing of the facets on the dash is disabled by default now, as saving mechanism isn't fully implemented yet and there are some bugs. If you want to play with it, press Ctrl+F2 on your keyboard to toggle editing on and off. THIS IS UNFINISHED FEATURE, please do not report bugs at this stage.

[sirkitree]

Thanks @shiftyscales for surfacing that (sometimes it's difficult to find this info in Discord).

I'm mainly looking to create a new container and use the drag and drop functionality of placing facets. Any steps on how to do that?

[Frooxius]

I'm getting the Workspace system working now. The Facets on the dash are now saving when modified, added or removed. However there's one question I was thinking to get some thoughts on: When should the modifications save?

I have the system setup so they save automatically after the last modification within a time period. This means that you don't really have to worry about the saving at all, you just tweak it and it will save. I think having this as default behavior is the best.

However there might be cases where you don't want to save your changes either and discard them. Would having an option to disable the auto-save and have an explicit one be important to you? How important would it be (as in something that can come a fair bit later or something that should be there early on)?

The system could also ask whether to save the changes to dash/workspace on exit too of course if the auto-save is off, in case you forget to hit save.

[TehTurk]

I feel like having different Workspaces saved/loaded in would be more important in the long run. So while an auto save would be handy, having a manual one would be more realistic, so you could do Workspace Swapping if need be.

[Frooxius]

Having separate workspaces is already on the roadmap, this is more about how is the workspace itself saved when you make changes to it.

[TehTurk]

I understood the initial question , you wouldn't want something automatically changing on you just because say you put something in there temporarily. You could do auto saving, but that always requires the user to clean up stuff. An option to disable it would be beneficial. Also leaving the behavior to a close window sounds neat, or turning off the edit mode causes you to do some sort of confirm/save/discard..

[Frooxius]

Well like I said, separate workspaces are kinda orthogonal to what I'm asking here though. I don't see having to do much cleanup, since usually you're not going to be even editing your workspace if you're not wanting to rearrange it.

The autosave is most likely best choice there, as it will help make it seamless - your UI will just be the way you left it last time without having to worry (similarly to the Settings for example, which don't require an explicit save either).

Having an option to disable it though will definitely be good, so it's more of a question of prioritizing it. Although perhaps I'm too early asking that right now, as the workflows are conceptual right now, so it's harder to tell on how you'll interact with it in practice.

[TehTurk]

I wouldn't worry to much in concepting the workflows, letting them be organic has always kinda worked out no? See originally I'm thinking in the context of games or experiences, say people use the facets as a way to interact between worlds or like ID/Memory cards as an intial concept. Autosaving can be handy here, but say if you wana quit and your progress had to be reset? It's there till your done, or remove for your workspace/dash. Correct me if I'm wrong but I wasn't sure worlds would have dash control.

[shiftyscales]

My only concern would be in having some mechanism in place to restore a last-known-good configuration, e.g. snapshots of some kind. That way incase someone accidentally softlocks themselves from accessing their workspace in some way, they wouldn't have to go to the means of resetting everything and starting from scratch. (This would also be mitigated once the mechanism is in place to save/load workspaces.)

Saving after exiting edit mode seems like the best plan, as you effectively 'commit' your changes/acknowledge you're satisfied with them. If you mess something up, you can either fix it, or exit Neos without leaving edit mode so the changes aren't committed.