-
Notifications
You must be signed in to change notification settings - Fork 1
3D Wheel with VNyan
- Remote Wheel Sender (v0.3.0 or higher)
- VNyan (v1.1.7 or higher)
- VSeeFace; other 3D motion capture software may also work as well, as long as it can produce VMC output
- Wheel/Pedal VNyan Props (see the vnyan directory in the repository)
These instructions assume an existing setup of VNyan and VSeeFace, and the specifics of setting them up are not covered here. Necessary configuration changes to accommodate use with the Sender will be covered here, though.
The normal flow when using VNyan is that motion capture software (such as VSeeFace) will send tracking data to VNyan via the VMC protocol. VNyan then uses this tracking data to pose the avatar it displays.
The Remote Wheel Sender is intended to be placed in between VSeeFace and VNyan, so that it can take the tracking data produced by VSeeFace and alter it to pose the avatar's hands on the virtual wheel. Additional information can also be added (via blendshapes and/or VMC trackers), so that the 3D wheel can be rotated accordingly as well.
The Remote Wheel Sender provides configurations to accommodate two common configurations:
- The controller is connected to the same PC that runs the motion capture and avatar display (single PC).
- The controller is connected to one PC, while the motion capture and avatar display run on another PC (dual PC; referred to as the controller PC and the avatar PC below).
In the single PC case, the Sender both reads the controller input from the wheel and processes the motion tracking data using that input.
In the dual PC case, an instance of the Sender is run on each PC. On the controller PC, the Sender is used to read the controller values and send them to the avatar PC over the network. On the avatar PC, the Sender is configured to receive the wheel values via the network and use them to process the motion tracking data.
In both cases, when the Sender is first run, it will prompt for a default configuration to use. The options should be self-explanatory, but just in case, the options you want are:
- Single PC:
3D wheel overlay with VNyan (this PC has the controller connected and is also running VNyan) - Avatar PC:
3D wheel overlay with VNyan (this PC is running VNyan; the controller is connected to another PC) - Controller PC:
3D wheel overlay with VNyan (this PC has the controller connected; VNyan is on another PC)
A file named remote-wheel-sender.toml will be created in the same directory as the Sender executable. This file can be edited in any plain text editor (e.g. Notepad, Notepad++, VS Code, etc.). This file will contain notes about what should be changed, marked with CHANGEME, but these changes will be covered below as well.
The default configuration provides space to configure inputs for the wheel axis, a full set of pedal axes (gas, brake, clutch), and paddle shifter buttons. You will likely want the wheel and paddle shifters, but the pedals only need to be configured if you want to use the 3D pedals as well. If you don't wish to use the pedals, you may remove them from the configuration entirely or simply leave them; the default controller name is unlikely to match any real controllers, and no network traffic will be sent as long as no matching controller is found.
For each input, the name of the controller that it should be read from, as well as which axis/button to read, should be configured.
The controller name should match the one displayed in Windows's Game Controllers dialog; note that it must match exactly, and is case-sensitive. The Sender application will also log any connected controllers when it is started, which gives an opportunity to copy-paste the controller name.
For axis inputs, the axis number starts at 1 and is arbitrary to an extent. The numbering might correspond to the standard axes (X=1, Y=2, Z=3, RX=4, RY=5, RZ=6, Slider=7, Dial=8), but this is not guaranteed to be the case. This may take a little trial and error to find the right one, unfortunately; there's currently no tool built in in the Sender to help identify the correct one. This will likely be automated when it receives a UI!
For buttons, the button number starts at 1 and should match the button number shown in the Properties dialog for the game controller, in Windows's Game Controllers list.
When configuring the controller PC, the IP address of the stream PC should ideally be specified so that the wheel inputs can be sent directly to the stream PC. The address that needs to be changed will be indicated by a CHANGEME note in the default configuration file. The default IP address will be 255.255.255.255, which may work as-is for your setup, but the IP address should ideally be specified to prevent unnecessary network traffic.
Finding the IP address of your stream PC depends on the exact version of Windows that you are running, but broadly speaking, you want to go to network settings and view the properties for your network adapter. Note that you are looking for the local IP address, not the public one. The IP address that you want will almost always be of the form 192.168.x.x, but may also be of the form 172.16-31.x.x or 10.x.x.x. This IP address should be put in the Sender's configuration on the controller PC.
The default configuration for the single/avatar PC setup has the VMC output address set to 127.0.0.1:3333; if you've changed the port on which VNyan receives VMC packets, the Sender's configuration should be changed to match.
Additionally, the default configuration for the single/avatar PC setup provides a default position for where the virtual wheel will be, for avatar posing purposes. However, the defaults are for a smaller-than-average avatar and may not be appropriate for your avatar. This will be adjusted later on The rest of the settings should work as-is.
Sending data with the VMC protocol must be enabled. The IP address should be set to 127.0.0.1 and the port should be set to match the VMC input port specified in the Sender's configuration on the single/avatar PC. By default, this will be port 3332. By default, the Sender application reports how many VMC packets it has received every 60 seconds. Once VSeeFace is configured for the correct address/port, the Sender application should report that it is sending packets.
You may also want to adjust the Arm Angle slider here so your arms are at your sides. The avatar's pose is used as a starting point, and so this will impact the position of the elbows when holding the wheel in VNyan.
All other settings should be able to be set as normal. Any settings that affect the tracking (e.g. body stiffness) will be passed through the Sender as-is; only the arms will be reposed.
There are 2 major things you'll need to set up in VNyan: the props for the wheel and pedals (as desired) and the VMC tracker (which is used to position and rotate the wheel).
To set up the VMC tracker, open General Settings in VNyan, and under the Tracking Layers tab, find the "VMC Tracker Mapping" button at the end of section 2 (below the VMC receiver settings but above the ARKit settings). In the dropdown, select an unused tracker slot and note its number. If in doubt, Tracker 1 should be fine. Check that the Serial Number/Name field is empty, then enter Wheel (case sensitive). You may then close the dialogs.
To add the prop for the steering wheel, click the Props dialog, then click the Add Prop button. Select the vnprop file for the wheel. With the wheel's Linked Bone still set to World, position the wheel so that it is roughly where you want it in front of the avatar. You may also need to adjust the wheel's rotation; when using the provided wheel prop, the correct rotation should be X = 270, Y = 0, Z = 0. Once you have it where you like it, copy the position (but not the rotation) of the wheel prop in VNyan into the configuration for the Sender on the avatar PC, under vmc.device.wheel. If it's already running, restart the Sender on the avatar PC so that it loads the updated position.
Now, ensure that VSeeFace and the Remote Wheel Sender are running on the avatar PC. You should see your avatar's hands posed close to where you had placed the wheel in the previous step. Now, change the Linked Bone on the wheel prop to the tracker you configured above and set its position back to X=0/Y=0/Z=0. The wheel should be in the same position as it was before, but it should now turn in sync with the physical wheel, and the avatar's hands should move with it.
Finally, adjust the wheel prop's position, rotation, and scale inside of VNyan so that it fits correctly in the avatar's hands. You may also need to tweak the wheel's radius in the Sender's configuration on the avatar PC if your avatar's hands are too close or too far from each other. You will most likely want to leave the X and Y values of the wheel's position at 0, but you will probably want to tweak the Z value as the wheel will probably be between the avatar's wrists.
If you do a mix of racing streams and other streams, you may want to be able to quickly bypass the wheel tracking. The Sender application itself currently provides no convenient option for disabling the arm posing while it is running, and when it isn't running, VNyan won't receive tracking data.
The simplest way to switch back to a non-racing pose is to switch VSeeFace to send its tracking data directly to VNyan instead. You may also want to hide the wheel/pedal props in VNyan. This will cause the avatar to stop holding the wheel. Note that this will also cause the wheel (and any other props) to stop reacting to input.