The HoloToolkit.Sharing library allows applications to span multiple devices, and enables holographic collaboration.
Originally developed for OnSight, a collaboration between SOTA (a Microsoft studio) and NASA to enhance their existing Mars rover planning tool with HoloLens, HoloToolkit.Sharing enables users to use multiple devices for a task by allowing the apps running on each device communicate and stay in sync seamlessly in real time.
Users can also collaborate with other users (who are also using multiple devices) who may be in the same room or working remotely.
- Discover available sessions or create your own
- Permanent or ‘until empty’ session lifetime
- See user status: current session, mute state
- Easy to discover and hop between sessions
- Users in a session can be in the same or different physical rooms
- Users can share the location of Holographic 'anchors' they place in their room with other users in the same room
- Users joining late can download all anchors in the session
- Allows multiple users to see shared holograms
Synchronize data across all participants in session
- Everyone in session guaranteed to see the same thing
- Automatic conflict resolution for simultanious conflicting remote changes
- Real-time: See remote changes as they happen
- Shared data sets automatically merged when new users join a session
- Responsive: no delay in your own changes
- Ownership: your data leaves session when you do
Support for VOIP is built-in
- Server-based mixing lowers processing and bandwidth requirements for clients
Connect devices just by looking at them
- One device displays a QR code with connection info and security code
- Other device sees QR code, connects, and validates with security code
- Can also detect location in 3D space using built-in fiducial marker support
Profiling and debugging an experience that spans multiple devices is challenging. So HoloToolkit.Sharing provides an app that can connect to multiple devices at once and aggregates their timings and debug output in a single place
HoloToolkit.Sharing has the ability to synchronize data across any application connected to a given session. Conflict resolution is automatically handled by the framework, at whichever level the conflict occurs.
The following primitives are natively supported by the sync system. The C# class that corresponds to each primitive is written in parentheses.
- Boolean (SyncBool)
- Double (SyncDouble)
- Float (SyncFloat)
- Integer (SyncInteger)
- Long (SyncLong)
- Object, which is a container class that can have child primitives (SyncObject
- String (SyncString)
On top of the native primitives above, the following types are supported in the C# layer:
- Quaternion (SyncQuaternion)
- Transform (SyncTransform)
- Unordered array (SyncArray)
- Vector3 (SyncVector3)
Other types can be built for your own application as needed by inheriting from SyncObject in a similar way to what SyncVector3 and SyncTransform do.
By default, the SyncRoot object (which inherits from SyncObject) only contains an array of InstantiatedPrefabs, which may not be enough for your application.
For any type inheriting from SyncObject, you can easily add new children primitives by using the SyncData attribute, such as in the following example:
public class MySyncObject : SyncObject
{
[SyncData]
public SyncSpawnArray<MyOtherSyncObject> OtherSyncObject;
[SyncData]
public SyncFloat FloatValue;
}
Any SyncPrimitive tagged with the [SyncData] attribute will automatically be added to the data model and synced in the current HoloToolkit.Sharing session.
Ensure you have the Sharing Service Feature enabled in HoloToolkit -> Configure -> Apply HoloLens Project Settings.
Enabling the Sharing Service will also enable these UWP capabilities:
- InternetClientServer
- PrivateNetworkClientServer
Enabling the Sharing Service will also uppack a new directory in your projects root folder named External.
To run the Sharing Service HoloToolkit -> Launch Sharing Service.
This will create a new instance of the server on your machine.
For a production envionment, follow the instructions from the main HoloToolkit: Running the Server.
- Double check the Server Address on your sharing stage component in your scene matches the address shown in the sharing service console.
- Make sure all devices are connected to the same Wireless Local Area Network.
- Ensure all firewall settings are correct. Windows firewall gives you options to enable/disable by network type (private, public, home), make sure you're enabling the firewall for your connection's type.
SharingService [..\..\Source\Common\Private\SessionListHandshakeLogic.cpp (67)]:
***************************************************************
List Server Handshake Failed: Invalid schema version.
Expected: 17, got 15
Please sync to latest XTools
***************************************************************
- Ensure you're using the latest binaries of the sharing service found at
HoloToolkit-Unity\External\HoloToolkit\Sharing\Server.
Contains compiled architecture specific binaries for SharingClient.dll which are required by the Unity application for accessing sharing APIs. Binaries are compiled from the native HoloToolkit\Sharing.
Prefabs related to the sharing and networking features.
- Enables sharing and networking in your Unity application.
- Allows you to communicate between a Windows and non-Windows device.
SharingStage.cs allows you to be a Primary Client (typical case). Server Address is the IP address of the machine running the HoloToolkit -> Launch Sharing Service. Server Port displays the port being used for communicating.
AutoJoinSession.cs creates a shared session with Session Name 'Default' which is customizable. Joins a player to that session if once already exists.
Scripts related to the sharing and networking features.
Scripts for in editor use only.
Enables users to start the Sharing Service, Sharing Manager, and Profiler from the Unity Editor via the HoloToolkit Menu.
Draws the default Sharing Stage Inspector and adds the SyncRoot Hierarchy view so users can quickly verify Sync Object updates.
Contains scripts compiled from the native HoloToolkit\Sharing repository and using the SWIG tool to generate different language bindings.
Scripts for spawning objects using the sharing service.
A SpawnManager that creates a GameObject based on a prefab when a new SyncSpawnedObject is created in the data model. This class can spawn prefabs in reaction to the addition/removal of any object that inherits from SyncSpawnedObject, thus allowing applications to dynamically spawn prefabs as needed.
The PrefabSpawnManager registers to the SyncArray of InstantiatedPrefabs in the SyncRoot object.
The various classes can be linked to a prefab from the editor by specifying which class corresponds to which prefab. Note that the class field is currently a string that has to be typed in manually ("SyncSpawnedObject", for example): this could eventually be improved through a custom editor script.
A SpawnManager is in charge of spawning the appropriate objects based on changes to an array of data model objects. It also manages the lifespan of these spawned objects.
This is an abstract class from which you can derive a custom SpawnManager to react to specific synchronized objects being added or removed from the data model.
This array is meant to hold SyncSpawnedObject and objects of subclasses. Compared to SyncArray, this supports dynamic types for objects.
A SpawnedObject contains all the information needed for another device to spawn an object in the same location as where it was originally created on this device.
Scripts for syncing data over sharing service.
The SyncArray class provides the functionality of an array in the data model. The array holds entire objects, not primitives, since each object is indexed by unique name. Note that this array is unordered.
This class implements the boolean primitive for the syncing system. It does the heavy lifting to make adding new bools to a class easy.
Used to markup SyncObject classes and SyncPrimitives inside those classes, so that they properly get instantiated when using a hierarchical data model that inherits from SyncObject.
This class implements the double primitive for the syncing system. It does the heavy lifting to make adding new doubles to a class easy.
This class implements the float primitive for the syncing system. It does the heavy lifting to make adding new floats to a class easy.
This class implements the integer primitive for the syncing system. It does the heavy lifting to make adding new integers to a class easy.
This class implements the long primitive for the syncing system. It does the heavy lifting to make adding new longs to a class easy.
The SyncObject class is a container object that can hold multiple SyncPrimitives.
Base primitive used to define an element within the data model. The primitive is defined by a field and a value.
This class implements the Quaternion object primitive for the syncing system. It does the heavy lifting to make adding new Quaternion to a class easy.
This class implements the string primitive for the syncing system. It does the heavy lifting to make adding new strings to a class easy.
This class implements the Transform object primitive for the syncing system. It does the heavy lifting to make adding new transforms to a class easy. A transform defines the position, rotation and scale of an object.
This class implements the Vector3 object primitive for the syncing system. It does the heavy lifting to make adding new Vector3 to a class easy.
Scripts used to implement unity specific sync services.
Default implementation of a behaviour that allows other components of a game object access the shared data model as a raw SyncObject instance.
Interface that allows a components of a game object access the shared data model set by a SpawnManager.
Synchronizer to update and broadcast a transform object through our data model.
Scripts for sync service utilities.
A behaviour component that allows the application to automatically join the specified session as soon as the application has a valid server connection. This class will also maintain the session connection throughout the application lifetime.
This is mostly meant to be used to quickly test networking in an application. In most cases, some session management code should be written to allow users to join/leave sessions according to the desired application flow.
Utility class that writes the sharing service log messages to the Unity Engine's console.
This class enables users to pair with a remote client directly. One side should use the Receiver role, the other side should use the Connector role. RemoteAddress and RemotePort are used by the Connector role, LocalPort is used by the Receiver.
Scripts for Voice Chat service.
Receives and plays voice data transmitted through the session server. This data comes from other clients running the MicrophoneTransmitter behaviour.
Transmits data from your microphone to other clients connected to a SessionServer. Requires any receiving client to be running the MicrophoneReceiver script.
The ServerSessionsTracker is in charge of listing the various sessions that exist on the server, and exposes events related to all of these sessions. This is also the class that allows the application to join or leave a session. Instance is created by Sharing Stage when a connection is found.
The SessionUsersTracker keeps track of the current session and its users. It also exposes events that are triggered whenever users join or leave the current session. Instance is created by Sharing Stage when a connection is found.
A Singleton behaviour that is in charge of managing the core networking layer for the application. The SharingStage has the following responsibilities:
- Server configuration (address, port and client role)
- Establish and manage the server connection
- Create and initialize the synchronized data model (SyncRoot)
- Create the ServerSessionsTracker that tracks all sessions on the server
- Create the SessionUsersTracker that tracks the users in the current session
Root of the synchronized data model, under which every element of the model should be located. The SharingStage will automatically create and initialize the SyncRoot at application startup.
Collection of sharing sync settings, used by the HoloToolkit Sharing sync system to figure out which data model classes need to be instantiated when receiving data that inherits from SyncObject.
C# wrapper for the Sharing SyncListener, making changes available through the Action class.
Prefabs used in the various test scenes, which you can use as inspiration to build your own.
Simple Cube prefab with a Transform, Mesh Filter, Box Collider, Mesh Renderer, and Default Sync Model Accessor components.
A simple Sphere prefab with a Transform, Mesh Filter, Sphere Collider, and Mesh Renderer components. Purposefully missing Default Sync Model Accessor component for SharingSpawnTest.
Test Scripts.
Test class for demonstrating how to send custom messages between clients.
Manages creating anchors and sharing the anchors with other clients.
Broadcasts the head transform of the local user to other users in the session, and adds and updates the head transforms of remote users. Head transforms are sent and received in the local coordinate space of the GameObject this component is on.
Test class for demonstrating creating rooms and anchors.
Class that handles spawning and deleteing sync objects for the SpawningTest.scene. Uses the KeywordManager to spawn objects using voice and keyboard input.
| Voice Command | Key Command | Description |
|---|---|---|
| Spawn Basic | Key I |
Spawns a cube with a SyncSpawnedObject basic sync model. |
| Spawn Custom | Key O |
Spawns a sphere with a SyncSpawnTestSphere custom sync model. |
| Delete Object | Key M |
Deletes both sync model types. |
Class that demonstrates a custom class using sync model attributes.
Used to demonstrate how to get notifications when users leave and enter room.
Tests related to the sharing features. To use the each scene:
- Navigate to the Tests folder.
- Double click on the test scene you wish to explore.
- Either click "Play" in the unity editor or File -> Build Settings.
- Add Open Scenes, Platform -> Windows Store, SDK -> Universal 10, Build Type -> D3D, Check 'Unity C# Projects'.
- Enable all required capabilities.
- Click 'Build' and create an App folder. When compile is done, open the solution and deploy to device.
This test demonstrates how to use the Sharing prefabs for networking and sharing custom messages with clients. It also demonstrates how to share world anchors between clients to establish a shared space.
- Ensure to launch the sharing service using: HoloToolkit -> Launch Sharing service
- Enter the IP address displayed in the console window into the Server Address of the Sharing object.
- CustomMessages.cs shows how to communicate specific information across clients.
- ImportExportAnchorManager.cs shows how to create anchors and share them with other clients using the sharing service.
- RemoteHeadManager.cs draw cubes on remote heads of users joining the session.
This test demonstrates how to create new rooms and anchors inside your application. It also demonstrates how to upload and download new anchors.
- Ensure to launch the sharing service using: HoloToolkit -> Launch Sharing service
- Enter the IP address displayed in the console window into the Server Address of the Sharing object.
- RoomTest.cs shows how to create, join, and leave rooms; also shows how to create and download anchors.
This test demonstrates how to spawn and delete sync objects in your scene and across your networked clients.
- Ensure to launch the sharing service using: HoloToolkit -> Launch Sharing service
- Enter the IP address displayed in the console window into the Server Address of the Sharing object.
- PrefabSpawnManager.cs enables you to store prefab references to use when spawning.
- SyncObjectSpawner.cs demonstrates how to spawn and delete sync objects, as well as custom class types.