Skip to content

How to write a Smart Glasses App in 30 minutes

Jump to bottom
cayden edited this page Aug 29, 2023 · 7 revisions

How to write a Smart Glasses App in 30 minutes

This guide aims to take you from knowing nothing about the Smart Glasses Manager to having a fully-functioning smart glasses app in under 30 minutes.

Prerequisites

  • An Android 10+ smart phone
  • Experience with Android Studio + Java
  • A pair of one of these compatible smart glasses
    • Vuzix Ultralite OEM Reference Platform (requires Android 12+)
    • Vuzix Shield
    • Activelook Engo 1
    • Activelook Engo 2
    • Inmo Air
    • Vuzix Blade 2
    • TCL RayNeo X2

Example Apps

We provide example applications which use the SGMLib to implement smart glasses use cases. Use these apps as a reference or starting point for your own applications:

Make Your Own Smart Glasses App

Step 1: Understanding the Smart Glasses Manager

We highly suggest you skimming the Smart Glasses Manager Docs for more information on how it works. In summary, the Smart Glasses Manager (SGM) handles the following things:

  • Compatibility with a variety of smart glasses devices
  • The wireless connection between a phone and smart glasses
  • The logic as to what information / apps are to be displayed on the connected smart glasses at any given moment
  • Commands UI
  • Transcription
  • Receiving and transmitting sensor data to relevant apps

Step 2: Understanding SGMLib

SGMLib is an Android library that provides a simple interface for developers to write apps that display information on smart glasses through the SGM. It supports the following features:

  • Displaying video and audio information on the connected smart glasses. The following screens are currently available:
    • ReferenceCard: static screen with a title and body
    • ScrollingText: dynamic screen with a title and body of scrolling text
  • Registering your app and commands with the SGM
  • Provides an Android Service (SmartGlassesAndroidService) that runs continuously in the background and can be started by the SGM
  • Subscribing to data streams from the SGM, such as transcripts

Step 3: Understanding How They Work Together

SGMLib Debug App walkthrough

In this section, we'll be walking through our Debug App in order to give you a good idea of how to use SGMLib.

Once you have it opened in Android Studio, note the two main Java files: MainActivity, and SgmLibDebugAppService.

In our example app, we use MainActivity to start and manage the state of SgmLibDebugAppService, which extends SmartGlassesAndroidService, which is a service the SGM can interface with.

Now, open SgmLibDebugAppService.java. We start by calling super in the constructor to initialize the service.

Next, we handle the rest of SGMLib's implementation in onCreate. We start by creating an SGMLib object, and passing it getApplicationContext():

sgmLib = new SGMLib(getApplicationContext());

Next, we define a command. Each command needs a name, UUID, list of trigger phrases, and a description:

UUID myUUID = new UUID.fromString("someUUID");
String[] triggerPhrases = {"hello world"};
SGMCommand helloWorldCommand = new SGMCommand("Debug One Shot", myUUID, triggerPhrases, "Hello world command desc");

We can also define a command that accepts arguments. This can be used, for example, in a search engine app, where the user needs to provide a search query after the initial command. Here's what that may look like:

String argumentPrompt = "Search query:";
ArrayList<String> exampleArgs = new ArrayList<String>(Arrays.asList("Dogs", "Spongebob", "Chicago"));
SGMCommand helloWorldWithArgsCommand = new SGMCommand("Debug With Args", myUUID, triggerPhrases, "Hello world command with args", true, argumentPrompt, exampleArgs);

Next, we register the command with a callback, which will be executed by the SGM when a user says one of your trigger phrases:

sgmLib.registerCommand(helloWorldCommand, this::helloWorldCallback);

Finally, we create our callback function helloWorldCallback(String args, long commandTime). Please note that SGMLib requires the args parameter to be defined regardless of whether the command requires an argument from the user:

public void helloWorldCallback(String args, long commandTime){
    // Callback triggered... Do something!
    // For example, display a reference card:

    sgmLib.sendReferenceCard("TitleText", "BodyText");
}

Troubleshooting

If you're having trouble getting your app to work properly with the SGM after following this guide, make sure your app meets all of the requirements here: TPA Requirements

Help! I'm Still Stuck!

If you have attempted to follow this guide and hit problems you are unable to solve with the help of StackOverflow, ChatGPT, some tinkering, etc., then send a message in the Discord asking for help with DETAILED DESCRIPTION OF WHAT YOU DID, WHAT HAPPENED, WHAT YOU ARE TRYING TO DO, THE COPY-PASTE OR SCREENSHOT OF THE ERROR, ETC.

Clone this wiki locally