How to write a Smart Glasses App in 30 minutes

How to write a Smart Glasses App in 30 minutes

This guide aims to take you from knowing nothing about the Smart Glasses Manager to having a fully-functioning smart glasses app in under 30 minutes.

Prerequisites

• An android device
  • (Optionally) a second android device for testing
• Experience with AndroidSDK, Android Studio, Java
• A pair of compatible smart glasses

Step 1: Understanding the Smart Glasses Manager

If you haven't already, we highly suggest you skim our Smart Glasses Manager Wiki page for more information on how it works. However, if you just want a basic synopsis, the Smart Glasses Manager (SGM) handles the following things:

• Compatibility with a variety of smart glasses devices
• The Bluetooth connection between a phone and smart glasses
• The logic as to what information / apps are to be displayed on the connected smart glasses at any given moment
• Receiving data from outside sources and relaying said data to relevant apps
  • Example: SGM receives audio and generates transcripts which can be accessed from other smart glasses apps

Step 2: Understanding SGMLib

SGMLib is an Android library that provides a simple interface for apps to interact with the SGM. It currently supports the following features:

• Registering your app - and individual commands - with the SGM

• Provides a service (SmartGlassesAndroidService) that can be started in the background by the SGM when required

• Subscribing to data streams from the SGM, such as audio transcripts

• Requesting the SGM to display information on the connected smart glasses. The following screens are currently available:

  • ReferenceCard: static screen with a title and body
  • ScrollingText: dynamic screen with a title and body that can be appended to

Step 3: Understanding how they work together

a) SGMLib Debug App walkthrough

In this section, we'll be walking through our Debug App in order to give you a good idea of how to use SGMLib.

Once you have it opened in Android Studio, note the two main Java files: MainActivity, and SgmLibDebugAppService.

In our example app, we use MainActivity to start and manage the state of SgmLibDebugAppService, which extends SmartGlassesAndroidService, which is a service the SGM can interface with.

Now, open SgmLibDebugAppService.java. We start by calling super in the constructor to initialize the service.

Next, we handle the rest of SGMLib's implementation in onCreate. We start by creating an SGMLib object, and passing it getApplicationContext():

sgmLib = new SGMLib(getApplicationContext());

Next, we define a command. Each command needs a name, UUID, list of trigger phrases, and a description:

UUID myUUID = new UUID.fromString("someUUID");
String[] triggerPhrases = {"hello world"};
SGMCommand helloWorldCommand = new SGMCommand("Debug One Shot", myUUID, triggerPhrases, "Hello world command desc");

We can also define a command that accepts arguments. This can be used, for example, in a search engine app, where the user needs to provide a search query after the initial command. Here's what that may look like:

String argumentPrompt = "Search query:";
ArrayList<String> exampleArgs = new ArrayList<String>(Arrays.asList("Dogs", "Spongebob", "Chicago"));
SGMCommand helloWorldWithArgsCommand = new SGMCommand("Debug With Args", myUUID, triggerPhrases, "Hello world command with args", true, argumentPrompt, exampleArgs);

Next, we register the command with a callback, which will be executed by the SGM when a user says one of your trigger phrases:

sgmLib.registerCommand(helloWorldCommand, this::helloWorldCallback);

Finally, we create our callback function helloWorldCallback(String args, long commandTime). Please note that SGMLib requires the args parameter to be defined regardless of whether the command requires an argument from the user:

public void helloWorldCallback(String args, long commandTime){
    // Callback triggered... Do something!
    // For example, display a reference card:

    sgmLib.sendReferenceCard("TitleText", "BodyText");
}