Skip to content

Android AR

Jump to bottom
David Podroužek edited this page Sep 20, 2018 · 3 revisions

Android ARCore codelab

Requirements

  • Android Studio version 3.1 or higher.

  • Android SDK Platform version 7.0 or higher.

  • Android device that supports ARCore. See a full list of supported devices.

    or

    Android Emulator version 27.2.9 or later with OpenGL ES 3.0 or later (it is already installed on this computer)

    Side Note: In the emulator, you must sign into the Play Store in case ARCore is not already installed.

A step-by-step Tutorial

Creating the project

  1. Open Android Studio and create a new project: File -> New -> New Project.
  2. Enter Application Name and click "Next".
  3. Select Phone and Tablet and choose the minimum SDK from the dropdown menu as API 27: Android 8.1 (Oreo), click "Next".
  4. Then, to add an empty activity to the project, choose Empty Activity and click "Next".
  5. Leave the name of the Activity as MainActivity and click "Finish".

Adding libraries and dependencies

  1. In the build.gradle script, located in the root, add Sceneform dependency, in the dependencies section:

    dependencies {
 ..
    classpath 'com.google.ar.sceneform:plugin:1.4.0'
}

    The program will suggest you to "Sync Now". So press the "Sync now" link to update the project.

    In this tutorial we will be using Sceneform, which is a framework that enables Android developers to build ARCore apps without the need to learn 3D graphics and OpenGL. If you would like to use the ARCore without Sceneform, you need to Enable ARcore. Check out how to do that by following the link.

  2. Next, in the app/build.gradle file, add compileOptions after buildTypes:

    buildTypes {
 ..
}
 compileOptions {
         targetCompatibility 1.8
         sourceCompatibility 1.8
}

    The app won't compile unless the compatibility version is set to 1.8.

  3. Add dependencies in the same file, in the dependencies section:

    dependencies {
 ..
    implementation "com.google.ar.sceneform:core:1.4.0"
    implementation "com.google.ar.sceneform.ux:sceneform-ux:1.4.0"
 }

  4. Next, create a sampledata folder. To do that, right click on the app folder in the directory tree and select: New -> Sample Data Directory.

  5. Paste the models folder into the sampledata folder. The models folder is located in the downloads directory, containing the files for converting the object into 3D (logo.mtl and logo.obj). Your directory tree should look like this:

    alt text

  6. Add this text at the end of the app/build.gradle file:

     apply plugin: 'com.google.ar.sceneform.plugin'
 sceneform.asset('sampledata/models/logo.obj',
        'default',
        'sampledata/models/logo.sfa',
        'src/main/res/raw/logo')

    The first line applies the Sceneform plugin, while the rest of the code converts the .obj file into .sfa and .sfb file (or 3D object) which will be further used in the app. .sfa and .sfb files are generated automatically.

    Again, press the "Sync now" link to update the project.

  7. Finally, the app requires to have Camera permission, in order to use the app as intended, and, at the same time, the app must show that it requires ARCore. To solve that, go to manifests/AndroidManifest.xml file and add the following code before the "application" tag:

    <uses-permission android:name="android.permission.CAMERA" />
<uses-feature android:name="android.hardware.camera.ar" android:required="true"/>

    Also, in the "application" tag, just before the "activity" tag, add the code below:

    <meta-data android:name="com.google.ar.core" android:value="required" />

Final steps

  1. Now, we just need to add an AR fragment to the activity .xml and edit the MainActivity.java file, where all the work is done. Open the res/layout/activity_main.xml file and add the following code, instead of TextView:

    <fragment android:name="com.google.ar.sceneform.ux.ArFragment"
         android:id="@+id/ar_fragment"
         android:layout_width="match_parent"
         android:layout_height="match_parent" />

  2. Then, instead of "android.support.constraint.ConstraintLayout" write FrameLayout. The file should look like this:

     <?xml version="1.0" encoding="utf-8"?>
 <FrameLayout xmlns:android="http://schemas.android.com/apk/res/android"
     xmlns:app="http://schemas.android.com/apk/res-auto"
     xmlns:tools="http://schemas.android.com/tools"
     android:layout_width="match_parent"
     android:layout_height="match_parent"
     tools:context=".MainActivity">
 
     <fragment android:name="com.google.ar.sceneform.ux.ArFragment"
         android:id="@+id/ar_fragment"
         android:layout_width="match_parent"
         android:layout_height="match_parent" />
 
 </FrameLayout>

  3. Then, it is important to import all the necessary libraries into the MainActivity.java file (located in the 'java' folder, for example, under 'com.example.codelab_android_ar' package):

    import android.app.Activity;
import android.app.ActivityManager;
import android.content.Context;
import android.os.Build;
import android.os.Build.VERSION_CODES;
import android.os.Bundle;
import android.support.v7.app.AppCompatActivity;
import android.util.Log;
import android.view.Gravity;
import android.view.MotionEvent;
import android.widget.Toast;
import com.google.ar.core.Anchor;
import com.google.ar.core.HitResult;
import com.google.ar.core.Plane;
import com.google.ar.sceneform.AnchorNode;
import com.google.ar.sceneform.rendering.ModelRenderable;
import com.google.ar.sceneform.ux.ArFragment;
import com.google.ar.sceneform.ux.TransformableNode;

  4. As we added a fragment in the activity, we can now add an ArFragment member variable, at the beginning of MainActivity class (class is located in the MainActivity.java file), as such:

    private ArFragment arFragment;

    and initialize it at the bottom of onCreate() function, like so:

    arFragment = (ArFragment) getSupportFragmentManager().findFragmentById(R.id.ar_fragment);

    Pay attention to the id attribute. It should be the same as referenced in the "activity_main.xml" file in the "fragment" tag: android:id="@+id/ar_fragment"

  5. We will be using one of the many ways in how Sceneform can create a renderable (3D model is there). We already have .sfa and .sfb files, which represent Sceneform binary assets. So, we will build them into ModelRenderable, using builder() function. Paste the following code after "arFragment" variable initialization:

    ModelRenderable.builder()
                .setSource(this, R.raw.logo)
                .build()
                .thenAccept(renderable -> logoRenderable = renderable)
                .exceptionally(
                        throwable -> {
                            Toast toast =
                                    Toast.makeText(this, "Unable to load logo renderable", Toast.LENGTH_LONG);
                            toast.show();
                            return null;
                        });

  6. Next, we would need to set a listener on the ArFragment. It will detect when and where on screen a tap is made (called HitResult), create an anchor and place our 3D object on the position of the tap.

    arFragment.setOnTapArPlaneListener(
                (HitResult hitResult, Plane plane, MotionEvent motionEvent) -> {
                    if (logoRenderable == null) {
                        return;
                    }

                    Anchor anchor = hitResult.createAnchor();
                    AnchorNode anchorNode = new AnchorNode(anchor);
                    anchorNode.setParent(arFragment.getArSceneView().getScene());

                    TransformableNode logo = new TransformableNode(arFragment.getTransformationSystem());
                    logo.setParent(anchorNode);
                    logo.setRenderable(logoRenderable);
                    logo.select();
                });
    }

  7. This step is optional (if needed). Lastly, we need to create a function that will check if the device you are using has the Android version which is supported, as well as, the supported OpenGL version (3.0 or later). The function will use 2 constant variables. Declare them at the top of the class, along with other variables:

    private static final String TAG = MainActivity.class.getSimpleName();
private static final double MIN_OPENGL_VERSION = 3.0;

    Next, create the function by pasting the following code after onCreate() function:

    public static boolean checkIsSupported (final Activity activity) {
        if (Build.VERSION.SDK_INT < VERSION_CODES.N) {
            Log.e(TAG, "Sceneform requires Android N or later");
            Toast.makeText(activity, "Sceneform requires Android N or later", Toast.LENGTH_LONG).show();
            activity.finish();
            return false;
        }
        String openGlVersionString =
                ((ActivityManager) activity.getSystemService(Context.ACTIVITY_SERVICE))
                        .getDeviceConfigurationInfo()
                        .getGlEsVersion();
        if (Double.parseDouble(openGlVersionString) < MIN_OPENGL_VERSION) {
            Log.e(TAG, "Sceneform requires OpenGL ES 3.0 or later");
            Toast.makeText(activity, "Sceneform requires OpenGL ES 3.0 or later", Toast.LENGTH_LONG)
                    .show();
            activity.finish();
            return false;
        }
        return true;
    }

    Now, we need to use this function by placing a conditional (since function returns boolean) at the top of onCreate() function, just after the super.onCreate(savedInstanceState);. The conditional should look like this:

    if (!checkIsSupported(this)) {
    return;
}

  8. You are now ready to run the app. Plug in your device. From the top menu select Run -> Run 'app' and choose the device you want to run it on. You can also use the emulator installed. alt text

Side Note:

In case you are using an emulator, and the application doesn't show the virtual scene, but just a black screen, locate the emulator tool where Android sdk is installed. On OSX, go to ~/Library/Android/sdk,
and run ./emulator -avd NEXUS_5X_API_27  -gpu host. Then, find the application in the menu and open it. This should work like a charm, otherwise, check the previous steps in case you've done some errors in the code.

What to do next?

  1. You can try to download and run these sample projects from Sceneform SDK to see what else can be done using this framework.

  2. You can read about Augmented Images and how an AR app can render a 3D version of 2D images located in real environment.

  3. You can explore Cloud Anchors, which enable developers to create AR experience for both Android and iOS users.

  4. If you would like to understand Sceneform more in-depth, read the documentation.

Clone this wiki locally