BlinkID SDK for Android

The BlinkID Android SDK lets you build a fantastic onboarding experience in your Android app.

With one quick scan, your users will be able to extract information from their identity cards, passports, driver licenses and virtually any other government-issued ID there is.

BlinkID is:

Fast. Real-time data extraction in less than 400ms. Way better than minutes-long form-filling.
Secure. Privacy first, always. Scanning works even if the user’s phone is in airplane mode, meaning personal information never touches a third-party server.
Intelligent. Machine learning models, optimized to read and parse identity documents from more than 180 countries worldwide, automatically, no need to preselect any of them.
Lightweight. Designed to increase your app’s usability, not weight.
What you make of it. Customize and rebrand the default UI or leave it as it is. It’s up to you.
More than just a powerful ID scanner. Powerful data extraction, coupled with powerful perks. Get a cropped document image back, spot printed documents or data match both sides of the ID for parity.

To see all of these features at work download our free demo app:

Feeling ready to crack on with the integration? First make sure we support your document type ➡️ full list. And then follow the guidelines below.

Quick Start
- Quick start with the sample app
- SDK integration
Device requirements
BlinkID SDK integration levels
Available activities and overlays
Handling processing events with RecognizerRunner and RecognizerRunnerView
Recognizer concept and RecognizerBundle
- The Recognizer concept
- RecognizerBundle
  - Passing Recognizer objects between activities
List of available recognizers
Embedding BlinkID inside another SDK
Processor architecture considerations
- Reducing the final size of your app
  - Consequences of removing processor architecture
- Combining BlinkID with other native libraries
Troubleshooting
FAQ and known issues
Additional info

Quick Start

Quick start with the sample app

Open Android Studio.
In Quick Start dialog choose Import project (Eclipse ADT, Gradle, etc.).
In File dialog select BlinkIDSample folder.
Wait for the project to load. If Android studio asks you to reload project on startup, select Yes.

Included sample apps

BlinkID-aMinimalSample demonstrates quick and simple integration of BlinkID library
BlinkID-AllRecognizersSample demonstrates integration of almost all available features. This sample application is best for performing a quick test of supported features
BlinkID-CustomCombinedSample demonstrates advanced custom UI integration and usage of the combined recognizers within a custom scan activity
BlinkID-CustomUISample demonstrates advanced integration within custom scan activity
BlinkID-DirectApiSample demonstrates how to perform scanning of Android Bitmaps
BlinkID-ImagesSample demonstrates how to obtain document images
BlinkID-OverlaySample demonstrates how to use RecognizerRunnerFragment and built in camera overlay controller within your activity

SDK integration

Adding BlinkID dependency

In your build.gradle, add BlinkID maven repository to repositories list

repositories {
    maven { url 'https://maven.microblink.com' }
}

Add BlinkID as a dependency and make sure transitive is set to true

dependencies {
    implementation('com.microblink:blinkid:5.19.0@aar') {
        transitive = true
    }
}

Importing Javadoc

Android studio 3.0 should automatically import javadoc from maven dependency. If that doesn't happen, you can do that manually by following these steps:

In Android Studio project sidebar, ensure project view is enabled
Expand External Libraries entry (usually this is the last entry in project view)
Locate blinkid-5.19.0 entry, right click on it and select Library Properties...
A Library Properties pop-up window will appear
Click the second + button in bottom left corner of the window (the one that contains + with little globe)
Window for defining documentation URL will appear
Enter following address: https://blinkid.github.io/blinkid-android/
Click OK

Performing your first scan

A valid license key is required to initialize scanning. You can request a free trial license key, after you register, at Microblink Developer Hub. License is bound to package name of your app, so please make sure you enter the correct package name when asked.

Download your licence file and put it in your application's assets folder. Make sure to set the license key before using any other classes from the SDK, otherwise you will get a runtime exception.

We recommend that you extend Android Application class and set the license in onCreate callback like this:
```
public class MyApplication extends Application {
    @Override
    public void onCreate() {
        MicroblinkSDK.setLicenseFile("path/to/license/file/within/assets/dir", this);
    }
}
```

In your main activity, create recognizer objects that will perform image recognition, configure them and put them into RecognizerBundle object. You can see more information about available recognizers and RecognizerBundle here.

For example, to scan supported document, configure your recognizer like this:

public class MyActivity extends Activity {
    private BlinkIdCombinedRecognizer mRecognizer;
    private RecognizerBundle mRecognizerBundle;
    
    @Override
    protected void onCreate(Bundle bundle) {
        super.onCreate(bundle);
        
        // setup views, as you would normally do in onCreate callback
        
        // create BlinkIdCombinedRecognizer
        mRecognizer = new BlinkIdCombinedRecognizer();
        
        // bundle recognizers into RecognizerBundle
        mRecognizerBundle = new RecognizerBundle(mRecognizer);
    }
}

Start recognition process by creating BlinkIdUISettings and calling ActivityRunner.startActivityForResult:

// method within MyActivity from previous step
public void startScanning() {
    // Settings for BlinkIdActivity
    BlinkIdUISettings settings = new BlinkIdUISettings(mRecognizerBundle);
    
    // tweak settings as you wish
    
    // Start activity
    ActivityRunner.startActivityForResult(this, MY_REQUEST_CODE, settings);
}

onActivityResult will be called in your activity after scanning is finished, here you can get the scanning results.

@Override
protected void onActivityResult(int requestCode, int resultCode, Intent data) {
    super.onActivityResult(requestCode, resultCode, data);
    
    if (requestCode == MY_REQUEST_CODE) {
        if (resultCode == Activity.RESULT_OK && data != null) {
            // load the data into all recognizers bundled within your RecognizerBundle
            mRecognizerBundle.loadFromIntent(data);
            
            // now every recognizer object that was bundled within RecognizerBundle
            // has been updated with results obtained during scanning session
            
            // you can get the result by invoking getResult on recognizer
            BlinkIdCombinedRecognizer.Result result = mRecognizer.getResult();
            if (result.getResultState() == Recognizer.Result.State.Valid) {
                // result is valid, you can use it however you wish
            }
        }
    }
}

For more information about available recognizers and RecognizerBundle, see RecognizerBundle and available recognizers.

Device requirements

Android Version

BlinkID requires Android API level 16 or newer. For best performance and compatibility, we recommend at least Android 5.0.

Camera

Camera video preview resolution also matters. In order to perform successful scans, camera preview resolution must be at least 720p. Note that camera preview resolution is not the same as video recording resolution.

Processor architecture

BlinkID is distributed with ARMv7, ARM64, x86 and x86_64 native library binaries.

BlinkID is a native library, written in C++ and available for multiple platforms. Because of this, BlinkID cannot work on devices with obscure hardware architectures. We have compiled BlinkID native code only for the most popular Android ABIs.

Even before setting the license key, you should check if the BlinkID is supported on the current device (see next section: Compatibility check). Attempting to call any method from the SDK that relies on native code, such as license check, on a device with unsupported CPU architecture will crash your app.

If you are combining BlinkID library with other libraries that contain native code into your application, make sure you match the architectures of all native libraries.

For example, if a third party library has got only ARMv7 and ARM64 versions, you must use exactly ARMv7 and ARM64 versions of BlinkID with that library, but not x86. Using these architectures will crash your app at the initialization step because JVM will try to load all its native dependencies in the same preferred architecture and will fail with UnsatisfiedLinkError.

For more information, see Processor architecture considerations section.

Compatibility check

Here's how you can check whether the BlinkID is supported on the device:

// check if BlinkID is supported on the device,
RecognizerCompatibilityStatus status = RecognizerCompatibility.getRecognizerCompatibilityStatus(this);
if (status == RecognizerCompatibilityStatus.RECOGNIZER_SUPPORTED) {
    Toast.makeText(this, "BlinkID is supported!", Toast.LENGTH_LONG).show();
} else if (status == RecognizerCompatibilityStatus.NO_CAMERA) {
    Toast.makeText(this, "BlinkID is supported only via Direct API!", Toast.LENGTH_LONG).show();
} else if (status == RecognizerCompatibilityStatus.PROCESSOR_ARCHITECTURE_NOT_SUPPORTED) {
    Toast.makeText(this, "BlinkID is not supported on current processor architecture!", Toast.LENGTH_LONG).show();
} else {
    Toast.makeText(this, "BlinkID is not supported! Reason: " + status.name(), Toast.LENGTH_LONG).show();
}

Some recognizers require camera with autofocus. If you try using them on a device that doesn't support autofocus, you will get an error. To prevent that, you can check whether a recognizer requires autofocus by calling its requiresAutofocus method.

If you already have an array of recognizers, you can easily filter out recognizers that require autofocus from array using the following code snippet:

Recognizer[] recArray = ...;
if(!RecognizerCompatibility.cameraHasAutofocus(CameraType.CAMERA_BACKFACE, this)) {
    recArray = RecognizerUtils.filterOutRecognizersThatRequireAutofocus(recArray);
}

BlinkID SDK integration levels

You can integrate BlinkID into your app in four different ways, depending on your use case and customisation needs:

Built-in activities (UISettings) - SDK handles everything and you just need to start our built-in activity and handle result, customisation options are limited
Built-in fragment (RecognizerRunnerFragment) - reuse scanning UX from our built-in activities in your own activity
Custom UX (RecognizerRunnerView) - SDK handles camera management while you have to implement completely custom scanning UX
Direct Api (RecognizerRunner) - SKD only handles recognition while you have to provide it with the images, either from camera or from a file

Built-in activities (`UISettings`)

UISettings is a class that contains all the necessary settings for SDK's built-in scan activities. It configures scanning activity behaviour, strings, icons and other UI elements. As shown in the first scan example, you should use ActivityRunner to start the scan activity configured by UISettings.

We provide multiple UISettings classes specialised for different scanning scenarios. Each UISettings object has properties which can be changed via appropriate setter methods. For example, you can customise camera settings with setCameraSettings metod.

All available UISettings classes are listed here.

Built-in fragment (`RecognizerRunnerFragment`)

If you want to reuse our built-in activity UX inside your own activity, use RecognizerRunnerFragment. Activity that will host RecognizerRunnerFragment must implement ScanningOverlayBinder interface. Attempting to add RecognizerRunnerFragment to activity that does not implement that interface will result in ClassCastException.

The ScanningOverlayBinder is responsible for returning non-null implementation of ScanningOverlay - class that will manage UI on top of RecognizerRunnerFragment. It is not recommended to create your own ScanningOverlay implementation, use one of our implementations listed here instead.

Here is the minimum example for activity that hosts the RecognizerRunnerFragment:

public class MyActivity extends AppCompatActivity implements RecognizerRunnerFragment.ScanningOverlayBinder {
    private BlinkIdCombinedRecognizer mRecognizer;
    private RecognizerBundle mRecognizerBundle;
    private BlinkIdOverlayController mScanOverlay;
    private RecognizerRunnerFragment mRecognizerRunnerFragment;

    @Override
    protected void onCreate(Bundle savedInstanceState) {
        super.onCreate();
        setContentView(R.layout.activity_my_activity);
        mScanOverlay = createOverlay();
        if (null == savedInstanceState) {
            // create fragment transaction to replace R.id.recognizer_runner_view_container with RecognizerRunnerFragment
            mRecognizerRunnerFragment = new RecognizerRunnerFragment();
            FragmentTransaction fragmentTransaction = getSupportFragmentManager().beginTransaction();
            fragmentTransaction.replace(R.id.recognizer_runner_view_container, mRecognizerRunnerFragment);
            fragmentTransaction.commit();
        } else {
            // obtain reference to fragment restored by Android within super.onCreate() call
            mRecognizerRunnerFragment = (RecognizerRunnerFragment) getSupportFragmentManager().findFragmentById(R.id.recognizer_runner_view_container);
        }
    }

    @Override
    @NonNull
    public ScanningOverlay getScanningOverlay() {
        return mScanOverlay;
    }

    private BlinkIdOverlayController createOverlay() {
        // create BlinkIdCombinedRecognizer
        mRecognizer = new BlinkIdCombinedRecognizer();

        // bundle recognizers into RecognizerBundle
        mRecognizerBundle = new RecognizerBundle(mRecognizer);

        BlinkIdUISettings settings = new BlinkIdUISettings(mRecognizerBundle);

        return settings.createOverlayController(this, mScanResultListener);
    }

    private final ScanResultListener mScanResultListener = new ScanResultListener() {
        @Override
        public void onScanningDone(@NonNull RecognitionSuccessType recognitionSuccessType) {
            // pause scanning to prevent new results while fragment is being removed
            mRecognizerRunnerFragment.getRecognizerRunnerView().pauseScanning();

            // now you can remove the RecognizerRunnerFragment with new fragment transaction
            // and use result within mRecognizer safely without the need for making a copy of it

            // if not paused, as soon as this method ends, RecognizerRunnerFragments continues
            // scanning. Note that this can happen even if you created fragment transaction for
            // removal of RecognizerRunnerFragment - in the time between end of this method
            // and beginning of execution of the transaction. So to ensure result within mRecognizer
            // does not get mutated, ensure calling pauseScanning() as shown above.
        }
        @Override
        public void onUnrecoverableError(@NonNull Throwable throwable) {
        }
    };
    
}

Please refer to sample apps provided with the SDK for more detailed example and make sure your host activity's orientation is set to nosensor or has configuration changing enabled (i.e. is not restarted when configuration change happens). For more information, check scan orientation section.

Custom UX with `RecognizerRunnerView`

This section discusses how to embed RecognizerRunnerView into your scan activity and perform scan.

First make sure that RecognizerRunnerView is a member field in your activity. This is required because you will need to pass all activity's lifecycle events to RecognizerRunnerView.
It is recommended to keep your scan activity in one orientation, such as portrait or landscape. Setting sensor as scan activity's orientation will trigger full restart of activity whenever device orientation changes. This will provide very poor user experience because both camera and BlinkID native library will have to be restarted every time. There are measures against this behaviour that are discussed later.
In your activity's onCreate method, create a new RecognizerRunnerView, set RecognizerBundle containing recognizers that will be used by the view, define CameraEventsListener that will handle mandatory camera events, define ScanResultListener that will receive call when recognition has been completed and then call its create method. After that, add your views that should be layouted on top of camera view.
Pass in your activity's lifecycle using setLifecycle method to enable automatic handling of lifeceycle events.

Here is the minimum example of integration of RecognizerRunnerView as the only view in your activity:

public class MyScanActivity extends AppCompatActivity {
    private static final int PERMISSION_CAMERA_REQUEST_CODE = 42;
    private RecognizerRunnerView mRecognizerRunnerView;
    private BlinkIdCombinedRecognizer mRecognizer;
    private RecognizerBundle mRecognizerBundle;

    @Override
    protected void onCreate(Bundle savedInstanceState) {
        super.onCreate(savedInstanceState);

        // create BlinkIdCombinedRecognizer
        mRecognizer = new BlinkIdCombinedRecognizer();

        // bundle recognizers into RecognizerBundle
        mRecognizerBundle = new RecognizerBundle(mRecognizer);
        // create RecognizerRunnerView
        mRecognizerRunnerView = new RecognizerRunnerView(this);
        
        // set lifecycle to automatically call recognizer runner view lifecycle methods
        mRecognizerRunnerView.setLifecycle(getLifecycle());

        // associate RecognizerBundle with RecognizerRunnerView
        mRecognizerRunnerView.setRecognizerBundle(mRecognizerBundle);

        // scan result listener will be notified when scanning is complete
        mRecognizerRunnerView.setScanResultListener(mScanResultListener);
        // camera events listener will be notified about camera lifecycle and errors
        mRecognizerRunnerView.setCameraEventsListener(mCameraEventsListener);

        setContentView(mRecognizerRunnerView);
    }

    @Override
    public void onConfigurationChanged(Configuration newConfig) {
        super.onConfigurationChanged(newConfig);
        // changeConfiguration is not handled by lifecycle events so call it manually
        mRecognizerRunnerView.changeConfiguration(newConfig);
    }

    private final CameraEventsListener mCameraEventsListener = new CameraEventsListener() {
        @Override
        public void onCameraPreviewStarted() {
            // this method is from CameraEventsListener and will be called when camera preview starts
        }

        @Override
        public void onCameraPreviewStopped() {
            // this method is from CameraEventsListener and will be called when camera preview stops
        }

        @Override
        public void onError(Throwable exc) {
            /**
             * This method is from CameraEventsListener and will be called when
             * opening of camera resulted in exception or recognition process
             * encountered an error. The error details will be given in exc
             * parameter.
             */
        }

        @Override
        @TargetApi(23)
        public void onCameraPermissionDenied() {
            /**
             * Called in Android 6.0 and newer if camera permission is not given
             * by user. You should request permission from user to access camera.
             */
            requestPermissions(new String[]{Manifest.permission.CAMERA}, PERMISSION_CAMERA_REQUEST_CODE);
            /**
             * Please note that user might have not given permission to use
             * camera. In that case, you have to explain to user that without
             * camera permissions scanning will not work.
             * For more information about requesting permissions at runtime, check
             * this article:
             * https://developer.android.com/training/permissions/requesting.html
             */
        }

        @Override
        public void onAutofocusFailed() {
            /**
             * This method is from CameraEventsListener will be called when camera focusing has failed.
             * Camera manager usually tries different focusing strategies and this method is called when all
             * those strategies fail to indicate that either object on which camera is being focused is too
             * close or ambient light conditions are poor.
             */
        }

        @Override
        public void onAutofocusStarted(Rect[] areas) {
            /**
             * This method is from CameraEventsListener and will be called when camera focusing has started.
             * You can utilize this method to draw focusing animation on UI.
             * Areas parameter is array of rectangles where focus is being measured.
             * It can be null on devices that do not support fine-grained camera control.
             */
        }

        @Override
        public void onAutofocusStopped(Rect[] areas) {
            /**
             * This method is from CameraEventsListener and will be called when camera focusing has stopped.
             * You can utilize this method to remove focusing animation on UI.
             * Areas parameter is array of rectangles where focus is being measured.
             * It can be null on devices that do not support fine-grained camera control.
             */
        }
    };
    
    private final ScanResultListener mScanResultListener = new ScanResultListener() {
        @Override
        public void onScanningDone(@NonNull RecognitionSuccessType recognitionSuccessType) {
            // this method is from ScanResultListener and will be called when scanning completes
            // you can obtain scanning result by calling getResult on each
            // recognizer that you bundled into RecognizerBundle.
            // for example:

            BlinkIdCombinedRecognizer.Result result = mRecognizer.getResult();
            if (result.getResultState() == Recognizer.Result.State.Valid) {
                // result is valid, you can use it however you wish
            }

            // Note that mRecognizer is stateful object and that as soon as
            // scanning either resumes or its state is reset
            // the result object within mRecognizer will be changed. If you
            // need to create a immutable copy of the result, you can do that
            // by calling clone() on it, for example:

            BlinkIdCombinedRecognizer.Result immutableCopy = result.clone();

            // After this method ends, scanning will be resumed and recognition
            // state will be retained. If you want to prevent that, then
            // you should call:
            mRecognizerRunnerView.resetRecognitionState();
            // Note that reseting recognition state will clear internal result
            // objects of all recognizers that are bundled in RecognizerBundle
            // associated with RecognizerRunnerView.

            // If you want to pause scanning to prevent receiving recognition
            // results or mutating result, you should call:
            mRecognizerRunnerView.pauseScanning();
            // if scanning is paused at the end of this method, it is guaranteed
            // that result within mRecognizer will not be mutated, therefore you
            // can avoid creating a copy as described above

            // After scanning is paused, you will have to resume it with:
            mRecognizerRunnerView.resumeScanning(true);
            // boolean in resumeScanning method indicates whether recognition
            // state should be automatically reset when resuming scanning - this
            // includes clearing result of mRecognizer
        }
    };  
    
}

Scan activity's orientation

If activity's screenOrientation property in AndroidManifest.xml is set to sensor, fullSensor or similar, activity will be restarted every time device changes orientation from portrait to landscape and vice versa. While restarting activity, its onPause, onStop and onDestroy methods will be called and then new activity will be created anew. This is a potential problem for scan activity because in its lifecycle it controls both camera and native library - restarting the activity will trigger both restart of the camera and native library. This is a problem because changing orientation from landscape to portrait and vice versa will be very slow, thus degrading a user experience. We do not recommend such setting.

For that matter, we recommend setting your scan activity to either portrait or landscape mode and handle device orientation changes manually. To help you with this, RecognizerRunnerView supports adding child views to it that will be rotated regardless of activity's screenOrientation. You add a view you wish to be rotated (such as view that contains buttons, status messages, etc.) to RecognizerRunnerView with addChildView method. The second parameter of the method is a boolean that defines whether the view you are adding will be rotated with device. To define allowed orientations, implement OrientationAllowedListener interface and add it to RecognizerRunnerView with method setOrientationAllowedListener. This is the recommended way of rotating camera overlay.

However, if you really want to set screenOrientation property to sensor or similar and want Android to handle orientation changes of your scan activity, then we recommend to set configChanges property of your activity to orientation|screenSize. This will tell Android not to restart your activity when device orientation changes. Instead, activity's onConfigurationChanged method will be called so that activity can be notified of the configuration change. In your implementation of this method, you should call changeConfiguration method of RecognizerView so it can adapt its camera surface and child views to new configuration.

Direct API

This section will describe how to use direct API to recognize android Bitmaps without the need for camera. You can use direct API anywhere from your application, not just from activities.

Image recognition performance highly depends on the quality of the input images. When our camera management is used (scanning from a camera), we do our best to get camera frames with the best possible quality for the used device. On the other hand, when Direct API is used, you need to provide high-quality images without blur and glare for successful recognition.

Using Direct API for recognition of Android Bitmaps and custom camera frames

First, you need to obtain reference to RecognizerRunner singleton using getSingletonInstance.
Second, you need to initialize the recognizer runner.
After initialization, you can use singleton to process:

Still Android Bitmaps obtained, for example, from the gallery. Use recognizeBitmap or recognizeBitmapWithRecognizers.
Video Images that are built from custom camera video frames, for example, when you use your own or third party camera management. Recognition will be optimized for speed and will rely on time-redundancy between consecutive video frames in order to yield best possible recognition result. Use recognizeVideoImage or recognizeVideoImageWithRecognizers.
Still Images when you need thorough scanning of single or few images which are not part of the video stream and you want to get best possible results from the single Image. Image type comes from our SDK or it can be created by using ImageBuilder. Use recognizeStillImage or recognizeStillImageWithRecognizers.

When you want to delete all cached data from multiple recognitions, for example when you want to scan other document and/or restart scanning, you need to reset the recognition state.
Do not forget to terminate the recognizer runner singleton after usage (it is a shared resource).

Here is the minimum example of usage of direct API for recognizing android Bitmap:

public class DirectAPIActivity extends Activity {
    private RecognizerRunner mRecognizerRunner;
    private BlinkIdCombinedRecognizer mRecognizer;
    private RecognizerBundle mRecognizerBundle;

    @Override
    protected void onCreate(Bundle savedInstanceState) {
        super.onCreate();
        // initialize your activity here
        // create BlinkIdCombinedRecognizer
        mRecognizer = new BlinkIdCombinedRecognizer();

        // bundle recognizers into RecognizerBundle
        mRecognizerBundle = new RecognizerBundle(mRecognizer);

        try {
            mRecognizerRunner = RecognizerRunner.getSingletonInstance();
        } catch (FeatureNotSupportedException e) {
            Toast.makeText(this, "Feature not supported! Reason: " + e.getReason().getDescription(), Toast.LENGTH_LONG).show();
            finish();
            return;
        }

        mRecognizerRunner.initialize(this, mRecognizerBundle, new DirectApiErrorListener() {
            @Override
            public void onRecognizerError(Throwable t) {
                Toast.makeText(DirectAPIActivity.this, "There was an error in initialization of Recognizer: " + t.getMessage(), Toast.LENGTH_SHORT).show();
                finish();
            }
        });
    }

    @Override
    protected void onResume() {
        super.onResume();
        // start recognition
        Bitmap bitmap = BitmapFactory.decodeFile("/path/to/some/file.jpg");
        mRecognizerRunner.recognizeBitmap(bitmap, Orientation.ORIENTATION_LANDSCAPE_RIGHT, mScanResultListener);
    }

    @Override
    protected void onDestroy() {
        super.onDestroy();
        mRecognizerRunner.terminate();
    }

    private final ScanResultListener mScanResultListener = new ScanResultListener() {
        @Override
        public void onScanningDone(@NonNull RecognitionSuccessType recognitionSuccessType) {
            // this method is from ScanResultListener and will be called
            // when scanning completes
            // you can obtain scanning result by calling getResult on each
            // recognizer that you bundled into RecognizerBundle.
            // for example:

            BlinkIdCombinedRecognizer.Result result = mRecognizer.getResult();
            if (result.getResultState() == Recognizer.Result.State.Valid) {
                // result is valid, you can use it however you wish
            }
        }
    };

}

ScanResultListener.onScanningDone method is called for each input image that you send to the recognition. You can call RecognizerRunner.recognize* method multiple times with different images of the same document for better reading accuracy until you get a successful result in the listener's onScanningDone method. This is useful when you are using your own or third-party camera management.

Using Direct API for `String` recognition (parsing)

Some recognizers support recognition from String. They can be used through Direct API to parse given String and return data just like when they are used on an input image. When recognition is performed on String, there is no need for the OCR. Input String is used in the same way as the OCR output is used when image is being recognized.

Recognition from String can be performed in the same way as recognition from image, described in the previous section.

The only difference is that one of the RecognizerRunner singleton methods for recognition from string should be called:

Understanding DirectAPI's state machine

Direct API's RecognizerRunner singleton is a state machine that can be in one of 3 states: OFFLINE, READY and WORKING.

When you obtain the reference to RecognizerRunner singleton, it will be in OFFLINE state.
You can initialize RecognizerRunner by calling initialize method. If you call initialize method while RecognizerRunner is not in OFFLINE state, you will get IllegalStateException.
After successful initialization, RecognizerRunner will move to READY state. Now you can call any of the recognize* methods.
When starting recognition with any of the recognize* methods, RecognizerRunner will move to WORKING state. If you attempt to call these methods while RecognizerRunner is not in READY state, you will get IllegalStateException
Recognition is performed on background thread so it is safe to call all RecognizerRunner's methods from UI thread
When recognition is finished, RecognizerRunner first moves back to READY state and then calls the onScanningDone method of the provided ScanResultListener.
Please note that ScanResultListener's onScanningDone method will be called on background processing thread, so make sure you do not perform UI operations in this callback. Also note that until the onScanningDone method completes, RecognizerRunner will not perform recognition of another image or string, even if any of the recognize* methods have been called just after transitioning to READY state. This is to ensure that results of the recognizers bundled within RecognizerBundle associated with RecognizerRunner are not modified while possibly being used within onScanningDone method.
By calling terminate method, RecognizerRunner singleton will release all its internal resources. Note that even after calling terminate you might receive onScanningDone event if there was work in progress when terminate was called.
terminate method can be called from any RecognizerRunner singleton's state
You can observe RecognizerRunner singleton's state with method getCurrentState

Using Direct API while RecognizerRunnerView is active

Both RecognizerRunnerView and RecognizerRunner use the same internal singleton that manages native code. This singleton handles initialization and termination of native library and propagating recognizers to native library. It is possible to use RecognizerRunnerView and RecognizerRunner together, as internal singleton will make sure correct synchronization and correct recognition settings are used. If you run into problems while using RecognizerRunner in combination with RecognizerRunnerView, let us know!

Using Direct API with combined recognizers

When you are using combined recognizer and images of both document sides are required, you need to call RecognizerRunner.recognize* multiple times. Call it first with the images of the first side of the document, until it is read, and then with the images of the second side. The combined recognizer automatically switches to second side scanning, after it has successfully read the first side. To be notified when the first side scanning is completed, you have to set the FirstSideRecognitionCallback through MetadataCallbacks. If you don't need that information, e.g. when you have only one image for each document side, don't set the FirstSideRecognitionCallback and check the RecognitionSuccessType in ScanResultListener.onScanningDone, after the second side image has been processed.

Available activities and overlays

`BlinkIdUISettings` and `BlinkIdOverlayController`

BlinkIdOverlayController implements new UI for scanning identity documents, which is optimally designed to be used with new BlinkIdCombinedRecognizer and BlinkIdRecognizer. It implements several new features:

clear indication for searching phase, when BlinkID is searching for an ID document
clear progress indication, when BlinkID is busy with OCR and data extraction
clear message when the document is not supported
visual indications when the user needs to place the document closer to the camera
when BlinkIdCombinedRecognizer is used, visual indication that the data from the front side of the document doesn't match the data on the back side of the document.

The new UI allows the user to scan the document at an any angle, in any orientation. We recommend forcing landscape orientation if you scan barcodes on the back side, because in that orientation success rate will be higher.

To launch a built-in activity that uses BlinkIdOverlayController use BlinkIdUISettings.

Scan overlay theming

To customise overlay, provide your custom style resource via BlinkIdUISettings.setOverlayViewStyle() method or via ReticleOverlayView constructor. You can customise elements labeled on screenshots above by providing the following attributes in your style:

exit

mb_exitScanDrawable - icon drawable

torch

mb_torchOnDrawable - icon drawable that is shown when the torch is enabled
mb_torchOffDrawable - icon drawable that is show when the torch is disabled

instructions

mb_instructionsTextAppearance - style that will be used as android:textAppearance
mb_instructionsBackgroundDrawable - drawable used for background

flashlight warning

mb_flashlightWarningTextAppearance - style that will be used as android:textAppearance
mb_flashlightWarningBackgroundDrawable - drawable used for background
note that you can disable this element by using BlinkIdUISettings.setShowFlashlightWarning(false)

card icon

mb_cardFrontDrawable - icon drawable shown during card flip animation, representing front side of the card
mb_cardBackDrawable - icon drawable shown during card flip animation, representing back side of the card

reticle

mb_reticleDefaultDrawable - drawable shown when reticle is in neutral state
mb_reticleSuccessDrawable - drawable shown when reticle is in success state (scanning was successful)
mb_reticleErrorDrawable - drawable shown when reticle is in error state

pulse

mb_pulseColor - color of the pulse animation that is active before a card is detected

progress

mb_progressDrawable - drawable used as indeterminate drawable for progress bar, shown inside reticle while recognition is in progress

`DocumentUISettings`

DocumentUISettings launches activity that uses BlinkIdOverlayController with alternative UI. It is best suited for scanning single document side of various card documents and it shouldn't be used with combined recognizers as it provides no user instructions on when to switch to the back side.

`LegacyDocumentVerificationUISettings`

LegacyDocumentVerificationUISettings launches activity that uses BlinkIdOverlayController with alternative UI. It is best suited for combined recognizers because it manages scanning of multiple document sides in the single camera opening and guides the user through the scanning process. It can also be used for single side scanning of ID cards, passports, driver's licenses, etc.

Translation and localization

Strings used within built-in activities and overlays can be localized to any language. If you are using RecognizerRunnerView (see this chapter for more information) in your custom scan activity or fragment, you should handle localization as in any other Android app. RecognizerRunnerView does not use strings nor drawables, it only uses assets from assets/microblink folder. Those assets must not be touched as they are required for recognition to work correctly.

However, if you use our built-in activities or overlays, they will use resources packed within LibBlinkID.aar to display strings and images on top of the camera view. We have already prepared strings for several languages which you can use out of the box. You can also modify those strings, or you can add your own language.

To use a language, you have to enable it from the code:

To use a certain language, on application startup, before opening any UI component from the SDK, you should call method LanguageUtils.setLanguageAndCountry(language, country, context). For example, you can set language to Croatian like this:
```
// define BlinkID language
LanguageUtils.setLanguageAndCountry("hr", "", this);
```

Adding new language

BlinkID can easily be translated to other languages. The res folder in LibBlinkID.aar archive has folder values which contains strings.xml - this file contains english strings. In order to make e.g. croatian translation, create a folder values-hr in your project and put the copy of strings.xml inside it (you might need to extract LibBlinkID.aar archive to access those files). Then, open that file and translate the strings from English into Croatian.

Changing strings in the existing language

To modify an existing string, the best approach would be to:

Choose a language you want to modify. For example Croatian ('hr').
Find strings.xml in folder res/values-hr of the LibBlinkID.aar archive
Choose a string key which you want to change. For example: <string name="MBBack">Back</string>
In your project create a file strings.xml in the folder res/values-hr, if it doesn't already exist
Create an entry in the file with the value for the string which you want. For example: <string name="MBBack">Natrag</string>
Repeat for all the string you wish to change

Handling processing events with `RecognizerRunner` and `RecognizerRunnerView`

Processing events, also known as Metadata callbacks are purely intended for giving processing feedback on UI or to capture some debug information during development of your app using BlinkID SDK. For that reason, built-in activities and fragments handle those events internally. If you need to handle those events yourself, you need to use either RecognizerRunnerView or RecognizerRunner.

Callbacks for all events are bundled into the MetadataCallbacks object. Both RecognizerRunner and RecognizerRunnerView have methods which allow you to set all your callbacks.

We suggest that you check for more information about available callbacks and events to which you can handle in the javadoc for MetadataCallbacks class.

Please note that both those methods need to pass information about available callbacks to the native code and for efficiency reasons this is done at the time setMetadataCallbacks method is called and not every time when change occurs within the MetadataCallbacks object. This means that if you, for example, set QuadDetectionCallback to MetadataCallbacks after you already called setMetadataCallbacks method, the QuadDetectionCallback will not be registered with the native code and you will not receive its events.

Similarly, if you, for example, remove the QuadDetectionCallback from MetadataCallbacks object after you already called setMetadataCallbacks method, your app will crash with NullPointerException when our processing code attempts to invoke the method on removed callback (which is now set to null). We deliberately do not perform null check here because of two reasons:

it is inefficient
having null callback, while still being registered to native code is illegal state of your program and it should therefore crash

Remember, each time you make some changes to MetadataCallbacks object, you need to apply those changes to to your RecognizerRunner or RecognizerRunnerView by calling its setMetadataCallbacks method.

`Recognizer` concept and `RecognizerBundle`

This section will first describe what is a Recognizer and how it should be used to perform recognition of the images, videos and camera stream. Next, we will describe how RecognizerBundle can be used to tweak the recognition procedure and to transfer Recognizer objects between activities.

RecognizerBundle is an object which wraps the Recognizers and defines settings about how recognition should be performed. Besides that, RecognizerBundle makes it possible to transfer Recognizer objects between different activities, which is required when using built-in activities to perform scanning, as described in first scan section, but is also handy when you need to pass Recognizer objects between your activities.

List of all available Recognizer objects, with a brief description of each Recognizer, its purpose and recommendations how it should be used to get best performance and user experience, can be found here .

The `Recognizer` concept

The Recognizer is the basic unit of processing within the BlinkID SDK. Its main purpose is to process the image and extract meaningful information from it. As you will see later, the BlinkID SDK has lots of different Recognizer objects that have various purposes.

Each Recognizer has a Result object, which contains the data that was extracted from the image. The Result object is a member of corresponding Recognizer object and its lifetime is bound to the lifetime of its parent Recognizer object. If you need your Result object to outlive its parent Recognizer object, you must make a copy of it by calling its method clone().

Every Recognizer is a stateful object, that can be in two states: idle state and working state. While in idle state, you can tweak Recognizer object's properties via its getters and setters. After you bundle it into a RecognizerBundle and use either RecognizerRunner or RecognizerRunnerView to run the processing with all Recognizer objects bundled within RecognizerBundle, it will change to working state where the Recognizer object is being used for processing. While being in working state, you cannot tweak Recognizer object's properties. If you need to, you have to create a copy of the Recognizer object by calling its clone(), then tweak that copy, bundle it into a new RecognizerBundle and use reconfigureRecognizers to ensure new bundle gets used on processing thread.

While Recognizer object works, it changes its internal state and its result. The Recognizer object's Result always starts in Empty state. When corresponding Recognizer object performs the recognition of given image, its Result can either stay in Empty state (in case Recognizer failed to perform recognition), move to Uncertain state (in case Recognizer performed the recognition, but not all mandatory information was extracted), move to StageValid state (in case Recognizer successfully scanned one part/side of the document and there are more fields to extract) or move to Valid state (in case Recognizer performed recognition and all mandatory information was successfully extracted from the image).

As soon as one Recognizer object's Result within RecognizerBundle given to RecognizerRunner or RecognizerRunnerView changes to Valid state, the onScanningDone callback will be invoked on same thread that performs the background processing and you will have the opportunity to inspect each of your Recognizer objects' Results to see which one has moved to Valid state.

As already stated in section about RecognizerRunnerView, as soon as onScanningDone method ends, the RecognizerRunnerView will continue processing new camera frames with same Recognizer objects, unless paused. Continuation of processing or resetting recognition will modify or reset all Recognizer objects's Results. When using built-in activities, as soon as onScanningDone is invoked, built-in activity pauses the RecognizerRunnerView and starts finishing the activity, while saving the RecognizerBundle with active Recognizer objects into Intent so they can be transferred back to the calling activities.

`RecognizerBundle`

The RecognizerBundle is wrapper around Recognizers objects that can be used to transfer Recognizer objects between activities and to give Recognizer objects to RecognizerRunner or RecognizerRunnerView for processing.

The RecognizerBundle is always constructed with array of Recognizer objects that need to be prepared for recognition (i.e. their properties must be tweaked already). The varargs constructor makes it easier to pass Recognizer objects to it, without the need of creating a temporary array.

The RecognizerBundle manages a chain of Recognizer objects within the recognition process. When a new image arrives, it is processed by the first Recognizer in chain, then by the second and so on, iterating until a Recognizer object's Result changes its state to Valid or all of the Recognizer objects in chain were invoked (none getting a Valid result state). If you want to invoke all Recognizers in the chain, regardless of whether some Recognizer object's Result in chain has changed its state to Valid or not, you can allow returning of multiple results on a single image.

You cannot change the order of the Recognizer objects within the chain - no matter the order in which you give Recognizer objects to RecognizerBundle, they are internally ordered in a way that provides best possible performance and accuracy. Also, in order for BlinkID SDK to be able to order Recognizer objects in recognition chain in the best way possible, it is not allowed to have multiple instances of Recognizer objects of the same type within the chain. Attempting to do so will crash your application.

Passing `Recognizer` objects between activities

Besides managing the chain of Recognizer objects, RecognizerBundle also manages transferring bundled Recognizer objects between different activities within your app. Although each Recognizer object, and each its Result object implements Parcelable interface, it is not so straightforward to put those objects into Intent and pass them around between your activities and services for two main reasons:

Result object is tied to its Recognizer object, which manages lifetime of the native Result object.
Result object often contains large data blocks, such as images, which cannot be transferred via Intent because of Android's Intent transaction data limit.

Although the first problem can be easily worked around by making a copy of the Result and transfer it independently, the second problem is much tougher to cope with. This is where, RecognizerBundle's methods saveToIntent and loadFromIntent come to help, as they ensure the safe passing of Recognizer objects bundled within RecognizerBundle between activities according to policy defined with method setIntentDataTransferMode:

if set to STANDARD, the Recognizer objects will be passed via Intent using normal Intent transaction mechanism, which is limited by Android's Intent transaction data limit. This is same as manually putting Recognizer objects into Intent and is OK as long as you do not use Recognizer objects that produce images or other large objects in their Results.
if set to OPTIMISED, the Recognizer objects will be passed via internal singleton object and no serialization will take place. This means that there is no limit to the size of data that is being passed. This is also the fastest transfer method, but it has a serious drawback - if Android kills your app to save memory for other apps and then later restarts it and redelivers Intent that should contain Recognizer objects, the internal singleton that should contain saved Recognizer objects will be empty and data that was being sent will be lost. You can easily provoke that condition by choosing No background processes under Limit background processes in your device's Developer options, and then switch from your app to another app and then back to your app.
if set to PERSISTED_OPTIMISED, the Recognizer objects will be passed via internal singleton object (just like in OPTIMISED mode) and will additionaly be serialized into a file in your application's private folder. In case Android restarts your app and internal singleton is empty after re-delivery of the Intent, the data will be loaded from file and nothing will be lost. The files will be automatically cleaned up when data reading takes place. Just like OPTIMISED, this mode does not have limit to the size of data that is being passed and does not have a drawback that OPTIMISED mode has, but some users might be concerned about files to which data is being written.
- These files will contain end-user's private data, such as image of the object that was scanned and the extracted data. Also these files may remain saved in your application's private folder until the next successful reading of data from the file.
- If your app gets restarted multiple times, only after first restart will reading succeed and will delete the file after reading. If multiple restarts take place, you must implement onSaveInstanceState and save bundle back to file by calling its saveState method. Also, after saving state, you should ensure that you clear saved state in your onResume, as onCreate may not be called if activity is not restarted, while onSaveInstanceState may be called as soon as your activity goes to background (before onStop), even though activity may not be killed at later time.
- If saving data to file in private storage is a concern to you, you should use either OPTIMISED mode to transfer large data and image between activities or create your own mechanism for data transfer. Note that your application's private folder is only accessible by your application and your application alone, unless the end-user's device is rooted.

List of available recognizers

This section will give a list of all Recognizer objects that are available within BlinkID SDK, their purpose and recommendations how they should be used to get best performance and user experience.

Frame Grabber Recognizer

The FrameGrabberRecognizer is the simplest recognizer in BlinkID SDK, as it does not perform any processing on the given image, instead it just returns that image back to its FrameCallback. Its Result never changes state from Empty.

This recognizer is best for easy capturing of camera frames with RecognizerRunnerView. Note that Image sent to onFrameAvailable are temporary and their internal buffers all valid only until the onFrameAvailable method is executing - as soon as method ends, all internal buffers of Image object are disposed. If you need to store Image object for later use, you must create a copy of it by calling clone.

Also note that FrameCallback interface extends Parcelable interface, which means that when implementing FrameCallback interface, you must also implement Parcelable interface.

This is especially important if you plan to transfer FrameGrabberRecognizer between activities - in that case, keep in mind that the instance of your object may not be the same as the instance on which onFrameAvailable method gets called - the instance that receives onFrameAvailable calls is the one that is created within activity that is performing the scan.

Success Frame Grabber Recognizer

The SuccessFrameGrabberRecognizer is a special Recognizer that wraps some other Recognizer and impersonates it while processing the image. However, when the Recognizer being impersonated changes its Result into Valid state, the SuccessFrameGrabberRecognizer captures the image and saves it into its own Result object.

Since SuccessFrameGrabberRecognizer impersonates its slave Recognizer object, it is not possible to give both concrete Recognizer object and SuccessFrameGrabberRecognizer that wraps it to same RecognizerBundle - doing so will have the same result as if you have given two instances of same Recognizer type to the RecognizerBundle - it will crash your application.

This recognizer is best for use cases when you need to capture the exact image that was being processed by some other Recognizer object at the time its Result became Valid. When that happens, SuccessFrameGrabber's Result will also become Valid and will contain described image. That image can then be retrieved with getSuccessFrame() method.

BlinkID recognizers

Unless stated otherwise for concrete recognizer, single side BlinkID recognizers from this list can be used in any context, but they work best with BlinkIdUISettings and DocumentScanUISettings, with UIs best suited for document scanning.

Combined recognizers should be used with BlinkIdUISettings. They manage scanning of multiple document sides in the single camera opening and guide the user through the scanning process. Some combined recognizers support scanning of multiple document types, but only one document type can be scanned at a time.

BlinkID recognizer

The BlinkIdRecognizer scans and extracts data from the single side of the supported document. You can find the list of the currently supported documents here. We will continue expanding this recognizer by adding support for new document types in the future. Star this repo to stay updated.

The BlinkIdRecognizer works best with the BlinkIdUISettings and BlinkIdOverlayController.

BlinkID combined recognizer

Use BlinkIdCombinedRecognizer for scanning both sides of the supported document. First, it scans and extracts data from the front, then scans and extracts data from the back, and finally, combines results from both sides. The BlinkIdCombinedRecognizer also performs data matching and returns a flag if the extracted data captured from the front side matches the data from the back. You can find the list of the currently supported documents here. We will continue expanding this recognizer by adding support for new document types in the future. Star this repo to stay updated.

The BlinkIdCombinedRecognizer works best with the BlinkIdUISettings and BlinkIdOverlayController.

Machine Readable Travel Document recognizer

The MrtdRecognizer is used for scanning and data extraction from the Machine Readable Zone (MRZ) of the various Machine Readable Travel Documents (MRTDs) like ID cards and passports. This recognizer is not bound to the specific country, but it can be configured to only return data that match some criteria defined by the MrzFilter.