Plugin.Xamarin.OCR and Plugin.Maui.OCR provide the ability to do simple text from image OCR using nothing but platform APIs.
YES. Let me know how it works for you.
| Platform | iOS | Android | Windows | macOS |
|---|---|---|---|---|
| Xamarin | Yes | Yes | WIP | WIP |
| MAUI | Yes | Yes | Yes | Yes |
Why am I making this? I'm doing this because I want to make it easier for developers to do OCR in their apps. I want to make it so that you can just use this plugin and not have to worry about the platform specifics.
Too many times I've tried to do OCR and had to wrestle with external dependencies like Tesseract (with its dependencies Leptonica, etc) and these types of native dependencies can be a real pain to work with.
Well, I still have to maintain a Xamarin app that uses Tesseract and I'm tired of all the problems that come with it. I want to make it easier for myself and others to do OCR in their apps.
Available on NuGet for MAUI and Xamarin.
Install with the dotnet CLI: dotnet add package Plugin.Maui.OCR or dotnet add package Plugin.Xamarin.OCR, or through the NuGet Package Manager in Visual Studio.
| Platform | Minimum Version Supported |
|---|---|
| iOS | 13+ |
| macOS | 10.15+ |
| Android | 5.0 (API 21) |
| Windows | 11 and 10 version 1809+ |
One of the more common things I do with OCR is recognize a text pattern. For example, I might want to read a date, a phone number or an email address. This is where the OcrPatternConfig class comes in.
Let's say you want to recognize an Ontario Health Card Number (HCN) in the text of your image. Numbers of those types have some specific qualities that make it easy to match.
- An Ontario HCN is 10 digits long.
- The number must be Luhn valid (meaning it has a check digit and it's correct).
To do this, you can create an OcrPatternConfig object like so:
bool IsValidLuhn(string number)
{
// Convert the string to an array of digits
int[] digits = number.Select(d => int.Parse(d.ToString())).ToArray();
int checkDigit = 0;
// Luhn algorithm implementation
for (int i = digits.Length - 2; i >= 0; i--)
{
int currentDigit = digits[i];
if ((digits.Length - 2 - i) % 2 == 0) // check if it's an even index from the right
{
currentDigit *= 2;
if (currentDigit > 9)
{
currentDigit -= 9;
}
}
checkDigit += currentDigit;
}
return (10 - (checkDigit % 10)) % 10 == digits.Last();
}
var ohipPattern = new OcrPatternConfig(@"\d{10}", IsLuhnValid);
var options = new OcrOptions.Builder().SetTryHard(true).SetPatternConfig(ohipPattern).Build();
var result = await OcrPlugin.Default.RecognizeTextAsync(imageData, options);
var patientHcn = result.MatchedValues.FirstOrDefault(); // This will be the HCN (and only the HCN) if it's foundFor MAUI, to initialize make sure you use the MauiAppBuilder extension UseOcr() like so:
public static class MauiProgram
{
public static MauiApp CreateMauiApp()
{
var builder = MauiApp.CreateBuilder();
builder
.UseMauiApp<App>()
.ConfigureFonts(fonts =>
{
fonts.AddFont("OpenSans-Regular.ttf", "OpenSansRegular");
fonts.AddFont("OpenSans-Semibold.ttf", "OpenSansSemibold");
}).
UseOcr(); // <-- add this line
return builder.Build();
}
}And then you can just inject IOcrService into your classes and use it like so:
/// <summary>
/// Takes a photo and processes it using the OCR service.
/// </summary>
/// <param name="photo">The photo to process.</param>
/// <returns>The OCR result.</returns>
private async Task<OcrResult> ProcessPhoto(FileResult photo)
{
// Open a stream to the photo
using var sourceStream = await photo.OpenReadAsync();
// Create a byte array to hold the image data
var imageData = new byte[sourceStream.Length];
// Read the stream into the byte array
await sourceStream.ReadAsync(imageData);
// Process the image data using the OCR service
return await _ocr.RecognizeTextAsync(imageData);
}For Xamarin, if you have some kind of DI framework in place then you can just register the OcrPlugin with it.
public App()
{
InitializeComponent();
DependencyService.RegisterSingleton(OcrPlugin.Default);
MainPage = new MainPage();
}If you don't have a DI framework in place, you can use the OcrPlugin.Default property to access the IOcrService instance.
private readonly IOcrService _ocr;
public MainPage(IOcrService? ocr)
{
InitializeComponent();
_ocr = ocr ?? OcrPlugin.Default;
}The IOcrService interface exposes the following methods:
public interface IOcrService
{
event EventHandler<OcrCompletedEventArgs> RecognitionCompleted;
IReadOnlyCollection<string> SupportedLanguages { get; }
Task InitAsync(CancellationToken ct = default);
Task<OcrResult> RecognizeTextAsync(byte[] imageData, bool tryHard = false, CancellationToken ct = default);
Task<OcrResult> RecognizeTextAsync(byte[] imageData, OcrOptions options, CancellationToken ct = default);
Task StartRecognizeTextAsync(byte[] imageData, OcrOptions options, CancellationToken ct = default);
}
public class OcrResult
{
public bool Success { get; set; }
public string AllText { get; set; }
public IList<OcrElement> Elements { get; set; } = new List<OcrElement>();
public IList<string> Lines { get; set; } = new List<string>();
public class OcrElement
{
public string Text { get; set; }
public float Confidence { get; set; }
// Useful for bounding boxes
public int X { get; set; }
public int Y { get; set; }
public int Height { get; set; }
public int Width { get; set; }
}
}Before you can start using Feature, you will need to request the proper permissions on each platform.
If you're handling camera, you'll need the usual permissions for that.
If you're handling camera, you'll need the usual permissions for that. The only extra part you'll want in the AndroidManifest.xml is the following:
<application ..>
<meta-data android:name="com.google.mlkit.vision.DEPENDENCIES" android:value="ocr" />
</application>This will cause the model necessary to be installed when the application is installed.
The OcrOptions class provides a flexible way to configure OCR settings. You can use the OcrOptions.Builder class to create instances of OcrOptions with various configurations.
The OcrOptions class holds the configuration for OCR operations.
public class OcrOptions
{
public string? Language { get; }
public bool TryHard { get; }
public List<OcrPatternConfig> PatternConfigs { get; }
public CustomOcrValidationCallback? CustomCallback { get; }
private OcrOptions(string? language, bool tryHard, List<OcrPatternConfig> patternConfigs, CustomOcrValidationCallback? customCallback)
{
Language = language;
TryHard = tryHard;
PatternConfigs = patternConfigs;
CustomCallback = customCallback;
}
public class Builder
{
private string? _language;
private bool _tryHard;
private List<OcrPatternConfig> _patternConfigs = new List<OcrPatternConfig>();
private CustomOcrValidationCallback? _customCallback;
public Builder SetLanguage(string language)
{
_language = language;
return this;
}
public Builder SetTryHard(bool tryHard)
{
_tryHard = tryHard;
return this;
}
public Builder AddPatternConfig(OcrPatternConfig patternConfig)
{
_patternConfigs.Add(patternConfig);
return this;
}
public Builder SetPatternConfigs(List<OcrPatternConfig> patternConfigs)
{
_patternConfigs = patternConfigs ?? new List<OcrPatternConfig>();
return this;
}
public Builder SetCustomCallback(CustomOcrValidationCallback customCallback)
{
_customCallback = customCallback;
return this;
}
public OcrOptions Build()
{
return new OcrOptions(_language, _tryHard, _patternConfigs, _customCallback);
}
}
}Using the OcrOptions.Builder to create an OcrOptions instance is straightforward and flexible:
var options = new OcrOptions.Builder()
.SetLanguage("en-US")
.SetTryHard(true)
.AddPatternConfig(new OcrPatternConfig(@"\d{10}"))
.SetCustomCallback(myCustomCallback)
.Build();You will first need to register the OcrPlugin with the MauiAppBuilder following the same pattern that the .NET MAUI Essentials libraries follow.
builder.Services.AddSingleton(OcrPlugin.Default);You can then enable your classes to depend on IOcrService as per the following example.
public class OcrViewModel
{
readonly IOcrService _ocr;
public OcrViewModel(IOcrService? ocr)
{
_ocr = ocr ?? OcrPlugin.Default;
}
public void DoSomeOcr()
{
byte[] imageData = GetImageData();
var result = await _ocr.RecognizeTextAsync(imageData);
}
}Alternatively if you want to skip using the dependency injection approach you can use the Feature.Default property.
public class OcrViewModel
{
public void DoSomeOcr()
{
byte[] imageData = GetImageData();
var result = await OcrPlugin.Default.RecognizeTextAsync(imageData);
}
}Once you have the OCR instance, you can interact with it in the following ways:
This event is fired when the OCR service has completed recognizing text from an image. The event args contain the OcrResult object. Only fires if the StartRecognizeTextAsync method is called.
A list of supported languages for the OCR service. This is populated after calling InitAsync. Allows you to know what language codes can be used in OcrOptions.
Initialize the feature. If supported on the platform (like iOS), SupportedLanguages will be populated with the available languages.
Recognize text from an image. Specify "tryHard" if you want to tell the platform API to do a better job (fast vs accurate, and use language correction (ios/mac)) though it seems very accurate normally.
Recognize text from an image. OcrOptions contains options for the OCR service, including the language to use and whether to try hard.
Start recognizing text from an image. This is a task that will fire the RecognitionCompleted event when it completes with the result.
Thanks to the great Gerald Versluis for making an amazing video about using this project: https://www.youtube.com/watch?v=alY_6Qn0_60