Xamarin.Android - Text Recognition By Mobile Camera

Introduction

In this article, I shall show you how to build a text recognition application by using the device camera. Through this tutorial, I would like to present to readers the amazing feature of Mobile Vision API: Text recognition by using a mobile camera.

Text Recognition 

Mobile Vision is an API which helps us to find the text in pictures and video streams to observe the content checked in that.

Structure of Text

The text recognizer breaks the content into pieces, lines, and words.

1. A word is a contiguous arrangement of alphanumeric characters on a similar vertical axis.
2. A line is an adjoining set of words on a similar vertical axis.
3. A chunk is a bordering set of text lines.

Step 1

Open Visual Studio. Go to New Project-> Templates-> Visual C#-> Android-> Blank app. Select Blank app. Give the project

a name like TextRecognizerByCamera.

ProjectName: TextRecognizerByCamera



Step 2

First of all, we need to add a component that is required for text recognition. Open Solution Explorer and go to Components -> Get More Components. In this way, you can move on Xamarin Components Store, then search for Vision and click "Add to App".

 

Step 3

We need a permission from the device because we shall be using the device’s camera to capture texts. Please add CAMERA permission to your AndroidManifest.xml.

Let's open Solution Explorer-> Properties-> AndroidManifest and let's add the code inside application tags.

 

 

Camera Permission

<uses-permission android:name="android.permission.CAMERA" />
<application android:allowBackup="true" android:label="@string/app_name">
    <meta-data android:name="com.google.android.gms.vision.DEPENDENCIES" android:value="ocr" />
</application>

Step 4

Open Solution Explorer-> Project Name-> Resources-> Layout-> Main.axml and add the following code.

The layout will have a SurfaceView in order to display the preview frames captured by the camera. I also added a TextView to display the contents of the recognized text.

(FileName: Main.axml)

XAML Code

<?xml version="1.0" encoding="utf-8"?>
<RelativeLayout xmlns:android="http://schemas.android.com/apk/res/android" android:padding="16dp" android:layout_width="match_parent" android:layout_height="match_parent">
    <SurfaceView android:id="@+id/surface_view" android:layout_width="match_parent" android:layout_height="match_parent" />
    <TextView android:id="@+id/txtview" android:text="No Text" android:textSize="20sp" android:textColor="@android:color/white" android:layout_width="match_parent" android:layout_height="wrap_content" android:layout_alignParentBottom="true" />
</RelativeLayout>

Layout

 

Step 6

Open Solution Explorer-> Project Name-> MainActivity and add the following code to main activity and use appropriate namespaces.

 

Complete Code of Main Activity

using Android.App;
using Android.Widget;
using Android.OS;
using Android.Views;
using Android.Gms.Vision;
using Android.Gms.Vision.Texts;
using Android.Util;
using Android.Graphics;
using Android.Runtime;
using Android.Support.V4.App;
using Android;
using Android.Content.PM;
using static Android.Gms.Vision.Detector;
using System.Text;
namespace CameraTextRecognizer {
    [Activity(Label = "CameraTextRecognizer", MainLauncher = true)]
    public class MainActivity: Activity, ISurfaceHolderCallback, IProcessor {
        private SurfaceView cameraView;
        private TextView txtView;
        private CameraSource cameraSource;
        private
        const int RequestCameraPermissionID = 1001;
        public override void OnRequestPermissionsResult(int requestCode, string[] permissions, [GeneratedEnum] Permission[] grantResults) {
            switch (requestCode) {
                case RequestCameraPermissionID:
                    {
                        if (grantResults[0] == Permission.Granted) {
                            cameraSource.Start(cameraView.Holder);
                        }
                    }
                    break;
            }
        }
        protected override void OnCreate(Bundle savedInstanceState) {
            base.OnCreate(savedInstanceState);
            // Set our view from the "main" layout resource
            SetContentView(Resource.Layout.Main);
            cameraView = FindViewById < SurfaceView > (Resource.Id.surface_view);
            txtView = FindViewById < TextView > (Resource.Id.txtview);
            TextRecognizer txtRecognizer = new TextRecognizer.Builder(ApplicationContext).Build();
            if (!txtRecognizer.IsOperational) {
                Log.Error("Main Activity", "Detector dependencies are not yet available");
            } else {
                cameraSource = new CameraSource.Builder(ApplicationContext, txtRecognizer).SetFacing(CameraFacing.Back).SetRequestedPreviewSize(1280, 1024).SetRequestedFps(2.0 f).SetAutoFocusEnabled(true).Build();
                cameraView.Holder.AddCallback(this);
                txtRecognizer.SetProcessor(this);
            }
        }
        public void SurfaceChanged(ISurfaceHolder holder, [GeneratedEnum] Format format, int width, int height) {}
        public void SurfaceCreated(ISurfaceHolder holder) {
            if (ActivityCompat.CheckSelfPermission(ApplicationContext, Manifest.Permission.Camera) != Android.Content.PM.Permission.Granted) {
                //Request permission
                ActivityCompat.RequestPermissions(this, new string[] {
                    Android.Manifest.Permission.Camera
                }, RequestCameraPermissionID);
                return;
            }
            cameraSource.Start(cameraView.Holder);
        }
        public void SurfaceDestroyed(ISurfaceHolder holder) {
            cameraSource.Stop();
        }
        public void ReceiveDetections(Detections detections) {
            SparseArray items = detections.DetectedItems;
            if (items.Size() != 0) {
                txtView.Post(() => {
                    StringBuilder strBuilder = new StringBuilder();
                    for (int i = 0; i < items.Size(); ++i) {
                        strBuilder.Append(((TextBlock) items.ValueAt(i)).Value);
                        strBuilder.Append("\n");
                    }
                    txtView.Text = strBuilder.ToString();
                });
            }
        }
        public void Release() {}
    }
}

Output 

Running this project, and scanning a text block on the paper, you may have result like this.