Cognitive Services - Extract Handwritten Text From An Image Using Computer Vision API With ASP.NET Core And C#

Introduction

In this article, we are going to learn how to extract handwritten text from an image using one of the important Cognitive Services APIs called Computer Vision API. So, we need a valid subscription key for accessing this feature. So before reading this article, you must read our previous articles related to Computer Vision API because we have explained other features of Computer Vision API in our previous article.

This technology is currently in preview and is only available for English text.

Before reading this article, you must read the articles given below for Computer Vision API Knowledge.
Prerequisites
  1. Subscription key ( Azure Portal ).
  2. Visual Studio 2015 or 2017
Subscription Key Free Trial

If you don’t have Microsoft Azure Subscription and want to test the Computer Vision API because it requires a valid Subscription key for processing the image information. Don’t worry!! Microsoft gives a 7-day trial Subscription Key ( Click here ). We can use the Subscription key for testing purposes. If you sign up using the Computer Vision free trial, then your subscription keys are valid for the west-central region (https://westcentralus.api.cognitive.microsoft.com ).

Requirements

These are the major requirements mentioned in the Microsoft docs.
  1. Supported input methods: Raw image binary in the form of an application/octet-stream or image URL.
  2. Supported image formats: JPEG, PNG, BMP.
  3. Image file size: Less than 4 MB.
  4. Image dimensions must be at least 40 x 40, at most 3200 x 3200. 
Computer Vision API

First, we need to log into the Azure Portal with our Azure credentials. Then we need to create an Azure Computer Vision Subscription Key in the Azure portal.

Computer Vision API
 
Click on "Create a resource" on the left side menu and it will open an "Azure Marketplace". There, we can see the list of services. Click "AI + Machine Learning" then click on the "Computer Vision".

Provision a Computer Vision Subscription Key

After clicking the "Computer Vision", it will open another section. There, we need to provide the basic information about Computer Vision API.

Provision a Computer Vision Subscription Key
  • Name - Name of the Computer Vision API ( Eg. HandwrittenApp ).
  • Subscription - We can select our Azure subscription for Computer Vision API creation.
  • Location - We can select our location of the resource group. The best thing is we can choose a location closest to our customer.
  • Pricing tier - Select an appropriate pricing tier for our requirement.
  • Resource group - We can create a new resource group or choose from an existing one.
Now click on the "HandwrittenApp" in dashboard page and it will redirect to the details page of HandwrittenApp ( "Overview" ). Here, we can see the Manage Key ( Subscription key details ) & Endpoint details. Click on the "Show access keys…" links and it will redirect to another page.

Show access keys
 
We can use any of the subscription keys or regenerate the given key for getting image information using Computer Vision API.

Computer Vision API
 
Endpoint

As we mentioned above the location is the same for all the free trial Subscription Keys. In Azure, we can choose available locations while creating a Computer Vision API. We have used the following endpoint in our code.

https://westus.api.cognitive.microsoft.com/vision/v1.0/recognizeText

View Model

The following model contains the API image response information.
  1. using System.Collections.Generic;  
  2. namespace HandwrittenTextApp.Models {  
  3.     public class Word {  
  4.         public List < int > boundingBox {  
  5.             get;  
  6.             set;  
  7.         }  
  8.         public string text {  
  9.             get;  
  10.             set;  
  11.         }  
  12.     }  
  13.     public class Line {  
  14.         public List < int > boundingBox {  
  15.             get;  
  16.             set;  
  17.         }  
  18.         public string text {  
  19.             get;  
  20.             set;  
  21.         }  
  22.         public List < Word > words {  
  23.             get;  
  24.             set;  
  25.         }  
  26.     }  
  27.     public class RecognitionResult {  
  28.         public List < Line > lines {  
  29.             get;  
  30.             set;  
  31.         }  
  32.     }  
  33.     public class ImageInfoViewModel {  
  34.         public string status {  
  35.             get;  
  36.             set;  
  37.         }  
  38.         public RecognitionResult recognitionResult {  
  39.             get;  
  40.             set;  
  41.         }  
  42.     }  
  43. }  
Request URL

We can add additional parameters or request parameters ( optional ) in our API "endPoint" and it will provide more information for the given image.

https://[location].api.cognitive.microsoft.com/vision/v1.0/recognizeText[?handwriting]

Request parameters

These are the following optional parameters available in Computer Vision API.

mode 

mode

The mode will be different for different versions of Vision API. So don’t get confused while we are using Version v1 that is given in our Azure portal. If this parameter is set to "Printed", printed text recognition is performed. If "Handwritten" is specified, handwriting recognition is performed. (Note: This parameter is case sensitive.) This is a required parameter and cannot be empty.

Interface

The "IVisionApiService" contains two signatures for processing or extracting handwritten content in an image. So we have injected this interface in the ASP.NET Core "Startup.cs" class as "AddTransient".
  1. using System.Threading.Tasks;  
  2. namespace HandwrittenTextApp.Business_Layer.Interface {  
  3.     interface IVisionApiService {  
  4.         Task < string > ReadHandwrittenText();  
  5.         byte[] GetImageAsByteArray(string imageFilePath);  
  6.     }  
  7. }  
Vision API Service

The following code will process and generate image information using Computer Vision API and its response is mapped into the "ImageInfoViewModel". We can add the valid Computer Vision API Subscription Key into the following code.
  1. using HandwrittenTextApp.Business_Layer.Interface;  
  2. using HandwrittenTextApp.Models;  
  3. using Newtonsoft.Json;  
  4. using System;  
  5. using System.Collections.Generic;  
  6. using System.IO;  
  7. using System.Linq;  
  8. using System.Net.Http;  
  9. using System.Net.Http.Headers;  
  10. using System.Threading.Tasks;  
  11. namespace HandwrittenTextApp.Business_Layer {  
  12.     public class VisionApiService: IVisionApiService {  
  13.         // Replace <Subscription Key> with your valid subscription key.  
  14.         const string subscriptionKey = "<Subscription Key>";  
  15.         // You must use the same region in your REST call as you used to  
  16.         // get your subscription keys. The paid subscription keys you will get  
  17.         // it from microsoft azure portal.  
  18.         // Free trial subscription keys are generated in the westcentralus region.  
  19.         // If you use a free trial subscription key, you shouldn't need to change  
  20.         // this region.  
  21.         const string endPoint = "https://westus.api.cognitive.microsoft.com/vision/v1.0/recognizeText";  
  22.         ///  
  23.         < summary >  
  24.             /// Gets the handwritten text from the specified image file by using  
  25.             /// the Computer Vision REST API.  
  26.             /// </summary>  
  27.             /// <param name="imageFilePath">The image file with handwritten text.</param>  
  28.             public async Task < string > ReadHandwrittenText() {  
  29.                     string imageFilePath = @ "C:\Users\rajeesh.raveendran\Desktop\vaisakh.jpg";  
  30.                     var errors = new List < string > ();  
  31.                     ImageInfoViewModel responeData = new ImageInfoViewModel();  
  32.                     string extractedResult = "";  
  33.                     try {  
  34.                         HttpClient client = new HttpClient();  
  35.                         // Request headers.  
  36.                         client.DefaultRequestHeaders.Add("Ocp-Apim-Subscription-Key", subscriptionKey);  
  37.                         // Request parameter.  
  38.                         // Note: The request parameter changed for APIv2.  
  39.                         // For APIv1, it is "handwriting=true".  
  40.                         string requestParameters = "mode=Handwritten";  
  41.                         // Assemble the URI for the REST API Call.  
  42.                         string uri = endPoint + "?" + requestParameters;  
  43.                         HttpResponseMessage response;  
  44.                         // Two REST API calls are required to extract handwritten text.  
  45.                         // One call to submit the image for processing, the other call  
  46.                         // to retrieve the text found in the image.  
  47.                         // operationLocation stores the REST API location to call to  
  48.                         // retrieve the text.  
  49.                         string operationLocation;  
  50.                         // Request body.  
  51.                         // Posts a locally stored JPEG image.  
  52.                         byte[] byteData = GetImageAsByteArray(imageFilePath);  
  53.                         using(ByteArrayContent content = new ByteArrayContent(byteData)) {  
  54.                             // This example uses content type "application/octet-stream".  
  55.                             // The other content types you can use are "application/json"  
  56.                             // and "multipart/form-data".  
  57.                             content.Headers.ContentType = new MediaTypeHeaderValue("application/octet-stream");  
  58.                             // The first REST call starts the async process to analyze the  
  59.                             // written text in the image.  
  60.                             response = await client.PostAsync(uri, content);  
  61.                         }  
  62.                         // The response contains the URI to retrieve the result of the process.  
  63.                         if (response.IsSuccessStatusCode) operationLocation = response.Headers.GetValues("Operation-Location").FirstOrDefault();  
  64.                         else {  
  65.                             // Display the JSON error data.  
  66.                             string errorString = await response.Content.ReadAsStringAsync();  
  67.                             //Console.WriteLine("\n\nResponse:\n{0}\n",  
  68.                             // JToken.Parse(errorString).ToString());  
  69.                             return errorString;  
  70.                         }  
  71.                         // The second REST call retrieves the text written in the image.  
  72.                         //  
  73.                         // Note: The response may not be immediately available. Handwriting  
  74.                         // recognition is an async operation that can take a variable amount  
  75.                         // of time depending on the length of the handwritten text. You may  
  76.                         // need to wait or retry this operation.  
  77.                         //  
  78.                         // This example checks once per second for ten seconds.  
  79.                         string result;  
  80.                         int i = 0;  
  81.                         do {  
  82.                             System.Threading.Thread.Sleep(1000);  
  83.                             response = await client.GetAsync(operationLocation);  
  84.                             result = await response.Content.ReadAsStringAsync();  
  85.                             ++i;  
  86.                         }  
  87.                         while (i < 10 && result.IndexOf("\"status\":\"Succeeded\"") == -1);  
  88.                         if (i == 10 && result.IndexOf("\"status\":\"Succeeded\"") == -1) {  
  89.                             Console.WriteLine("\nTimeout error.\n");  
  90.                             return "Error";  
  91.                         }  
  92.                         //If it is success it will execute further process.  
  93.                         if (response.IsSuccessStatusCode) {  
  94.                             // The JSON response mapped into respective view model.  
  95.                             responeData = JsonConvert.DeserializeObject < ImageInfoViewModel > (result, new JsonSerializerSettings {  
  96.                                 NullValueHandling = NullValueHandling.Include,  
  97.                                     Error = delegate(object sender, Newtonsoft.Json.Serialization.ErrorEventArgs earg) {  
  98.                                         errors.Add(earg.ErrorContext.Member.ToString());  
  99.                                         earg.ErrorContext.Handled = true;  
  100.                                     }  
  101.                             });  
  102.                             var linesCount = responeData.recognitionResult.lines.Count;  
  103.                             for (int j = 0; j < linesCount; j++) {  
  104.                                 var imageText = responeData.recognitionResult.lines[j].text;  
  105.                                 extractedResult += imageText + Environment.NewLine;  
  106.                             }  
  107.                         }  
  108.                     } catch (Exception e) {  
  109.                         Console.WriteLine("\n" + e.Message);  
  110.                     }  
  111.                     return extractedResult;  
  112.                 }  
  113.                 ///  
  114.                 < summary >  
  115.                 /// Returns the contents of the specified file as a byte array.  
  116.                 /// </summary>  
  117.                 /// <param name="imageFilePath">The image file to read.</param>  
  118.                 /// <returns>The byte array of the image data.</returns>  
  119.                 public byte[] GetImageAsByteArray(string imageFilePath) {  
  120.                     using(FileStream fileStream = new FileStream(imageFilePath, FileMode.Open, FileAccess.Read)) {  
  121.                         BinaryReader binaryReader = new BinaryReader(fileStream);  
  122.                         return binaryReader.ReadBytes((int) fileStream.Length);  
  123.                     }  
  124.                 }  
  125.     }  
  126. }  
API Response – Based on the given Image

The successful JSON response.
  1. {  
  2.     "status""Succeeded",  
  3.     "recognitionResult": {  
  4.         "lines": [{  
  5.             "boundingBox": [  
  6.                 170,  
  7.                 34,  
  8.                 955,  
  9.                 31,  
  10.                 956,  
  11.                 78,  
  12.                 171,  
  13.                 81  
  14.             ],  
  15.             "text""Memories ! are born not made !",  
  16.             "words": [{  
  17.                 "boundingBox": [  
  18.                     158,  
  19.                     33,  
  20.                     378,  
  21.                     33,  
  22.                     373,  
  23.                     81,  
  24.                     153,  
  25.                     81  
  26.                 ],  
  27.                 "text""Memories"  
  28.             }, {  
  29.                 "boundingBox": [  
  30.                     359,  
  31.                     33,  
  32.                     407,  
  33.                     33,  
  34.                     402,  
  35.                     81,  
  36.                     354,  
  37.                     81  
  38.                 ],  
  39.                 "text""!"  
  40.             }, {  
  41.                 "boundingBox": [  
  42.                     407,  
  43.                     33,  
  44.                     508,  
  45.                     33,  
  46.                     503,  
  47.                     81,  
  48.                     402,  
  49.                     81  
  50.                 ],  
  51.                 "text""are"  
  52.             }, {  
  53.                 "boundingBox": [  
  54.                     513,  
  55.                     33,  
  56.                     662,  
  57.                     33,  
  58.                     657,  
  59.                     81,  
  60.                     508,  
  61.                     81  
  62.                 ],  
  63.                 "text""born"  
  64.             }, {  
  65.                 "boundingBox": [  
  66.                     676,  
  67.                     33,  
  68.                     786,  
  69.                     33,  
  70.                     781,  
  71.                     81,  
  72.                     671,  
  73.                     81  
  74.                 ],  
  75.                 "text""not"  
  76.             }, {  
  77.                 "boundingBox": [  
  78.                     786,  
  79.                     33,  
  80.                     940,  
  81.                     33,  
  82.                     935,  
  83.                     81,  
  84.                     781,  
  85.                     81  
  86.                 ],  
  87.                 "text""made"  
  88.             }, {  
  89.                 "boundingBox": [  
  90.                     926,  
  91.                     33,  
  92.                     974,  
  93.                     33,  
  94.                     969,  
  95.                     81,  
  96.                     921,  
  97.                     81  
  98.                 ],  
  99.                 "text""!"  
  100.             }]  
  101.         }, {  
  102.             "boundingBox": [  
  103.                 181,  
  104.                 121,  
  105.                 918,  
  106.                 112,  
  107.                 919,  
  108.                 175,  
  109.                 182,  
  110.                 184  
  111.             ],  
  112.             "text""Bloom of roses to my heart",  
  113.             "words": [{  
  114.                 "boundingBox": [  
  115.                     162,  
  116.                     123,  
  117.                     307,  
  118.                     121,  
  119.                     298,  
  120.                     185,  
  121.                     154,  
  122.                     187  
  123.                 ],  
  124.                 "text""Bloom"  
  125.             }, {  
  126.                 "boundingBox": [  
  127.                     327,  
  128.                     120,  
  129.                     407,  
  130.                     119,  
  131.                     398,  
  132.                     183,  
  133.                     318,  
  134.                     185  
  135.                 ],  
  136.                 "text""of"  
  137.             }, {  
  138.                 "boundingBox": [  
  139.                     422,  
  140.                     119,  
  141.                     572,  
  142.                     117,  
  143.                     563,  
  144.                     181,  
  145.                     413,  
  146.                     183  
  147.                 ],  
  148.                 "text""roses"  
  149.             }, {  
  150.                 "boundingBox": [  
  151.                     577,  
  152.                     117,  
  153.                     647,  
  154.                     116,  
  155.                     638,  
  156.                     180,  
  157.                     568,  
  158.                     181  
  159.                 ],  
  160.                 "text""to"  
  161.             }, {  
  162.                 "boundingBox": [  
  163.                     647,  
  164.                     116,  
  165.                     742,  
  166.                     115,  
  167.                     733,  
  168.                     179,  
  169.                     638,  
  170.                     180  
  171.                 ],  
  172.                 "text""my"  
  173.             }, {  
  174.                 "boundingBox": [  
  175.                     757,  
  176.                     115,  
  177.                     927,  
  178.                     113,  
  179.                     918,  
  180.                     177,  
  181.                     748,  
  182.                     179  
  183.                 ],  
  184.                 "text""heart"  
  185.             }]  
  186.         }, {  
  187.             "boundingBox": [  
  188.                 190,  
  189.                 214,  
  190.                 922,  
  191.                 201,  
  192.                 923,  
  193.                 254,  
  194.                 191,  
  195.                 267  
  196.             ],  
  197.             "text""Sometimes lonely field as",  
  198.             "words": [{  
  199.                 "boundingBox": [  
  200.                     178,  
  201.                     213,  
  202.                     468,  
  203.                     209,  
  204.                     467,  
  205.                     263,  
  206.                     177,  
  207.                     267  
  208.                 ],  
  209.                 "text""Sometimes"  
  210.             }, {  
  211.                 "boundingBox": [  
  212.                     486,  
  213.                     209,  
  214.                     661,  
  215.                     206,  
  216.                     660,  
  217.                     260,  
  218.                     485,  
  219.                     263  
  220.                 ],  
  221.                 "text""lonely"  
  222.             }, {  
  223.                 "boundingBox": [  
  224.                     675,  
  225.                     206,  
  226.                     840,  
  227.                     203,  
  228.                     839,  
  229.                     257,  
  230.                     674,  
  231.                     260  
  232.                 ],  
  233.                 "text""field"  
  234.             }, {  
  235.                 "boundingBox": [  
  236.                     850,  
  237.                     203,  
  238.                     932,  
  239.                     202,  
  240.                     931,  
  241.                     256,  
  242.                     848,  
  243.                     257  
  244.                 ],  
  245.                 "text""as"  
  246.             }]  
  247.         }, {  
  248.             "boundingBox": [  
  249.                 187,  
  250.                 304,  
  251.                 560,  
  252.                 292,  
  253.                 561,  
  254.                 342,  
  255.                 188,  
  256.                 354  
  257.             ],  
  258.             "text""sky kisses it",  
  259.             "words": [{  
  260.                 "boundingBox": [  
  261.                     173,  
  262.                     302,  
  263.                     288,  
  264.                     300,  
  265.                     288,  
  266.                     353,  
  267.                     173,  
  268.                     355  
  269.                 ],  
  270.                 "text""sky"  
  271.             }, {  
  272.                 "boundingBox": [  
  273.                     288,  
  274.                     300,  
  275.                     488,  
  276.                     295,  
  277.                     488,  
  278.                     348,  
  279.                     288,  
  280.                     353  
  281.                 ],  
  282.                 "text""kisses"  
  283.             }, {  
  284.                 "boundingBox": [  
  285.                     488,  
  286.                     295,  
  287.                     573,  
  288.                     293,  
  289.                     573,  
  290.                     346,  
  291.                     488,  
  292.                     348  
  293.                 ],  
  294.                 "text""it"  
  295.             }]  
  296.         }, {  
  297.             "boundingBox": [  
  298.                 191,  
  299.                 417,  
  300.                 976,  
  301.                 387,  
  302.                 979,  
  303.                 469,  
  304.                 194,  
  305.                 499  
  306.             ],  
  307.             "text""Three years iam gifted with",  
  308.             "words": [{  
  309.                 "boundingBox": [  
  310.                     173,  
  311.                     417,  
  312.                     324,  
  313.                     412,  
  314.                     318,  
  315.                     494,  
  316.                     167,  
  317.                     499  
  318.                 ],  
  319.                 "text""Three"  
  320.             }, {  
  321.                 "boundingBox": [  
  322.                     343,  
  323.                     411,  
  324.                     504,  
  325.                     405,  
  326.                     498,  
  327.                     488,  
  328.                     337,  
  329.                     493  
  330.                 ],  
  331.                 "text""years"  
  332.             }, {  
  333.                 "boundingBox": [  
  334.                     517,  
  335.                     405,  
  336.                     623,  
  337.                     401,  
  338.                     617,  
  339.                     483,  
  340.                     512,  
  341.                     487  
  342.                 ],  
  343.                 "text""iam"  
  344.             }, {  
  345.                 "boundingBox": [  
  346.                     646,  
  347.                     400,  
  348.                     839,  
  349.                     394,  
  350.                     833,  
  351.                     476,  
  352.                     640,  
  353.                     483  
  354.                 ],  
  355.                 "text""gifted"  
  356.             }, {  
  357.                 "boundingBox": [  
  358.                     839,  
  359.                     394,  
  360.                     977,  
  361.                     389,  
  362.                     971,  
  363.                     471,  
  364.                     833,  
  365.                     476  
  366.                 ],  
  367.                 "text""with"  
  368.             }]  
  369.         }, {  
  370.             "boundingBox": [  
  371.                 167,  
  372.                 492,  
  373.                 825,  
  374.                 472,  
  375.                 828,  
  376.                 551,  
  377.                 169,  
  378.                 572  
  379.             ],  
  380.             "text""gud friend happiness !",  
  381.             "words": [{  
  382.                 "boundingBox": [  
  383.                     159,  
  384.                     493,  
  385.                     274,  
  386.                     489,  
  387.                     274,  
  388.                     569,  
  389.                     159,  
  390.                     573  
  391.                 ],  
  392.                 "text""gud"  
  393.             }, {  
  394.                 "boundingBox": [  
  395.                     284,  
  396.                     489,  
  397.                     484,  
  398.                     483,  
  399.                     484,  
  400.                     563,  
  401.                     284,  
  402.                     569  
  403.                 ],  
  404.                 "text""friend"  
  405.             }, {  
  406.                 "boundingBox": [  
  407.                     504,  
  408.                     482,  
  409.                     814,  
  410.                     473,  
  411.                     814,  
  412.                     553,  
  413.                     504,  
  414.                     562  
  415.                 ],  
  416.                 "text""happiness"  
  417.             }, {  
  418.                 "boundingBox": [  
  419.                     794,  
  420.                     474,  
  421.                     844,  
  422.                     472,  
  423.                     844,  
  424.                     552,  
  425.                     794,  
  426.                     554  
  427.                 ],  
  428.                 "text""!"  
  429.             }]  
  430.         }, {  
  431.             "boundingBox": [  
  432.                 167,  
  433.                 608,  
  434.                 390,  
  435.                 628,  
  436.                 387,  
  437.                 664,  
  438.                 163,  
  439.                 644  
  440.             ],  
  441.             "text""50870 W,",  
  442.             "words": [{  
  443.                 "boundingBox": [  
  444.                     159,  
  445.                     603,  
  446.                     321,  
  447.                     623,  
  448.                     310,  
  449.                     661,  
  450.                     147,  
  451.                     641  
  452.                 ],  
  453.                 "text""50870"  
  454.             }, {  
  455.                 "boundingBox": [  
  456.                     309,  
  457.                     621,  
  458.                     409,  
  459.                     634,  
  460.                     397,  
  461.                     672,  
  462.                     297,  
  463.                     659  
  464.                 ],  
  465.                 "text""W,"  
  466.             }]  
  467.         }, {  
  468.             "boundingBox": [  
  469.                 419,  
  470.                 607,  
  471.                 896,  
  472.                 601,  
  473.                 897,  
  474.                 665,  
  475.                 420,  
  476.                 671  
  477.             ],  
  478.             "text""Seperation , sheds",  
  479.             "words": [{  
  480.                 "boundingBox": [  
  481.                     404,  
  482.                     609,  
  483.                     713,  
  484.                     604,  
  485.                     707,  
  486.                     669,  
  487.                     399,  
  488.                     674  
  489.                 ],  
  490.                 "text""Seperation"  
  491.             }, {  
  492.                 "boundingBox": [  
  493.                     703,  
  494.                     604,  
  495.                     749,  
  496.                     604,  
  497.                     743,  
  498.                     669,  
  499.                     698,  
  500.                     669  
  501.                 ],  
  502.                 "text"","  
  503.             }, {  
  504.                 "boundingBox": [  
  505.                     740,  
  506.                     604,  
  507.                     910,  
  508.                     602,  
  509.                     904,  
  510.                     667,  
  511.                     734,  
  512.                     669  
  513.                 ],  
  514.                 "text""sheds"  
  515.             }]  
  516.         }, {  
  517.             "boundingBox": [  
  518.                 161,  
  519.                 685,  
  520.                 437,  
  521.                 688,  
  522.                 436,  
  523.                 726,  
  524.                 160,  
  525.                 724  
  526.             ],  
  527.             "text""blood as in",  
  528.             "words": [{  
  529.                 "boundingBox": [  
  530.                     147,  
  531.                     687,  
  532.                     299,  
  533.                     684,  
  534.                     291,  
  535.                     726,  
  536.                     139,  
  537.                     729  
  538.                 ],  
  539.                 "text""blood"  
  540.             }, {  
  541.                 "boundingBox": [  
  542.                     311,  
  543.                     683,  
  544.                     387,  
  545.                     682,  
  546.                     379,  
  547.                     724,  
  548.                     303,  
  549.                     725  
  550.                 ],  
  551.                 "text""as"  
  552.             }, {  
  553.                 "boundingBox": [  
  554.                     398,  
  555.                     681,  
  556.                     440,  
  557.                     681,  
  558.                     432,  
  559.                     723,  
  560.                     390,  
  561.                     724  
  562.                 ],  
  563.                 "text""in"  
  564.             }]  
  565.         }, {  
  566.             "boundingBox": [  
  567.                 518,  
  568.                 678,  
  569.                 686,  
  570.                 679,  
  571.                 685,  
  572.                 719,  
  573.                 517,  
  574.                 718  
  575.             ],  
  576.             "text""tears !",  
  577.             "words": [{  
  578.                 "boundingBox": [  
  579.                     518,  
  580.                     677,  
  581.                     678,  
  582.                     682,  
  583.                     665,  
  584.                     723,  
  585.                     505,  
  586.                     717  
  587.                 ],  
  588.                 "text""tears"  
  589.             }, {  
  590.                 "boundingBox": [  
  591.                     658,  
  592.                     681,  
  593.                     708,  
  594.                     683,  
  595.                     695,  
  596.                     724,  
  597.                     645,  
  598.                     722  
  599.                 ],  
  600.                 "text""!"  
  601.             }]  
  602.         }, {  
  603.             "boundingBox": [  
  604.                 165,  
  605.                 782,  
  606.                 901,  
  607.                 795,  
  608.                 900,  
  609.                 868,  
  610.                 164,  
  611.                 855  
  612.             ],  
  613.             "text""I can't bear it Especially",  
  614.             "words": [{  
  615.                 "boundingBox": [  
  616.                     145,  
  617.                     785,  
  618.                     191,  
  619.                     786,  
  620.                     184,  
  621.                     862,  
  622.                     138,  
  623.                     861  
  624.                 ],  
  625.                 "text""I"  
  626.             }, {  
  627.                 "boundingBox": [  
  628.                     204,  
  629.                     786,  
  630.                     342,  
  631.                     787,  
  632.                     336,  
  633.                     863,  
  634.                     198,  
  635.                     862  
  636.                 ],  
  637.                 "text""can't"  
  638.             }, {  
  639.                 "boundingBox": [  
  640.                     370,  
  641.                     788,  
  642.                     513,  
  643.                     789,  
  644.                     506,  
  645.                     865,  
  646.                     364,  
  647.                     864  
  648.                 ],  
  649.                 "text""bear"  
  650.             }, {  
  651.                 "boundingBox": [  
  652.                     522,  
  653.                     789,  
  654.                     595,  
  655.                     790,  
  656.                     589,  
  657.                     866,  
  658.                     516,  
  659.                     865  
  660.                 ],  
  661.                 "text""it"  
  662.             }, {  
  663.                 "boundingBox": [  
  664.                     605,  
  665.                     790,  
  666.                     913,  
  667.                     794,  
  668.                     907,  
  669.                     869,  
  670.                     598,  
  671.                     866  
  672.                 ],  
  673.                 "text""Especially"  
  674.             }]  
  675.         }, {  
  676.             "boundingBox": [  
  677.                 165,  
  678.                 874,  
  679.                 966,  
  680.                 884,  
  681.                 965,  
  682.                 942,  
  683.                 164,  
  684.                 933  
  685.             ],  
  686.             "text""final year a bunch of white",  
  687.             "words": [{  
  688.                 "boundingBox": [  
  689.                     155,  
  690.                     872,  
  691.                     306,  
  692.                     875,  
  693.                     294,  
  694.                     936,  
  695.                     143,  
  696.                     933  
  697.                 ],  
  698.                 "text""final"  
  699.             }, {  
  700.                 "boundingBox": [  
  701.                     331,  
  702.                     876,  
  703.                     457,  
  704.                     878,  
  705.                     445,  
  706.                     939,  
  707.                     320,  
  708.                     936  
  709.                 ],  
  710.                 "text""year"  
  711.             }, {  
  712.                 "boundingBox": [  
  713.                     466,  
  714.                     878,  
  715.                     508,  
  716.                     879,  
  717.                     496,  
  718.                     940,  
  719.                     454,  
  720.                     939  
  721.                 ],  
  722.                 "text""a"  
  723.             }, {  
  724.                 "boundingBox": [  
  725.                     525,  
  726.                     879,  
  727.                     676,  
  728.                     882,  
  729.                     664,  
  730.                     943,  
  731.                     513,  
  732.                     940  
  733.                 ],  
  734.                 "text""bunch"  
  735.             }, {  
  736.                 "boundingBox": [  
  737.                     697,  
  738.                     882,  
  739.                     772,  
  740.                     884,  
  741.                     760,  
  742.                     945,  
  743.                     685,  
  744.                     943  
  745.                 ],  
  746.                 "text""of"  
  747.             }, {  
  748.                 "boundingBox": [  
  749.                     785,  
  750.                     884,  
  751.                     970,  
  752.                     888,  
  753.                     958,  
  754.                     948,  
  755.                     773,  
  756.                     945  
  757.                 ],  
  758.                 "text""white"  
  759.             }]  
  760.         }, {  
  761.             "boundingBox": [  
  762.                 174,  
  763.                 955,  
  764.                 936,  
  765.                 960,  
  766.                 935,  
  767.                 1006,  
  768.                 173,  
  769.                 1001  
  770.             ],  
  771.             "text""roses to me . I Loved it ! !",  
  772.             "words": [{  
  773.                 "boundingBox": [  
  774.                     164,  
  775.                     953,  
  776.                     348,  
  777.                     954,  
  778.                     341,  
  779.                     1002,  
  780.                     157,  
  781.                     1001  
  782.                 ],  
  783.                 "text""roses"  
  784.             }, {  
  785.                 "boundingBox": [  
  786.                     376,  
  787.                     955,  
  788.                     445,  
  789.                     955,  
  790.                     437,  
  791.                     1003,  
  792.                     368,  
  793.                     1003  
  794.                 ],  
  795.                 "text""to"  
  796.             }, {  
  797.                 "boundingBox": [  
  798.                     449,  
  799.                     955,  
  800.                     537,  
  801.                     956,  
  802.                     529,  
  803.                     1004,  
  804.                     442,  
  805.                     1003  
  806.                 ],  
  807.                 "text""me"  
  808.             }, {  
  809.                 "boundingBox": [  
  810.                     518,  
  811.                     956,  
  812.                     564,  
  813.                     957,  
  814.                     557,  
  815.                     1005,  
  816.                     511,  
  817.                     1004  
  818.                 ],  
  819.                 "text""."  
  820.             }, {  
  821.                 "boundingBox": [  
  822.                     569,  
  823.                     957,  
  824.                     615,  
  825.                     957,  
  826.                     607,  
  827.                     1005,  
  828.                     561,  
  829.                     1005  
  830.                 ],  
  831.                 "text""I"  
  832.             }, {  
  833.                 "boundingBox": [  
  834.                     629,  
  835.                     957,  
  836.                     799,  
  837.                     959,  
  838.                     791,  
  839.                     1007,  
  840.                     621,  
  841.                     1005  
  842.                 ],  
  843.                 "text""Loved"  
  844.             }, {  
  845.                 "boundingBox": [  
  846.                     817,  
  847.                     959,  
  848.                     886,  
  849.                     960,  
  850.                     879,  
  851.                     1008,  
  852.                     810,  
  853.                     1007  
  854.                 ],  
  855.                 "text""it"  
  856.             }, {  
  857.                 "boundingBox": [  
  858.                     881,  
  859.                     960,  
  860.                     927,  
  861.                     960,  
  862.                     920,  
  863.                     1008,  
  864.                     874,  
  865.                     1008  
  866.                 ],  
  867.                 "text""!"  
  868.             }, {  
  869.                 "boundingBox": [  
  870.                     909,  
  871.                     960,  
  872.                     955,  
  873.                     960,  
  874.                     948,  
  875.                     1008,  
  876.                     902,  
  877.                     1008  
  878.                 ],  
  879.                 "text""!"  
  880.             }]  
  881.         }, {  
  882.             "boundingBox": [  
  883.                 613,  
  884.                 1097,  
  885.                 680,  
  886.                 1050,  
  887.                 702,  
  888.                 1081,  
  889.                 635,  
  890.                 1129  
  891.             ],  
  892.             "text""by",  
  893.             "words": [{  
  894.                 "boundingBox": [  
  895.                     627,  
  896.                     1059,  
  897.                     683,  
  898.                     1059,  
  899.                     681,  
  900.                     1107,  
  901.                     625,  
  902.                     1107  
  903.                 ],  
  904.                 "text""by"  
  905.             }]  
  906.         }, {  
  907.             "boundingBox": [  
  908.                 320,  
  909.                 1182,  
  910.                 497,  
  911.                 1191,  
  912.                 495,  
  913.                 1234,  
  914.                 318,  
  915.                 1224  
  916.             ],  
  917.             "text""Vaisak",  
  918.             "words": [{  
  919.                 "boundingBox": [  
  920.                     309,  
  921.                     1183,  
  922.                     516,  
  923.                     1186,  
  924.                     492,  
  925.                     1229,  
  926.                     286,  
  927.                     1227  
  928.                 ],  
  929.                 "text""Vaisak"  
  930.             }]  
  931.         }, {  
  932.             "boundingBox": [  
  933.                 582,  
  934.                 1186,  
  935.                 964,  
  936.                 1216,  
  937.                 961,  
  938.                 1264,  
  939.                 578,  
  940.                 1234  
  941.             ],  
  942.             "text""Viswanathan",  
  943.             "words": [{  
  944.                 "boundingBox": [  
  945.                     574,  
  946.                     1186,  
  947.                     963,  
  948.                     1218,  
  949.                     945,  
  950.                     1265,  
  951.                     556,  
  952.                     1232  
  953.                 ],  
  954.                 "text""Viswanathan"  
  955.             }]  
  956.         }, {  
  957.             "boundingBox": [  
  958.                 289,  
  959.                 1271,  
  960.                 997,  
  961.                 1279,  
  962.                 996,  
  963.                 1364,  
  964.                 288,  
  965.                 1356  
  966.             ],  
  967.             "text""( Menonpara, Palakkad )",  
  968.             "words": [{  
  969.                 "boundingBox": [  
  970.                     274,  
  971.                     1264,  
  972.                     324,  
  973.                     1265,  
  974.                     306,  
  975.                     1357,  
  976.                     256,  
  977.                     1356  
  978.                 ],  
  979.                 "text""("  
  980.             }, {  
  981.                 "boundingBox": [  
  982.                     329,  
  983.                     1265,  
  984.                     679,  
  985.                     1273,  
  986.                     661,  
  987.                     1364,  
  988.                     311,  
  989.                     1357  
  990.                 ],  
  991.                 "text""Menonpara,"  
  992.             }, {  
  993.                 "boundingBox": [  
  994.                     669,  
  995.                     1273,  
  996.                     979,  
  997.                     1279,  
  998.                     961,  
  999.                     1371,  
  1000.                     651,  
  1001.                     1364  
  1002.                 ],  
  1003.                 "text""Palakkad"  
  1004.             }, {  
  1005.                 "boundingBox": [  
  1006.                     969,  
  1007.                     1279,  
  1008.                     1019,  
  1009.                     1280,  
  1010.                     1001,  
  1011.                     1371,  
  1012.                     951,  
  1013.                     1370  
  1014.                 ],  
  1015.                 "text"")"  
  1016.             }]  
  1017.         }]  
  1018.     }  
  1019. }  
Download
Output

Handwritten content from an image using Computer Vision API is given below. The content is extracted (around 99.99%) from the given image. If any failure occurs in detecting the image, it means that the Vision algorithm is not able to identify the written content.

Output
 
Note

Thank you, Vaisakh Viswanathan ( The author of the poem ).

Reference
See Also

You can download other ASP.NET Core source codes from MSDN Code, using the link, mentioned below. 

Summary

From this article, we have learned to extract handwritten content from an image using One of the important Cognitive Services APIs (Computer Vision API ). I hope this article is useful for all Azure Cognitive Services API beginners.


Similar Articles