Face API Cognitive Service - Day One - Face API On Azure Portal

Preface

 

This article series is a complete end to end tutorial that will explain the concept of face recognition and face detection using modern AI-based Azure cognitive service; i.e. Azure’s Face API service.

 

Introduction

 

With the world changing at a rapid pace in the technology space, we see ourselves surrounded by machine learning and artificial intelligence. AI-based technologies are booming and programming languages have become a medium to facilitate the techniques.

 

The Face API enables developers to easily perform face identification, detection and sentiment analysis with simple HTTP calls. This functionality could be directly integrated into any app and could be used to solve n number of business problems. In this article, we’ll cover the Azure Face API topic in detail. We’ll set up an Azure account, a Face API account and get the Face API running within seconds. We’ll see what all are the API’s that could be used to get accustomed with Face API. We’ll also integrate those HTTP services to an application that performs face detection, face grouping, face identification and finding similar faces as well from the group of faces.

 

You’ll see how easily you can detect a face in the picture, get its properties like - does a person whose face is detected wear glasses in the picture? Is the person a male or a female? What's the age of the person? And other attributes like facial hair, eyes, nose, lips, etc. Not only this, but we also get the on-demand details to find the facial expressions like is the person, happy, sad or angry and what is the scale of the emotion detected. Get more details here. So, it's a lot of fun altogether.

 

Tutorial Series

 

The entire series on learning Face API Cognitive Services is divided into four parts. The first part focuses on Azure Functions, Serverless Computing, and creating and testing Face API on Azure Portal.

 

The second part explains the use of Face API SDK. The third part will be focused on face identification, where the person groups will be created, and the identification of faces would be done via training the models i.e. via machine learning techniques. The fourth part is the most interesting part that gives a walkthrough of Face classification application which performs face detection, identification, grouping and finding look-alike faces. Following is the four-part series.

1. Face API Cognitive Service Day 1: Face API on Azure Portal.
2. Face API Cognitive Service Day 2: Exploring Face API SDK.
3. Face API Cognitive Service Day 3: Face Identification using Face API.
4. Face API Cognitive Service Day 4: Face classification using Face API.

Azure

 

Azure is a cloud platform from Microsoft and provides numerous resources in context on cloud computing. One of the resources is virtual machine; i.e. a fully functional machine of your choice with the choice of your configurations and operating system could be created within seconds with just a few clicks and you can access the machine remotely from anywhere with your secure credentials and do whatever you want; e.g. hosting your web site, developing applications, creating production or test environment for your software, etc. Let’s see step by step how we can achieve that.

 

Azure Functions and Serverless Computing

 

Talking about the definitions from here,

 

“Azure Functions is the serverless computing service hosted on the Microsoft Azure public cloud. Azure Functions, and serverless computing, in general, is designed to accelerate and simplify application development.”

 

The first thing to note about Azure Functions is that they operate in a fully-managed environment. Essentially what this means is that we don't have to go and create or manage virtual machines. Microsoft provides and provisions all the underlying hardware that our Azure Functions are based on. One of the benefits of this is that it gives us high reliability because we don't have to manually manage the underlying infrastructure. Another benefit is that we don't have to worry about going and applying security patches to the underlying infrastructure. Microsoft will take care of this for us.

 

It is, however, up to us to ensure that the systems that we're building on top of Azure Functions are appropriately secured and managed. Another benefit of this fully- managed environment is that it gives us automatic scaling. The Azure Functions platform will automatically scale our functions out to deal with variations in incoming request volume. If we're executing our Azure Functions on the Consumption plan, then we're only going to pay when the functions are executed. Another benefit of Azure Functions is that we need to write less boilerplate code, and one of the reasons for this is that our Azure Functions can easily integrate with a range of Azure services. For example, integrating our function with Azure Blob storage is as simple as configuring the correct attributes on our C# code.

 

 

 

Azure Account Setup

 

If one does not have a paid Azure account, one could leverage Azure’s new account’s benefits of giving $200 credits. That means if you are new to Azure and want to play around with its free trial, you’ll get $200 credits that you can use to explore Azure. If you are new to Azure and do not have an account, follow the following process, else directly login to your portal.

1. Open the Azure web site i.e. azure.microsoft.com
   
   

2. Click on Start free to create your free Azure account and get $200 as credits.
   
   

Creating an account and claiming $200 would need your credit/debit card for verification purposes only and will not deduct any amount from your card. You can play around with this credit and account for 30 days. You’ll see the signup page, where you fill all your information and signup step by step. Once signed-up successfully, you’ll see the link to the portal as shown below.

 

 

 

Click on the portal and you land up on the dashboard and are ready to use/play around with Azure.

 

 

 

Face API

 

Azure AI Services consist of many different services. The one that probably gets the most attention is Cognitive Services, which is Microsoft's prebuilt AI. The Face API is an example of a cognitive service, so it lives squarely in this category.

 

The Face API has five major feature areas. The most fundamental operation is face detection. The detection will provide the exact location of where faces appear in images, it will also provide metadata about the faces that it detects, such as age estimates, gender, and numerous face attributes such as facial hair, and whether the person has glasses.

 

Emotion metadata is detected here as well. Identification is what we think of as facial recognition. It can identify a specific known individual, provided that the Face API has been pre-trained to identify that individual.

 

Verification is a type of facial recognition, but with Verification, we're just trying to match a face against a specific person, not a group of people. The best example of this is when you try to unlock your computer or phone using your face.

 

Face Grouping attempts to group faces of the same individual provided a list of faces.

 

Finally, Find Similar functionality will find similar faces given a list of faces. This will either try to find the exact person or, depending on which mode you've selected, a face that looks similar. This second mode would be used, for example, if you're trying to find your celebrity look-alike. We're going to see each one of these features in action in this article.

 

Creating the Face API on Azure Portal

 

In this section, we'll get up and running with the Face API on Azure Portal. We’ll go step by step so that we do not miss any single detail.

1. Log in to the Azure portal and click on “Create a resource”. You can find this option on the left side panel.
   
   

2. On the screen select “AI + Machine Learning” and you’ll see all the ML offerings provided by Azure. There we see an offering with the name as “Face”. Select that option.
   
   

3. Once you click on that Face option, you’ll be asked to create the Face API and that’s what we wanted. Provide the necessary details like your API name (provide whatever is convenient for you to remember. I gave the name “face-api” to my API). Provide the subscription detail, the location. In my case, I chose Central India. Choose the pricing tier, you can get the list of pricing tiers by navigating to the link shown in the following image that says, “View full pricing details”. I chose F0. Choose a resource group or create a new one. A resource group is just for logical separation of your resources. For e.g. I would name my resource group as “azure-machine learning” because I want to keep all my AI or ML related apps and services in this group.
   
   

4. Once you fill all the details and submit the information, it will take a while to deploy the service for you (few seconds) and you’ll get the message the deployment is done, and a button will be shown that says “Go to resource” that will navigate you to the created Face API. You can also go to the created face API from the main portal’s page by clicking on the resources link or dashboard.
   
   

5. From the Dashboard, when you go to the created Face API resource, you can find the endpoint as shown in the following image. Copy the endpoint in a notepad or so. We’ll need that soon when we test the API.
   
   

6. One the left side panel under Resource Management on the same screen you can find the security key. Copy that as well and place that in notepad. We’ll need that as well.
   
   

Testing the Face API

 

Since the API is an endpoint and behaves like a REST service, we can use Postman to test the API. Open Postman and install if it is not installed on your computer. You can get the latest version of the app here.

 

 

Detect Call

 

We’ll start with making a detect call to the API. You can find all the endpoints details at this location that explains what all calls can be made to the API endpoint. These calls include find similar faces, detect the faces from the image, group faces and a lot more. We’ll cover each in detail one by one. Let’s start with detect.

1. Once the postman app is on, we can test the API. Copy the endpoint that we secured in notepad and paste the same to the Postman request. Make sure the request is a POST request. Append the “detect” at the end of the endpoint because it is a detect call as shown in the following image. In the Headers section, we must add a couple of headers, Content-Type, and we'll make this application/json. And the other one we're going to do for authentication purpose is Ocp-Apim-Subscription-Key, and this is the header that the Face API wants for authentication. Now we need to paste our key in here. Get the key that we copied in the last step and paste it in the value for the Ocp-Apim-Subscription-Key key.
   
   

2. In the Body tab i.e. just next to Headers in Postman, select “raw” as the type of the content and in the body section, provide a JSON that contains the URL of a group of pictures. I used a location on a public server to place my image. You can use Azure blob storage or any other cloud provider to store your image. Make sure your image is publicly accessible and even if it is not, you can also send the image as an upload while testing the API. You just need to change the data type from raw to binary and upload your image. And the request content type as application/octet-stream.
   
   
   
   Following is the image I used that is located at public server that contains faces of my friends and family members.
   
   

3. Press the blue send button to make a request and test the API. We see that within no seconds we get a response from the API. That first proves that our API is up and running and secondly it is working as well. We get the response as a JSON that gives us the information of each detected face in that picture and the attributes of the face such as the dimensions in the form of faceRectangle attribute and each face is uniquely detected and have a unique faceId as well.
   
   

On the following cognitive services URL, you’ll get to see all the calls you can make to the API apart from only detect URL.

 

 

Scroll down to check what all query parameters you can send with the detect call itself to get more information about the faces.

 

 

Face Landmarks

 

“An array of 27-point face landmarks pointing to the important positions of face components. To return this, it requires "returnFaceLandmarks" parameter to be true.”

 

Face landmarks is a query string parameter that could be sent along with detect call to get the positions of face components like eyes, nose, lips, etc. Just in your detect call, add a query string parameter named “returnFaceLandmarks” and give it a value as true as shown below. When you click on send, you get the response with a more detailed JSON telling you the position of each face component like pupil, eyebrows, mouth, nose, etc. Try to have fun.

 

 

Face Attributes

 

Getting the following details from the Azure documentation link,“Face Attributes,

• age: an estimated "visual age" number in years. It is how old a person looks like rather than the actual biological age.
• gender: male or female.
• smile: smile intensity, a number between [0,1].
• facialHair: return lengths in three facial hair areas: mustache, beard, and sideburns. Length is a number between [0,1]. 0 for no facial hair in this area, 1 for long or very thick facial hairs in this area.
• headPose: 3-D roll/yaw/pitch angles for face direction.
• glasses: glasses type. Values include 'NoGlasses', 'ReadingGlasses', 'Sunglasses', 'SwimmingGoggles'.
• emotion: emotion intensity, including neutral, anger, contempt, disgust, fear, happiness, sadness and surprise.
• hair: a group of hair values indicating whether the hair is visible, bald, and hair color if the hair is visible.
• makeup: whether eye, lip areas are made-up or not.
• accessories: accessories around the face, including 'headwear', 'glasses' and 'mask'. An empty array means no accessories detected. Note this is after a face is detected. Large mask could result in no face to be detected.
• blur: the face is blurry or not. Level returns 'Low', 'Medium' or 'High'. Value returns a number between [0,1], the larger the blurrier.
• exposure: face exposure level. Level returns 'GoodExposure', 'OverExposure' or 'UnderExposure'.
• noise: noise level of face pixels. Level returns 'Low', 'Medium' and 'High'. Value returns a number between [0,1], the larger the noisier”

Face attributes is also a query string parameter that could be sent along with the detect call. It returns the attribute values as given in the list above. For e.g., if you say that returnFaceAttributes = age, it will return the age of the face detected.

 

 

Similarly, you can ask for gender, emotions, check whether a person is wearing glass or not, and if the person is smiling or not. With the cognitive service AI technique, you get all these face attributes from the image.

 

 

Conclusion

 

This was an end to end article to show the capabilities of Azure Face API; i.e., one of the Azure’s cognitive service. The API is quite intelligent and strong to leverage the AI and machine learning capabilities and perform the actions. We saw in detail how to create an Azure account, how to create a Face API and get it up and running.

 

 

 

References

• https://azure.microsoft.com/en-in/services/cognitive-services/face/