Cognitive Service Emotion API In UWP With Azure, XAML And C#

Before reading this article, please go through the following articles.

1. Introduction To Universal Windows Platform (UWP) App Development Using Windows 10 And Visual Studio 2015
2. How To Create Microsoft Cognitive Service Emotion API On Azure Portal
   
3. Cognitive Service Face API, Using UWP With Azure, XAML And C#

Microsoft Cognitive Services let you build apps with powerful algorithms,  using just a few lines of code. They work across devices and platforms (such as iOS, Android, and Windows), keep improving, and are easy to set up.

The emotions detected are anger, contempt, disgust, fear, happiness, neutral, sadness, and surprise. These emotions are understood to be cross-culturally and universally communicated with particular facial expressions.

Reading this article, you can learn how to create Cognitive Service Emotion API in Universal Windows Apps development with Azure, XAML, and Visual C#. Here, input is a captured image and Emotion API returns different emotions like anger, contempt, disgust, fear, happiness, neutral, sadness, and surprise.

The following important tools are required for developing UWP.

1. Windows 10 (Recommended).
2. Visual Studio 2015 Community Edition (It is a free software available online).
3. Emotion API Key (How To Create Microsoft Cognitive Service Emotion API On Azure Portals).

Now, we can discuss step by step app development.

Step 1

Open Visual Studio 2015. Go to Start -> New Project-> select Universal (under Visual C# -> Windows) -> Blank App -> give your app a suitable name (UWPCoginitiveEmotion) -> OK.



After choosing the target and minimum platform versions that your app will support, the Project creates App.xaml and MainPage.xaml.



Add TextBlock control and change the name and text property for the title.



Step 2

Add Image control and set the name property for storing the captured image.



Add a Button control and set the name for taking a photo from stream to image control.



Add a Button control and set the name as Get Emotions.



Add the Click event method for both the button controls.



Add a Textblock control and set the name and text properties for Emotion result.



Step 3

Open (double click) the file MainPage.xaml in the Solution Explorer and add the following references in the project.

1. Microsoft.ProjectOxford.Face
2. Newtonsoft.Json

For adding Microsoft.ProjectOxford.Face Reference, right click your project (UWPCoginitiveEmotion) and select "Manage NuGet Packages".



Choose "Browse" and search Microsoft.ProjectOxford.Face. Select the package and install it.



Reference is added to your project.



For adding Newtonsoft.Json Reference, choose browse and search Newtonsoft.Json,  select the package and install it.



Reference added to your project.



Note

Automatically, the following code will be generated in XAML code view, while we are done in the design view.

<Page x:Class="UWPCoginitiveEmotion.MainPage" xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation" xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml" xmlns:local="using:UWPCoginitiveEmotion" xmlns:d="http://schemas.microsoft.com/expression/blend/2008" xmlns:mc="http://schemas.openxmlformats.org/markup-compatibility/2006" mc:Ignorable="d">
    <Grid Background="{ThemeResource ApplicationPageBackgroundThemeBrush}">
        <TextBlock x:Name="tblTitle" HorizontalAlignment="Left" Margin="255,98,0,0" TextWrapping="Wrap" Text="UWP Cognitive Service Emotion API Demo" VerticalAlignment="Top" FontSize="36" FontWeight="Bold" />
        <Image x:Name="imgEmotion" HorizontalAlignment="Left" Height="304" Margin="408,224,0,0" VerticalAlignment="Top" Width="310" />
        <Button x:Name="btnTakePhoto" Content="Take Photo" HorizontalAlignment="Left" Margin="172,548,0,0" VerticalAlignment="Top" FontSize="20" FontWeight="Bold" Click="btnTakePhoto_Click" />
        <Button x:Name="btnEmotions" Content="Get Emotions" HorizontalAlignment="Left" Margin="810,548,0,0" VerticalAlignment="Top" Height="39" Width="227" FontSize="20" FontWeight="Bold" Click="btnEmotions_Click" />
    </Grid>
</Page>

Step 4

Add the following namespaces in Mainpage.xaml.cs for capturing a photo and getting  emotions.

using Windows.Media.Capture;
using Windows.Storage;
using Windows.Graphics.Imaging;
using Windows.UI.Xaml.Media.Imaging;
using Windows.Storage.Streams;
using Microsoft.ProjectOxford.Emotion;
using Microsoft.ProjectOxford.Emotion.Contract;

Step 5

Add the EmotionServiceClient key by using Azure service and generate it (For more information, please refer to the article How To Create Microsoft Cognitive Service Emotion API On Azure Portal).

Add the following code with Azure generated key for EmotionServiceClient.

const string APIKey = "<Your Key>";
EmotionServiceClient emotionserviceclient = new EmotionServiceClient(APIKey);
Add the code
for Emotion Details from the captured image
CameraCaptureUI captureUI = new CameraCaptureUI();
StorageFile photo;
IRandomAccessStream imageStream;
const string APIKey = "<Your Key>";
EmotionServiceClient emotionserviceclient = new EmotionServiceClient(APIKey);
Emotion[] emotionresult;
public MainPage() {
    this.InitializeComponent();
    captureUI.PhotoSettings.Format = CameraCaptureUIPhotoFormat.Jpeg;
    captureUI.PhotoSettings.CroppedSizeInPixels = new Size(200, 200);
}
private async void btnTakePhoto_Click(object sender, RoutedEventArgs e) {
    try {
        photo = await captureUI.CaptureFileAsync(CameraCaptureUIMode.Photo);
        if (photo == null) {
            return;
        } else {
            imageStream = await photo.OpenAsync(FileAccessMode.Read);
            BitmapDecoder decoder = await BitmapDecoder.CreateAsync(imageStream);
            SoftwareBitmap softwarebitmap = await decoder.GetSoftwareBitmapAsync();
            SoftwareBitmap softwarebitmapBGRB = SoftwareBitmap.Convert(softwarebitmap, BitmapPixelFormat.Bgra8, BitmapAlphaMode.Premultiplied);
            SoftwareBitmapSource bitmapsource = new SoftwareBitmapSource();
            await bitmapsource.SetBitmapAsync(softwarebitmapBGRB);
            imgEmotion.Source = bitmapsource;
        }
    } catch {
        tblEmotion.Text = "error taking Photo";
    }
}
private async void btnEmotions_Click(object sender, RoutedEventArgs e) {
    try {
        emotionresult = await emotionserviceclient.RecognizeAsync(imageStream.AsStream());
        if (emotionresult != null) {
            Scores score = emotionresult[0].Scores;
            tblEmotion.Text = "Your Emotion are : \n" +
                "Happiness: " + score.Happiness + "\n" +
                "Sadness: " + score.Sadness + "\n" +
                "Surprise: " + score.Surprise + "\n" +
                "Neutral: " + score.Neutral + "\n" +
                "Anger: " + score.Anger + "\n" +
                "Contempt: " + score.Contempt + "\n" +
                "Disgust: " + score.Disgust + "\n" +
                "Fear: " + score.Fear + "\n";
        }
    } catch {
        tblEmotion.Text = "Error Returning the emotion";
    }
}

Step 6

Deploy your app on Local Machine. The output of the UWPCoginitiveEmotion app is shown below.

 

Clicking "Take Photo" button.

After clicking the "Take Photo" and "Apply" options, click the "Get Emotions"  button.


Another Emotion output is,


Summary

Now, you have successfully tested Cognitive Service Emotion API with Azure, XAML,  AND C# in UWP environment.