Post
Ask Question

Building Video Indexer Mobile Application Using Cognitive Service

Introduction

 
Microsoft introduced the public preview of Video Indexer as a part of Cognitive Service. Previously, we used Video API but now it's replaced with Video Indexer. Video Indexer automatically extracts the metadata and builds intelligent innovative AI applications based on Video and Audio.
 
Cognitive Service
 
In this article, I will show how to integrate Video Indexer embeddable widgets into the mobile application.
 
Create Video Indexer Account
 
For implementing Video Indexer into the application, you must create an account and upload your video using my previous article as a reference.
 
Video Indexer supports embedding two types of widgets into your Mobile application
  1. Cognitive Insights – it includes all visual insights that were extracted from the video indexer.
  2. Player -it enables you to stream the video.
Embedding Video
 
 I will show the below steps for how to get the embedding source from video indexer portal
 
Step 1
 
Login to Video Indexer Portal using Microsoft, Google or Azure Active Directory account
 
Step 2
 
Select your video, whichever one needs to integrate into the application.
 
Step 3
 
Click the "embed" button that appears below the video.
 
Cognitive Service
 
Step 4
 
After clicking on “embed” button, select the widget you want to embed with the desired options. (player/insights)
 
Cognitive Service
 
Step 5
 
 You can copy the embed codes from embed popups and start implementing them into your application for Public videos. If you want to embed a Private video, you have to pass an access token in the iframe src attribute as a query string.
 
Create a Xamarin / Web Application
 
Let's start by creating a new Xamarin Forms Project using Visual Studio. Open Run - Type “Devenev.Exe” and enter - New Project (Ctrl+Shift+N) - select Blank Xaml App (Xamarin.Forms) template.
 
Cognitive Service
 
It will automatically create multiple projects, like .NET Standard, Android, iOS, and UWP.
 
Implement Cognitive Insights Widget
 
Cognitive insights widget contains all the visual insights that were extracted from the video after the indexing process such as people's appearances, top keywords, sentiment analysis, transcript, translate, and search.
 
Let's start implementing it into the Xamarin application.
 
You can add webview control inside the content page as per the below XAML design
  1. <?xml version="1.0" encoding="utf-8" ?>  
  2. <ContentPage xmlns="https://xamarin.com/schemas/2014/forms"  
  3.              xmlns:x="https://schemas.microsoft.com/winfx/2009/xaml"  
  4.              xmlns:local="clr-namespace:VideoIndexer"  
  5.              x:Class="VideoIndexer.MainPage">  
  6.     <WebView x:Name="browser"  WidthRequest="1280" HeightRequest="780"></WebView>  
  7.   
  8. </ContentPage>  
You can use your cognitive insights widget code from c# code as a htmlwebviewsource . You can specify the query string for whatever widgets you need to display ( <url>?widgets=people&title=myInsights)
  1. public MainPage()  
  2. {  
  3.             InitializeComponent();  
  4.            // var browser = new WebView();  
  5.             var htmlSource = new HtmlWebViewSource();  
  6.             htmlSource.Html = @"<html><head><script src='https://breakdown.blob.core.windows.net/public/vb.widgets.mediator.js'></script></head>  
  7. <body><iframe width='580' height='780' src='https://www.videoindexer.ai/embed/insights/9123e16b12' frameborder='0' allowfullscreen></iframe>  
  8. </body></html> ";  
  9.             browser.Source = htmlSource;  
  10. }  
You can deploy the application, the output looks like below
 
Cognitive Service
 
Implement Video Insights Widget
 
The video player widget is a customized Media Player that provides video streaming, and contains extra features such as playback speed and closed captions. Refer to the below code for video insight wedge implementation
  1. public MainPage()  
  2. {  
  3.             InitializeComponent();  
  4.            // var browser = new WebView();  
  5.             var htmlSource = new HtmlWebViewSource();  
  6.             htmlSource.Html = @"<html><head><script src='https://breakdown.blob.core.windows.net/public/vb.widgets.mediator.js'></script></head>  
  7. <body><iframe width='1280' height='720' src='https://www.videoindexer.ai/embed/player/9123e16b12' frameborder='0' allowfullscreen></iframe>  
  8. </body></html> ";  
  9.             browser.Source = htmlSource;  
  10. }  
Implement Video and Cognitive Insights Widget
 
Copy the player and cognitive insights embed code and also include javascript communication file for communication with the other widgets
  1. var htmlSource = new HtmlWebViewSource();  
  2. htmlSource.Html = @"<html><head><script src='https://breakdown.blob.core.windows.net/public/vb.widgets.mediator.js'></script></head>  
  3. <body><iframe width='1280' height='720' src='https://www.videoindexer.ai/embed/player/9123e16b12' frameborder='0' allowfullscreen></iframe>  
  4. <iframe width = '1280' height = '780' src = 'https://www.videoindexer.ai/embed/insights/9123e16b12/?widgets=people&title=myInsights' frameborder = '0' allowfullscreen ></iframe >   
  5. </body></html> ";  
  6. browser.Source = htmlSource;  

Summary

 
In this article, we learned how to sign in to Video Indexer and integrate Video Indexer embeddable widgets into the mobile cross-platform application. If you have any questions/feedback/ issues, please write in the comments box.


Similar Articles