Generate A Description Of An Image Using Cognitive Service And Microsoft Flow

Introduction

In this article, you will see how to generate a description of an image using Cognitive Service and Microsoft Flow. A button will be used to trigger the flow manually and it will take the input as image file. With the help of Computer Vision API (it uses the "Describe Image" action) connector, the description of an image will be generated and will be shown as a mobile notification.

Flow Design

 

Get Subscription Key

1. Navigate to Azure Portal here.
2. In the left navigation, click "Create a resource".
3. In Azure marketplace, select "AI + Machine Learning" and then, click Computer Vision.
   
   

4. Enter the required details, as shown below and click "Create".
   
   

5. It might take a few minutes to deploy the resource. Click "All Resources" in the left navigation and then click the newly created computer vision resource.
6. Click Keys under Resource Management, and copy the Key value.
   
   

7. Click "Overview" and copy the endpoint.
   
   

Create Flow

1. Navigate to the flow site here.
2. In the top navigation, click "My flows".
3. Click "Create from blank".
4. Select "Manually trigger a flow" >> Add an input. Then, select the user input as File as shown below.
   
   

5. Click "Next Step" and search for Computer Vision as shown below. Select the "Describe Image" option.
   
   

6. Enter the following details and click "Create".
   
   

• Connection Name – a user-friendly name for the connection
• Account Key – Copied from the above Get Subscription Key section
• Site URL – Copied from the above Get Subscription Key

7. Select Image Content for Image Source drop-down. Select the dynamic content as File Content for Image Content.
   
   

8. Click "Next Step" and search for notification as shown below. Select "Send me a mobile notification".  
   
   

9. Select the dynamic content "Captions Caption Text" for the text field, as shown below.
   
   

10. Enter the title for the flow and click on "Save".

Test the Flow

1. Open Flow mobile app.
2. Click on Buttons. Click "Describe Image".
   
   

3. Capture the image using a mobile camera or upload the image from the gallery. Once the image is uploaded, click "Done".
   
   

4. After a few seconds, a notification will be received that provides the description of an image that was uploaded.
   
   

Summary

Thus, in this article, you saw how to generate a description of an image using Cognitive Service and Microsoft Flow.