Extract text from Image and PDF Using Power Automate and AI Builder

Introduction

In today's data-driven world, we often need to extract valuable information from various sources, including images and PDFs. Power Automate and AI Builder offer a powerful solution for extracting text from these documents, enabling businesses to streamline processes and gain insights from previously unstructured data.

Power Automate and AI Builder

Power Automate is a cloud-based automation platform that allows users to create automated workflows, or flows, to connect various services and applications.

AI Builder, a component of Power Automate, provides pre-built AI models that can be integrated into flows to perform tasks such as text recognition, sentiment analysis, and object detection.

Setting Up Text Extraction Automation

Before diving into automation, ensure that all necessary components are in place. This includes access to Power Automate, AI Builder, and any additional tools or services required for the specific workflow.

Go to the Power Automation dashboard and click on “Instant Cloud flow”. And select “Manually Triggered flow”. Give the name of Flow whatever suits you. I was given the name “Extract text from images or PDF documents using AI Builder and Power Automate”.

Then click on add action to add a new action in flow, and search for “AI builder” in the “Add new action dialog box”. And click on see all on the right side to see all actions available in that component.

In the expanded list you can see all available actions in the AI connector. We need to find the action “Recognize text in an Image or in a PDF document”.

“Recognize text in an image or in a PDF document” contains a file input parameter. The rest of things will be done by this action itself. You do not need to worry about it.

We need to add a variable to appending text to a string so we can use it further in our flow. I added an action “Initialize a variable” action. I named it “Extract Text.”

We will add one more variable to add a new line in String. I added a variable “NewLine” variable.

The next step to add “Apply to each” control and pass the response from the action “Recognize text in an image or in a PDF document” ai build action.

Again, we will add “Apply to each” control to fetch each line from the AI response.

Add new action “Append text to string” under variable control. And then from the name dropdown pick the name “Extract Text” created before in this flow. And in value, input adds values ‘line’ coming from ai response as a new line and “NewLine” variable to add a new line in the string.

Add a new action to fetch profile information from the office database ”GetProfile V2”.

At the end of this flow, we will send an email with extracted data to see the output and send the uploaded file as an attachment. For this, we will add “Send Email V2” from Outlook Connector.

In Parameter, we are using.

To – Mail (From Get profile v2 action)

"emailMessage/To": "@outputs('Get_my_profile_(V2)')?['body/mail']",

Subject - “Text Recognition result”

"emailMessage/Subject": "Text Recognition results",

Body

<p>
    <span style="font-size: 16px;">Congratulations </span>
    @{outputs('Get_my_profile_(V2)')?['body/givenName']}
    <span style="font-size: 16px;"></span>
    <span style="font-size: 16px;">! Your Power Automate flow ran successfully. Here is all the text extracted from the attached file:</span>
    <span>\n\n</span>
    @{variables('Extracted text')}
    <span style="font-size: 16px;"></span>
    <span>\n\n</span>
    <span style="font-size: 16px;">Learn more about AI Builder Text Recognition and the information it can extract: <a href="https://go.microsoft.com/fwlink/?linkid=2161167" style="font-size: 16px;">https://go.microsoft.com/fwlink/?linkid=2161167</a></span>
</p>

Attachment

"emailMessage/Attachments": [
    {
        "Name": "@triggerBody()?['file']?['name']",
        "ContentBytes": "@triggerBody()?['file']?['contentBytes']"
    }
]

After adding all these values, make sure you save the flow.

Testing

Run flow using the flow detail page. It will ask for an image upload. Upload an image. Once flow runs successfully. You will get a mail with data and upload an image like this.

Email body

Congratulations Sandeep! Your Power Automate flow ran successfully. Here is all the text extracted from the attached file:

 

Home Marketing Sales v Operations v Productivity v Business Analysis v HR Admin v Master v Settings v Edit Public group & 22 members ... S T W STW365 + New v 6] Promote Translation Page details J Analytics Published 8/27/2023 Share v Swati Chavan is editing this page 6 Edit LEADS DASHBOARD LEADS DASHBOARD T W FY FY 2023-2024 V Industry Source City Sales Person Status All V All V All V All V All V Disqualified Open Qualified 7 123 13 STATUS RECORD SOURCE TYPE

 

Learn more about AI Builder Text Recognition and the information it can extract: https://go.microsoft.com/fwlink/?linkid=2161167.

According to the uploaded image, we will use the same data in the email we got from power automate flow.

Hope this will help you to extract data from ID cards, pdfs, or any scanned document to automate your process in your business using Power Automate and AI builder.

Happy learning and process automation.

Thanks!!