AI Builder is an upcoming feature in the The Power platform that allows non-developers to integrate AI capabilities in their Power Apps. It currently offers 4 key features

• Binary Classification: binary classification uses historical data to predict whether new data falls into one of two categories.
• Form Processing: form processing identifies the structure of your documents based on examples you provide to extract text from any matching form.
• Object Detection: object detection lets you count, locate, and identify selected objects within any image. You can use this model in PowerApps to extract information from pictures you take with the camera.
• Text classification: text classification allows tagging of text entries with tags or labels that can be used for sentiment analysis, spam detection, and routing customer requests.

In this blog post we will have a look on how Form Processing can be used to automatically retrieve key parts of invoices you might receive monthly. To start working with the AI Builder you can go to https://make.powerapps.com and select AI Builder (preview) -> Build.

Since we want to build an application that recognises the structure of documents, select Form Processing

Let's name the model LunchInvoices

Once this model is created, the first step if to upload at least 5 sample documents that can be used by the AI Builder form processor to analyse the document structure.

Once the structure of the uploaded documents is analysed, the fields and/or tables in the documents we want to retrieve later must be selected and be given a name.

In the image below you can see the output of the Form Processor. The dotted lines represent fields or tables that the Form Processor did recognise. The fields/tables you later want to access in an Power App must be selected and can be given a more descriptive name. In the example below 3 fields (shown in green) where selected and a name for them was provided: TotalInvoiceAmount, IBAN, and PaymentCode. Also the upper table with one row was selected. This table contains fields like the due date and customer number. Currently it is not yet possible to provide a descriptive name for a table and the fields contained in it.

Once this is done, we can train the model so that it is able to recognise similar documents.

Now that the model is finished, we can publish it so that it can be used by PowerApps. Let's create a simple Canvas App that uses our trained form processing model. To be able to use the model that was created in the previous steps, a Form Processor control must be inserted into the App. The Form Processor control has an AI model property which must be set to the LunchInvoices model that was created in the previous steps.

Next, 2 label controls can be added to the form that will be used to display the total invoice amount and the due date of the invoice. The Text expression for the first label is set to

FormProcessor1.FormContent.Fields.TotalInvoiceAmount

The second label should display the due date of the invoice. This field value is comming from the table that was called table_0. This table has a column named Vervaldatum which stands for invoice date in Dutch. Since a table can have many rows the First function is used to retrieve the first row from the table. The text property of the second label can be set to the following expression:

First(FormProcessor1.FormContent.Tables.table_0).Vervaldatum

The Form Processing control turns blue if it has more then 80% of confidence that the document structure could be parsed successfully.