Object Recognition using the Microsoft PowerApps AI Builder

Harshani Perera, 19 September 2019

What comes to your mind when you hear Artificial Intelligence?

A whole bunch of Hollywood films? Something for coders to figure out? Something too expensive to implement? Too complicated or just out of reach?

This is no longer the case with Microsoft’s new AI Builder. The AI Builder easily allows us to put intelligence into your application in as little as 5 mins without any coding!

In this blog, I will show you how easy it is to create an AI Model to identify bananas in different images.

The following steps will show you how to do this.

Set Up the Model

Step 1: Login to PowerApps https://make.powerapps.com/ with your account.
(If you don’t have a Microsoft account, you can create a Microsoft Dynamics 365 Trial version by going to https://trials.dynamics.com/)

Step 2: Once you are in PowerApps, on the left pane you will see the AI Builder (preview). Currently this is in Public Review and available for North America and Europe environments only.

 
Step 3: Go to AI Builder (preview) and select Build. The following page will be displayed. There are four AI Models to choose from: Binary Classification, Form Processing, Object Detection and Text Classification. In this blog, we will be looking at the Object Detection model.
In simple terms, the Object Detection model identifies objects in images.

 
Step 4: Select Object Detection > Enter a Model name and click Create.

Set Up Data

Step 5: Click Select object names to select the entity.
If you don’t see the entity, you can create a new entity by going to Data > Entities > Click New entity > Enter a Display name > Click Create

 
Click Edit Data in Excel to add Data to your entity.

 
Sign into Excel.

Enter the names of your data and click Publish.

 
Go back to PowerApps and select the Data tab and now you will see the data you entered on the excel.

 
You can also go to Data tab and click Add record to add data as well.
Now that you have the entity and data, go back and click Select object names. Search and select the Entity > Select the field Name > Click Select field.

Upload Training Data

Step 6: Select the object names you want to detect and click Next.

 
Step 7: Now you will need to add images to train your model to detect the object. You will need at least 15 images and adding more than 50 will give better results. Remember that you can have many objects in one image.
Click Add Images to upload the image from your computer and click Upload

 
 
Click Close > Click Next.

Train the Model

Step 8: Now you will need to tag the objects in the images.
For each image, draw a rectangle around the objects you want to identify and tag with the correct name.

 
Now you will see the tag around the object

Go through all the images to tag the objects. In my example, I will be going through all the images and tag all the bananas. You can tag more than one object in one image.

 
Step 9: Once all the images have been tagged, click Done tagging.

 
Click Next > Click Train

You will see the following image once the model is in training. It will take several minutes.

Test the Model

Step 10: Once the training is complete, you can do a Quick Test to verify it is working. Click the Quick test button and upload an image that you have not used for training.

Results: The AI has detected the set of bananas within the test image.

Here is another test. It has detected the bananas in the fruit basket.

Publish the AI Model

Step 11: Once you are happy with the model and it is performing well, you can publish it by clicking the Publish button.

You can now create a Power App using this AI Model by clicking the Create app Button. 

My next blog will show you how to create a simple app to use this AI Model.

AI is no longer just for the experts, it is accessible through Microsoft’s AI builder that has made it easily accessible for everyday users and business users to implement into their business.