In this article, we’ll learn to deploy our machine learning model using low code functionality offered via Designer in Azure Machine Learning. We’ll go through step-by-step process to deploy our machine learning model for numerous services and application using the Inference Pipeline available in Azure Machine Learning. We’ll create a real-time inference pipeline and thus through the endpoint our ML model can then be used to predict from the input data. The external application will take input and send the data to our ML model through the endpoint and once our ML model processes and provided the predicted output, it is sent back and the output can be displayed or used in any application. This article is a part of the Azure Machine Learning Series.
- Azure Machine Learning - Create Workspace for Machine Learning
- Azure Machine Learning – Create Compute Instance and Compute Cluster
- Azure Machine Learning - Writing Python Script in Notebook
- Azure Machine Learning - Model Training
- Azure Machine Learning - Linear Regression
- Azure Machine Learning – Model Deployment
Pre-requisite
This article is a follow up to the previous articles, Azure Machine Learning - Create Workspace for Machine Learning, Azure Machine Learning – Create Compute Instance and Compute Cluster, and Azure Machine Learning - Linear Regression in order.
Let us get into the step-by-step process using designer to deploy our Machine Learning model in Azure Machine Learning.
Step 1
Once you have run the Linear Regression Model, the Canvas must look similar to this below. All of the components would be green with the Completed note.
![]()
Creating Inference Pipeline
Step 2
In order to deploy our machine learning pipeline, it is essential to convert the training pipeline into real-time inference pipeline.
Let us now create a real-time inference pipeline. First, Click on Create inference pipeline and then Real-time inference pipeline.
![]()
Step 3
We can see, the training components have been removed and web service input and output components have been added.
![]()
Click on Submit.
![]()
We need to select our experiment under our compute target. Here, mine is the learn-linear-regression. I’ve also added the job description which will be useful when we look at it later.
![]()
Select the Continue on failure step and click on Submit.
Step 4
We can see the run process.
![]()
Once the run process is complete, we can see the competition note.
![]()
![]()
We can see the difference between the Training Pipeline and Real-time inference pipeline by switching the canvas from the Menu.
![]()
![]()
Deployment
Step 5
Now, above the canvas of Real-time inference pipeline click on Deploy.
![]()
We can see the details to be filled. Under the Compute Name, we cannot select any resources. This is because the Compute Instances isn’t usable for deployment. We would need a inference cluster.
Let us go to the Compute Page. Here, click on Inference Clusters.
![]()
Click on + New Button.
Step 6
Here, under Virtual Machine for Kubernetes Service, Click on Create New and select the Location.
![]()
As I select the Central US, we can see the different options.
![]()
Let us go for Standard_A2_v2 as 2 cores, 4GB of RAM and 20GB of storage would be enough for us.
Step 7
Now, click on Next or Advanced Settings.
![]()
Name your compute. Here I’ve named my ojash-akscompute. Set the Cluster purpose to production – this will set it for production environment which we can use for real world usage. Number of nodes can be set to 3. Set the Network Configuration to Basic and then click on Create.
![]()
We can see the state of the cluster. As the process begins, we can see it is in Creating State.
![]()
As it is completed, the State will be shown as Succeeded.
![]()
We can see the details by clicking on it.
![]()
Step 8
Now, as we have created our inference cluster, lets proceed with the Deploy button on the Real-time inference pipeline.
![]()
Name your real-time endpoint. Here I’ve named mine ojashlrmodel-deploy.
![]()
The compute type would be Azure Kubernetes Service and select the Compute you’ve created which is the Kubernetes Service Inference cluster.
![]()
Make any changes necessary by checking the Advanced settings for Scoring time, Replace Numbers and Target Utilization as per your need.
Finally, click on Deploy.
Step 9
The deploying process begins and we are notified about it.
![]()
As the process is executing, we can see the Deployment state as Transitioning.
![]()
Once, it is completed, we can obtain the REST Endpoint and Swagger URL.
![]()
Step 10
Next, we’ll have four tabs including Details, Test, Consume and Deployment Logs.
![]()
The details page contains the details of the deployment service.
![]()
The consume page consists of the REST Endpoint, the Authentication keys – both Primary and Secondary. Further, we also have the consumption codes as per the platform for C#, Python and R.
![]()
The deployment logs consists of all the logs of the deployment and usage from external application of the ML Model and services.
Step 11
The Test page consists the input data to be tested for the real-time endpoint.
![]()
As we click on the Test Button, we can obtain the Test Result.
Input data to test real-time endpoint
{
"Inputs": {
"WebServiceInput0": [
{
"symboling": 3,
"normalized-losses": 1.0,
"make": "alfa-romero",
"fuel-type": "gas",
"aspiration": "std",
"num-of-doors": "two",
"body-style": "convertible",
"drive-wheels": "rwd",
"engine-location": "front",
"wheel-base": 88.6,
"length": 168.8,
"width": 64.1,
"height": 48.8,
"curb-weight": 2548,
"engine-type": "dohc",
"num-of-cylinders": "four",
"engine-size": 130,
"fuel-system": "mpfi",
"bore": 3.47,
"stroke": 2.68,
"compression-ratio": 1.0,
"horsepower": 1.0,
"peak-rpm": 1.0,
"city-mpg": 21,
"highway-mpg": 27,
"price": 1.0
},
{
"symboling": 3,
"normalized-losses": 1.0,
"make": "alfa-romero",
"fuel-type": "gas",
"aspiration": "std",
"num-of-doors": "two",
"body-style": "convertible",
"drive-wheels": "rwd",
"engine-location": "front",
"wheel-base": 88.6,
"length": 168.8,
"width": 64.1,
"height": 48.8,
"curb-weight": 2548,
"engine-type": "dohc",
"num-of-cylinders": "four",
"engine-size": 130,
"fuel-system": "mpfi",
"bore": 3.47,
"stroke": 2.68,
"compression-ratio": 1.0,
"horsepower": 1.0,
"peak-rpm": 1.0,
"city-mpg": 21,
"highway-mpg": 27,
"price": 1.0
},
{
"symboling": 1,
"normalized-losses": 1.0,
"make": "alfa-romero",
"fuel-type": "gas",
"aspiration": "std",
"num-of-doors": "two",
"body-style": "hatchback",
"drive-wheels": "rwd",
"engine-location": "front",
"wheel-base": 94.5,
"length": 171.2,
"width": 65.5,
"height": 52.4,
"curb-weight": 2823,
"engine-type": "ohcv",
"num-of-cylinders": "six",
"engine-size": 152,
"fuel-system": "mpfi",
"bore": 2.68,
"stroke": 3.47,
"compression-ratio": 1.0,
"horsepower": 1.0,
"peak-rpm": 1.0,
"city-mpg": 19,
"highway-mpg": 26,
"price": 1.0
},
{
"symboling": 2,
"normalized-losses": 1.0,
"make": "audi",
"fuel-type": "gas",
"aspiration": "std",
"num-of-doors": "four",
"body-style": "sedan",
"drive-wheels": "fwd",
"engine-location": "front",
"wheel-base": 99.8,
"length": 176.6,
"width": 66.2,
"height": 54.3,
"curb-weight": 2337,
"engine-type": "ohc",
"num-of-cylinders": "four",
"engine-size": 109,
"fuel-system": "mpfi",
"bore": 3.19,
"stroke": 3.4,
"compression-ratio": 1.0,
"horsepower": 1.0,
"peak-rpm": 1.0,
"city-mpg": 24,
"highway-mpg": 30,
"price": 1.0
},
{
"symboling": 2,
"normalized-losses": 1.0,
"make": "audi",
"fuel-type": "gas",
"aspiration": "std",
"num-of-doors": "four",
"body-style": "sedan",
"drive-wheels": "4wd",
"engine-location": "front",
"wheel-base": 99.4,
"length": 176.6,
"width": 66.4,
"height": 54.3,
"curb-weight": 2824,
"engine-type": "ohc",
"num-of-cylinders": "five",
"engine-size": 136,
"fuel-system": "mpfi",
"bore": 3.19,
"stroke": 3.4,
"compression-ratio": 1.0,
"horsepower": 1.0,
"peak-rpm": 1.0,
"city-mpg": 18,
"highway-mpg": 22,
"price": 1.0
}
]
},
"GlobalParameters": {}
}
Here, we can obtain the Score Label for a new set of input data.
![]()
Test Result
{
"Results": {
"WebServiceOutput0": [
{
"symboling": 3,
"make": "alfa-romero",
"fuel-type": "gas",
"aspiration": "std",
"num-of-doors": "two",
"body-style": "convertible",
"drive-wheels": "rwd",
"engine-location": "front",
"wheel-base": 88.6,
"length": 168.8,
"width": 64.1,
"height": 48.8,
"curb-weight": 2548,
"engine-type": "dohc",
"num-of-cylinders": "four",
"engine-size": 130,
"fuel-system": "mpfi",
"bore": 3.47,
"stroke": 2.68,
"compression-ratio": 1,
"horsepower": 1,
"peak-rpm": 1,
"city-mpg": 21,
"highway-mpg": 27,
"price": 1,
"Scored Labels": 2514.163465984617
},
{
"symboling": 3,
"make": "alfa-romero",
"fuel-type": "gas",
"aspiration": "std",
"num-of-doors": "two",
"body-style": "convertible",
"drive-wheels": "rwd",
"engine-location": "front",
"wheel-base": 88.6,
"length": 168.8,
"width": 64.1,
"height": 48.8,
"curb-weight": 2548,
"engine-type": "dohc",
"num-of-cylinders": "four",
"engine-size": 130,
"fuel-system": "mpfi",
"bore": 3.47,
"stroke": 2.68,
"compression-ratio": 1,
"horsepower": 1,
"peak-rpm": 1,
"city-mpg": 21,
"highway-mpg": 27,
"price": 1,
"Scored Labels": 2514.163465984617
},
{
"symboling": 1,
"make": "alfa-romero",
"fuel-type": "gas",
"aspiration": "std",
"num-of-doors": "two",
"body-style": "hatchback",
"drive-wheels": "rwd",
"engine-location": "front",
"wheel-base": 94.5,
"length": 171.2,
"width": 65.5,
"height": 52.4,
"curb-weight": 2823,
"engine-type": "ohcv",
"num-of-cylinders": "six",
"engine-size": 152,
"fuel-system": "mpfi",
"bore": 2.68,
"stroke": 3.47,
"compression-ratio": 1,
"horsepower": 1,
"peak-rpm": 1,
"city-mpg": 19,
"highway-mpg": 26,
"price": 1,
"Scored Labels": 4169.688613458313
},
{
"symboling": 2,
"make": "audi",
"fuel-type": "gas",
"aspiration": "std",
"num-of-doors": "four",
"body-style": "sedan",
"drive-wheels": "fwd",
"engine-location": "front",
"wheel-base": 99.8,
"length": 176.6,
"width": 66.2,
"height": 54.3,
"curb-weight": 2337,
"engine-type": "ohc",
"num-of-cylinders": "four",
"engine-size": 109,
"fuel-system": "mpfi",
"bore": 3.19,
"stroke": 3.4,
"compression-ratio": 1,
"horsepower": 1,
"peak-rpm": 1,
"city-mpg": 24,
"highway-mpg": 30,
"price": 1,
"Scored Labels": 1223.6365517221493
},
{
"symboling": 2,
"make": "audi",
"fuel-type": "gas",
"aspiration": "std",
"num-of-doors": "four",
"body-style": "sedan",
"drive-wheels": "4wd",
"engine-location": "front",
"wheel-base": 99.4,
"length": 176.6,
"width": 66.4,
"height": 54.3,
"curb-weight": 2824,
"engine-type": "ohc",
"num-of-cylinders": "five",
"engine-size": 136,
"fuel-system": "mpfi",
"bore": 3.19,
"stroke": 3.4,
"compression-ratio": 1,
"horsepower": 1,
"peak-rpm": 1,
"city-mpg": 18,
"highway-mpg": 22,
"price": 1,
"Scored Labels": 2959.4884974132037
}
]
}
}
This shows, how our machine learning model is capable to obtain input data and process to provide with Prediction based on the input through the end-point from any web application. We’ve now successfully deployed our Machine Learning Model using Inference Pipeline.
Deleting Resources
Step 12
In order to save ourselves from unnecessary charges to incur, it is a good practice to delete the entire resource groups once our tasks in completed or aren’t of any extended usage. Instead of deleting resources one by one which is more time consuming, let us visit the Azure portal and Delete the entire resource group itself.
![]()
Click on Delete Resource Group and retype the resource group name and finally hit the Delete button.
Conclusion
Thus, in this article, we went through a step-by-step process to deploy a machine learning model ready for usage in any other platform or web application through real-time endpoint we created through the inference pipeline and deploy through the Kubernetes Service.