Azure Machine Learning - Create ML Workspace And Compute Cluster

In the previous articles, Azure Machine Learning Pipelines and Azure AI Fundamentals, we’ve learned holistically about Microsoft AI and its various functionalities as well as about the processes to create pipelines in Azure. This article explores the Azure ML Studio and gives a hands-on guideline to create Machine Learning Workspace in Azure and on Creating Compute Cluster for machine learning projects.  

Microsoft AI 

Microsoft AI is a powerful framework that enables organizations, researchers, and non-profits to use AI technologies with its powerful framework which offers services and features across domains of Machine Learning, Robotics, Data Science, IoT, and many more.  Learn more about Microsoft AI from this article.  

Azure Machine Learning  

The Azure Machine Learning enriches and consolidates the functionalities to support model training and deployment which transitions from Machine Learning Studio. It provides tools for Machine Learning works for all skill levels, provides an open and interoperable framework with support to different languages, and enables robust end-to-end MLOps. It also supports Automated Machine Learning. Read this article Auto ML to learn more about it.  

So, where and how do we start if we want to create and deploy a Machine Learning project? Azure Machine Learning provides all the tools through its portal to create the resources and set up the infrastructure that is needed for any kind of machine learning works.  

First, we’ll create a Machine Learning Workspace and then setup the Compute Instance or Cluster – the computation infrastructure that will be required for our project. 

Creating Machine Learning Workspace 

Step 1  

Open the Azure Portal. The Welcome page will look something like this.  

Step 2 

Click on Create a Resource 

Step 3 

On the Search Tab, type Machine Learning  

Step 4 

The options of different resources available for Machine Learning is shown into the display to choose from 

Step 5 

We choose the Azure Service – Machine Learning  

Step 6 

Click on Create 

This Machine Learning will help data scientists and developers to build, train and deploy machine learning models which can then be used for different services such as the web.  

Step 7 

The details for the project are required to be filled in.  

Step 8 

Create a new Resource Group, name the Workspace. The rest would be filled automatically as per the resource group you use. You need not change any of those factors as of now.  

Click on Review + Create.

Step 9 

The validation is done and once passed, the green tick notifies it. We are now ready to create the Machine Learning Workspace.  

Click on Create 

Step 10  

Now, we can see, our Machine Learning workspace has now been created.  

Select the Workspace.  

Now, Click on Launch Studio.  

To learn more about, Azure Machine Learning and its ability to support machine learning projects, watch this video by Pragati Jain.  

Create a Compute Instance and Cluster 

Step 11 

We are welcomed to Microsoft Azure Machine Learning Studio 

Step 12 

Click on Compute on the left sidebar.  

Step 13 

Click on New  

Step 14 

Fill in the Compute Name and select the Virtual machine type we require that ie. CPU or GPU. The GPU is used for intensive tasks 

Now, when we choose, from the Recommended options, we are shown a few. Let us choose the Memory Optimized Workload which is highly optimized for Machine Learning for Notebooks and is sufficient for light works. Choosing this will save us cost in the long run as the Cost in Azure is charged upon usage per hour for this compute instance.  

Click on Create 

We can see, the Compute Instance is on State – Creating.  

Compute Cluster 

It is vital to understand the difference between Compute Instance and Compute Cluster. Compute Instance can be understood as this virtual machine that has all the essentials required for machine learning and data science projects. GPUs and CPUs can be leveraged to perform processing in Azure ML Notebooks using all the frameworks and libraries necessary. Compute Cluster is a bit different from the Compute Instance. The Compute Cluster provides the capability to set one or more nodes that aren’t available in compute instances. With more than one node enables the possibility of parallel processing as it supports hyperparameter tuning, multiple machine learning runs at once, and complex GPU-based computation. Automated Machine Learning requires the Compute Cluster too. Besides, if time is not a constraint, using the low priority nodes will take some time to process the experiment but will perform the required tasks in more economic value saving the dollars compared to using the dedicated VMs.  

Step 15 

Now, let us create a Compute Cluster. 

Click on Compute Clusters and the New option under it.  

Step 16 

Fill in the details and choose the workload resource as per the demand of your project. The location should be the same as the workspace.  

Click on Next 

Step 17 

Now, depending upon your requirement, select the nodes and idle seconds for scale down.  

Step 18 

Once, done. Click on Create.  

Step 19 

We can see, we have successfully created the compute clusters.  

Step 20 

In the upcoming articles, we’ll learn how to create machine learning projects following up from this article. The foundation for any machine learning projects in Azure – the ML workspace creation and compute resources has now been setup. We’ll continue from Pipeline creation with a specific machine learning goal in the next article.  

Conclusion

In this article, we learned about Azure Machine Learning and then dived into the hands-on tutorial to setup the machine learning workspace and compute resources for the machine learning project. We also learned the difference between the compute instance and cluster and where it can be used. In the next article, we’ll explore more on the different machine learning projects we can build on top of these resources available in Microsoft Azure and supported by the Azure Machine Learning.