Overview Of Azure Data Lake

Introduction

In this article, we will walk through some important concepts of Azure Data Lake and its implementation.

• Implementation – Creating Azure Data Lake account.
• Implementation - Creating a SQL job and executing that job on Data Lake.

Concepts

• Azure data lake is an on-demand analytics job service to simplify big data analytics.
• It has the capability of dynamic scaling.
• You can run U-SQL query language in Azure Data Lake.
• It is affordable and cost effective.
• It provides a high level of performance, throughputs, and penalization for your big data workloads.

Implementation – Create Azure Data Lake account.

• Open Azure portal and click on + sign in the left top corner of your screen.
  
  
  

• Click on Data Lake Analytics and give name, resource group, location etc. details. Then, click on "Create" button.
  
  
  
  
• While creating this, you need to create “Data Lake Store” and click “OK” button.
• Data Lake Analytics is successfully created.
  
  
  

Implementation – Creating an SQL job and executing that job on data lake.

• Now click on "+ New job".
  
  
  

• Now, you need to write the U-SQL query to see the output. Let's imagine we want to see which employees are overpaid. So, we will select the record by putting some conditions into it.
  
  
  

• See the job details in form of graph.
  
  
  

• Click on the Output tab and click “conditionalOperatorA.csv” to see the file and its data.

Summary

This is how we can run the Azure Data Lake as well as how we can put many conditions to manipulate it into and convert into CSV format. If you have any doubt, you can inbox me and comment below.

This is just a simple example of using Azure Data Lake; you can run tons of conditions and your own U-SQL commands to get the output in different formats.