Road To AZ 204 - Developing Solutions That Use Blob Storage

Introduction

This article's intention is to explain the main skills measured in this sub-topic of the AZ-204 Certification. Azure Storage and Blob Storage are the main components that will have their fundamentals explained here, alongside a practical example of their usage.

This certification is very extensive and this article approaches only the main topics, make sure you know deep down those components before taking the exam. Another great tip is to do exam simulators before the official exam in order to validate your knowledge.

What is the Certification AZ-204 - Developing Solutions for Microsoft Azure?

The AZ-204 - Developing Solutions for Microsoft Azure certification measures designing, building, testing, and maintaining skills of an application and/or service in the Microsoft Azure Cloud environment. It approaches, among others, the components.

• Azure Virtual Machines
• Docker
• Azure Containers
• Service Web App
• Azure Functions
• Cosmos DB
• Azure Storage
• Azure AD
• Azure Key Vault
• Azure Managed Identities
• Azure Redis Cache
• Azure Logic App
• Azure Event Grid
• Azure Event Hub
• Azure Notification Hub
• Azure Service Bus
• Azure Queue Storage

Check more information on the AZ - 204 Developing Solutions for Microsoft Azure Official Website.

Target Audience

Any IT professional willing to improve his knowledge in Microsoft Azure is encouraged to take this certification. It is a great way to measure your skills within trending technologies. But, some groups of professionals are keener to take maximum advantage of it.

• Azure Developers, with at least 1 year of experience with Microsoft Azure
• Experienced Software Developers, looking for an Architect position in a hybrid environment
• Software Developers, working to move applications to the cloud environment

Skills Measured

According to today's date, the skills that are measured in the exam are split as follows.

• Develop Azure compute solutions (25-30%)
  • Implement IaaS solutions;
  • Create Azure App Service Web Apps
  • Implement Azure functions
• Develop for Azure storage (10-15%)
  • Develop solutions that use Cosmos DB storage
  • Develop solutions that use blob storage
• Implement Azure security (15-20%)
  • Implement user authentication and authorization
  • Implement secure cloud solutions
• Monitor, troubleshoot, and optimize Azure solutions (10-15%)
  • Integrate caching and content delivery within solutions
  • Instrument solutions to support monitoring and logging
• Connect to and consume Azure services and third-party services (25- 30%)
  • Develop an App Service Logic App
  • Implement API Management
  • Develop event-based solutions
  • Develop message-based solutions

Updated skills can be found on the AZ - 204 Official Measured Skills Website.

Benefits of Getting Certified

The main benefit here is having a worldwide recognized certification that proves that you have knowledge of this topic. Among intrinsic and extrinsic benefits, we have.

• Higher growth potential, as certifications are a big plus.
• Discounts and deals in Microsoft products and partners, like PluralSight and UpWork.
• MCP Newsletters, with trending technologies.
• Higher exposure on LinkedIn, as recruiters usually search for specific certifications.
• Higher salary, you will be more valuable to your company.
• Unique happiness when getting the result and you were approved, knowing that all your efforts were worth it.

Official Microsoft Certification Program Benefits Website.

Main Skills Measured by this Topic
 

What is a Blob Storage?

Blob Storage is part of Azure Storage where we can only store Blobs. It is optimized for storing massive amounts of unstructured data. Blobs are stored inside blob containers and those blob containers are inside an Azure Storage Account.

Azure charges a Blob Storage for storing and accessing its data. It is great for.

• Sharing unstructured files, as far as each file has its own URI and security permissions.
• Streaming audio or video, Azure Blob Storage has excellent performance and availability.
• Storing highly accessible files, using its hot storage.
• Storing big and not so frequently used files, using its cool or archive storage. Backup files and logs, for example.

Blob Storage Access Tiers

In order to store and access your data in the most cost-effective manner, Azure offers Blob Storages in three different tiers, whereas their cost calculations differ from the usage storage and access frequency, as follows.

• Hot tier, ideal for highly frequently accessed small files. Highest storage cost and lowest accessibility cost.
• Cool tier, ideal for large files that are not so frequently accessed. Medium storage cost and medium accessibility cost.
• Archive tier, ideal for large files that are rarely accessed. Lowest storage cost and highest accessibility cost.

Blob Storage Lifecycle Management

Azure Blob Storage has a very important feature that allows you to move your items between access tiers according to their usage. Azure Blob Storage lifecycle management policies make the transition of an item between tiers automatically in order to optimize costs and performance, those policies can be configured to run once a day and at the level of containers and a subset of blobs. Those policies allow your Blob Storage to.

• Transition from cold to hot, for better performance
• Transition to cooler tiers in case of not being used
• Delete items if not accessed or modified, according to the retention period

System Properties and Metadata

Blob Storage allows us to set and retrieve information from the Container level or the Blob level as follows.

• System properties allow you to read the Blob Container Properties. Those properties can be updated by calling their own methods, not by updating their properties directly.
• Metadata allows you to write and read key-value attribute pairs assigned to each Blob or Blob Container.

Blob Storage SDKs

Azure has a wide range of SDKs in order to manage and manipulate your Blobs, as follows.

• Azure Portal
• PowerShell
• Azure CLI
• Azure Storage Explorer
• .Net
• Java
• Python
• Javascript
• C++
• Go
• PHP
• Ruby
• Xamarin

Practical Samples

In this study of the case, we are going to store images with Azure Blob Storage. Images that are going to be categorized as more frequently used, others not so often used and even others rarely used.

Creating a Blob container in an Azure storage account
 

Using Azure CLI

Setting variables

$resourceGroup = "your resource group here"
$storageAccountName = "samplestorageaccountblob"
$location = "West Europe"
$containerName = "sampleblobcontainer"
$subscription = "your subscription id here"

Creating the storage account.

Creating the Blob Container.

az storage account create --name $storageAccountName --resource-group $resourceGroup --location $location
az storage container create --account-name $storageAccountName --name $containerName --auth-mode login

Validating through Azure Portal.

Using Azure Portal

From your Azure Portal, search for the Storage Account resource and click on Add. Fill out the Basics, Networking, Data Protection, and Advanced forms then Create the Azure Storage Account.

After successfully deploying, go to your resource. Inside Blob Service, go to Containers and click on "+ Container", set your container name, and its access level, and click on Create

Success

Manipulating Blobs
 

Using .NET

Prerequisites

• Nuget Azure.Storage.Blobs.
• Azure Storage Account previously created.

First, retrieve your connection string. From your Storage Account, go to Settings and then Access Keys.

Setting Local Variables.

private static readonly string connectionString = "DefaultEndpointsProtocol=https;AccountName=samplestorageaccountblob;AccountKey=FyJw8U6tORM7DVBxw4G5nxUlbTy8l5so8lAGfsr1AEf6fSN63A6llM8kQfQO9bF1qABHkNHfCWtL/l/j9zk/WQ==;EndpointSuffix=core.windows.net";
private static readonly string containerName = "sampleblobcontainer";
private static readonly string imagesPath = @"C:\~~~~~~\AzureBlobStorage\AzureBlobStorage\Pictures\";

Connect to your Blob Container, create it if does not exist yet.

BlobServiceClient blobServiceClient = new BlobServiceClient(connectionString);   
BlobContainerClient containerClient = null;   

if (blobServiceClient.GetBlobContainers().Where(x => x.Name == containerName).Any())   
{   
    Console.WriteLine("Blob Container Client found.");   
    containerClient = blobServiceClient.GetBlobContainerClient(containerName);   
}   
else   
{   
    Console.WriteLine("Blob Container Client not found, creating a new one.");   
    containerClient = await blobServiceClient.CreateBlobContainerAsync(containerName); // Create the container and return a container client object   
    Console.WriteLine("Blob Container Client created.");   
}

Upload the images as Blobs.

// Get a reference to a blob
BlobClient blobClient = containerClient.GetBlobClient(filename);

Console.WriteLine("Uploading to Blob storage as blob:\n\t {0}\n", blobClient.Uri);

// Open the file and upload its data
using FileStream uploadFileStream = File.OpenRead(Path.Combine(imagesPath, filename));
await blobClient.UploadAsync(uploadFileStream, true);
uploadFileStream.Close();

List the Blobs in your container.

await foreach (BlobItem blobItem in containerClient.GetBlobsAsync())
{
    Console.WriteLine("\t" + blobItem.Name);
}

Project on GitHub.

Result

Azure Portal

Using Azure CLI

You need your key to manipulate your container, you find it at the same place where you found the connection string in the .NET sample.

Uploading the blob.

Validating in Azure Portal.

Listing Blobs

Setting and Retrieving properties and Metadata
 

Using Azure CLI

Getting Container Properties.

az storage container show --name $containerName --account-name $storageAccountName --account-key NSs2H9xg8DIpKH8y3EeYMiIpM1zYl/K9oAlBvDnPJaOpwUuS1GIwnbllIBuWngVpUYkpzmiRHdJiybgmGR3tag==

Setting and Getting Container Metadata, the attention that the update command overwrites the existing container metadata.

az storage container metadata update --account-name $storageAccountName --name $containerName --metadata creationType=AzureCli --auth-mode key --account-key "P3o/5qMhTrI8rJgt34SPSaeLuv7ED6Szft5kAf7hixh2UiSwltBW52CG4FejDs3nhV4t2lAE/2XEAIAsUvKorA=="

az storage container metadata show --account-name $storageAccountName --name $containerName --auth-mode key --account-key "P3o/5qMhTrI8rJgt34SPSaeLuv7ED6Szft5kAf7hixh2UiSwltBW52CG4FejDs3nhV4t2lAE/2XEAIAsUvKorA=="

Setting and Getting Blob Metadata, attention that the update command overwrites the existing blob metadata.

az storage blob metadata update --name "frequent.jpg" --account-name $storageAccountName --container-name $containerName --metadata creationType=azurecli --auth-mode key --account-key "P3o/5qMhTrI8rJgt34SPSaeLuv7ED6Szft5kAf7hixh2UiSwltBW52CG4FejDs3nhV4t2lAE/2XEAIAsUvKorA=="
az storage blob metadata show --name "frequent.jpg" --account-name $storageAccountName --container-name $containerName --auth-mode key --account-key "P3o/5qMhTrI8rJgt34SPSaeLuv7ED6Szft5kAf7hixh2UiSwltBW52CG4FejDs3nhV4t2lAE/2XEAIAsUvKorA=="

Using .NET

Getting Container Properties.

private static readonly string connectionString = "DefaultEndpointsProtocol=https;AccountName=samplestorageaccountblob;AccountKey=NSs2H9xg8DIpKH8y3EeYMiIpM1zYl/K9oAlBvDnPJaOpwUuS1GIwnbllIBuWngVpUYkpzmiRHdJiybgmGR3tag==;EndpointSuffix=core.windows.net";
private static readonly string containerName = "sampleblobcontainer";
static async Task Main(string[] args)
{
    Console.WriteLine("Program started. Connecting to Blob Client");
    BlobServiceClient blobServiceClient = new BlobServiceClient(connectionString);
    BlobContainerClient containerClient = null;

    if (blobServiceClient.GetBlobContainers().Where(x => x.Name == containerName).Any())
    {
        Console.WriteLine("Blob Container Client found.");
        containerClient = blobServiceClient.GetBlobContainerClient(containerName);
    }
    else
    {
        Console.WriteLine("Blob Container Client not found, creating a new one.");
        containerClient = await blobServiceClient.CreateBlobContainerAsync(containerName); // Create the container and return a container client object
        Console.WriteLine("Blob Container Client created.");
    }
    var properties = containerClient.GetProperties();
}

Setting and Getting Container Metadata, pay attention that the SetMetada method overwrites the existing metadata.

private static readonly string connectionString = "DefaultEndpointsProtocol=https;AccountName=samplestorageaccountblob;AccountKey=P3o/5qMhTrI8rJgt34SPSaeLuv7ED6Szft5kAf7hixh2UiSwltBW52CG4FejDs3nhV4t2lAE/2XEAIAsUvKorA==;EndpointSuffix=core.windows.net";
private static readonly string containerName = "sampleblobcontainer";

static async Task Main(string[] args)
{
    Console.WriteLine("Program started. Connecting to Blob Client");
    BlobServiceClient blobServiceClient = new BlobServiceClient(connectionString);
    BlobContainerClient containerClient = null;

    if (blobServiceClient.GetBlobContainers().Where(x => x.Name == containerName).Any())
    {
        Console.WriteLine("Blob Container Client found.");
        containerClient = blobServiceClient.GetBlobContainerClient(containerName);
    }
    else
    {
        Console.WriteLine("Blob Container Client not found, creating a new one.");
        containerClient = await blobServiceClient.CreateBlobContainerAsync(containerName); // Create the container and return a container client object
        Console.WriteLine("Blob Container Client created.");
    }

    Dictionary<string, string> containerMetadata = new Dictionary<string, string>();
    containerMetadata.Add("creationType", ".Net SDK");
    containerMetadata.Add("seccondMetadata", ".Net SDK");
    containerClient.SetMetadata(containerMetadata);
    var containerProperties = containerClient.GetProperties();
}

Setting and Getting Blob Metadata, pay attention that the SetMetada method overwrites the existing metadata.

private static readonly string connectionString = "DefaultEndpointsProtocol=https;AccountName=samplestorageaccountblob;AccountKey=P3o/5qMhTrI8rJgt34SPSaeLuv7ED6Szft5kAf7hixh2UiSwltBW52CG4FejDs3nhV4t2lAE/2XEAIAsUvKorA==;EndpointSuffix=core.windows.net";
private static readonly string containerName = "sampleblobcontainer";
static async Task Main(string[] args)
{
    Console.WriteLine("Program started. Connecting to Blob Client");
    BlobServiceClient blobServiceClient = new BlobServiceClient(connectionString);
    BlobContainerClient containerClient = null;

    if (blobServiceClient.GetBlobContainers().Where(x => x.Name == containerName).Any())
    {
        Console.WriteLine("Blob Container Client found.");
        containerClient = blobServiceClient.GetBlobContainerClient(containerName);
    }
    else
    {
        Console.WriteLine("Blob Container Client not found, creating a new one.");
        containerClient = await blobServiceClient.CreateBlobContainerAsync(containerName);
        Console.WriteLine("Blob Container Client created.");
    }

    Dictionary<string, string> blobMetadata = new Dictionary<string, string>();
    blobMetadata.Add("creationType", ".Net SDK");
    blobMetadata.Add("seccondMetadata", ".Net SDK");
    BlobClient client = containerClient.GetBlobClient("frequent.jpg");
    client.SetMetadata(blobMetadata);
    var blobMetadataValues = client.GetProperties().Value.Metadata;
}

The result from Azure Portal.

Moving items between Blob Storage and Containers.

You can copy a single Blob or a Batch of Blobs, here we will be moving a batch of Blobs from one container to another inside the same Blob Storage Account.

The error here is due to the Rare.jpg being rehydrated.

az storage blob copy start-batch \
    --account-name $storageAccountName \
    --destination-container "anothercontainer" \
    --source-container $containerName \
    --account-key NSs2H9xg8DIpKH8y3EeYMiIpM1zYl/K9oAlBvDnPJaOpwUuS1GIwnbllIBuWngVpUYkpzmiRHdJiybgmGR3tag==

Moving from one container to another inside another storage account.

az storage blob copy start-batch --account-name "cloudshell325844558" --source-account-name $storageAccountName --source-container "anothercontainer" --destination-container "containeranotheraccount" --account-key Oc3mEd/80DqJ65+nxMUJb/C9nq/hre+YLdVEfXeZ7M3gvcwJ7zqB7SAzNaiRtPyOrEtYXBub9Wn4xVUgTv5BTw== --source-account-key NSs2H9xg8DIpKH8y3EeYMiIpM1zYl/K9oAlBvDnPJaOpwUuS1GIwnbllIBuWngVpUYkpzmiRHdJiybgmGR3tag==

Hot, Cool, and Archiving Storage

By default, when you create a Blob its access tier is set automatically to the one configured when creating Azure Storage. By default, Blob Storages are created with the Hot Access Tier, in order to change the default access tier of the Storage Account, check the configuration tab under the settings sector.

To change the access tier of a blob you have to individually set its new access tier. We are going to organize the blobs according to their necessities: frequent images stay in the hot tier, not so frequent images stays in the cool tier and the rare images is going to be placed in the archiving tier.

az storage blob set-tier --account-name $storageAccountName --container-name $containerName --name "not so frequent.png" --tier Cool --account-key NSs2H9xg8DIpKH8y3EeYMiIpM1zYl/K9oAlBvDnPJaOpwUuS1GIwnbllIBuWngVpUYkpzmiRHdJiybgmGR3tag==

az storage blob set-tier --account-name $storageAccountName --container-name $containerName --name "rare.jpg" --tier Archive --account-key NSs2H9xg8DIpKH8y3EeYMiIpM1zYl/K9oAlBvDnPJaOpwUuS1GIwnbllIBuWngVpUYkpzmiRHdJiybgmGR3tag==

Result in Azure Portal.

To move one item from Archive Access Tier to another Access Tier you can also set the rehydrate priority to high. This feature is still in preview mode.

az storage blob set-tier --account-name $storageAccountName --container-name $containerName --name "rare.jpg" --tier Cool --rehydrate-priority high --account-key NSs2H9xg8DIpKH8y3EeYMiIpM1zYl/K9oAlBvDnPJaOpwUuS1GIwnbllIBuWngVpUYkpzmiRHdJiybgmGR3tag==

Implementing archiving and retention - Lifecycle management

In order to manage the lifecycle of our Blobs inside containers, let's create a new lifecycle management rule. From Azure Storage Account go to Lifecycle Management, under Blob Service, and Add a rule.

You may apply this rule to all blobs inside the storage account or filter the blobs to have this rule applied in this storage account.

Now you can configure how many days without being modified those blobs are going to be moved to other access tiers.

For now, that is it!

Thanks for reading this article, if you enjoyed it and it was helpful do not forget to like it. If you did not like it, do not forget to comment and tell me what I can improve in future articles.

C# Project on GitHub.

External References

• Azure Blob Storage Official Documentation;
• Blob Storage Access Tiers;
• Blob Storage Lifecycle Management Official Documentation;
• Az-CLI Official Documentation for Azure Storage;
• .NET SDK For Azure Blob Storage;
• Az-204 Measured Skills;
• Az-204 Official Website;