Azure Storage Account And Its Complete Overview

Azure Storage Account Overview

 

Each organization will have specific requirements for its cloud-hosted data. It can either store data within a particular region or need separate billing for different data categories. Azure storage accounts help to solemnize these types of policies and apply them to the Azure data. Azure offers enormous ways to store the data. There are several database options like Azure SQL Database, Azure Cosmos DB, and Azure Table Storage. Azure provides various ways to store and send messages, such as Azure Queues and Event Hubs. Even loose files can be stored using services like Azure Files and Azure Blobs.

 

What is Azure Storage Account?

 

A storage account is a container that bands a set of Azure Storage services together. Only data services from Azure Storage can be comprised in a storage account.

 

Integrating data services into a storage account allows the user to manage them as a group. The settings specified while creating the account, or setting that is changed after creation, is applicable everywhere. Once the storage account gets deleted, all the data stored inside gets removed.

 

 

Let us consider a machine manufacturing industry that produces machinery and instruments such as pulley, screws, and wedges. The products get marketed to stores which then sell them to consumers.

 

The designs and manufacturing processes of the industry get preserved as trade secrets. The spreadsheets, documents, and instructional videos that record this information are significant to the business and necessitate redundant storage. This data gets primarily retrieved from the main factory, so the user would prefer to store it in a nearby datacentre. The expenditure for this storage should be billed to the manufacturing department.

 

The industry also owns a sales group that creates demonstration and advertisement videos to promote consumers' products. Priority for this data is a low cost, rather than redundancy or location. This storage should be billed to the sales team. Such business handles require multiple Azure storage accounts, and each storage account will incorporate the appropriate settings for the data it holds.

 

Types of Azure Storage Accounts

 

Azure Storage provides different types of storage accounts. Each type supports unique features and has its pricing model. Consider these differences before creating a storage account to work out the best account for the applications. The types of storage accounts are,

• General-purpose v2 accounts: Basic storage account type for blobs, files, queues, and tables. Recommended for most scenarios using Azure Storage.
• General-purpose v1 accounts: Legacy account type for blobs, files, queues, and tables. Use general-purpose v2 accounts instead when possible.
• Block Blob Storage accounts: Storage accounts with premium performance characteristics for block blobs and appends blobs. It is recommended for scenarios with high transaction rates or scenarios that use smaller objects or require consistently low storage latency.
• File Storage accounts: Files-only storage accounts with premium performance characteristics. Recommended for enterprise or high-performance scale applications.
• Blob Storage accounts Legacy Blob-only storage accounts. Use general-purpose v2 accounts instead when possible.

Core Storage Services

 

Core storage services provide an enormously scalable object store for data objects, disk storage for Azure virtual machines (VMs), a file system service for the cloud, a messaging store for reliable messaging, and a NoSQL store.

 

 

The Azure Storage platform comprises the following data services,

• Azure Blobs are an immensely scalable object store for text and binary data.
• Azure Files are organized file shares for cloud or on-premises deployments.
• Azure Queue is a messaging store for consistent messaging between application components.
• Azure Tables are NoSQL store for schema-less storage of structured data.
• Azure Disks are block-level storage volumes for Azure Virtual Machines.

Azure Blob Storage

 

Azure Blob storage is an object storage solution designed for the cloud. Blob storage gets augmented for storing a massive amount of unstructured data. Unstructured data is data that does not stick to a specific data model or definition, like text or binary data. Blob storage objects can be accessed by the user or client application via HTTP/HTTPS from any part of the world. Azure Storage Rest API, Azure PowerShell, Azure CLI, or an Azure Storage client library is used to access Blob Storage objects. The flow and resources of a Blob Storage is pictured below,

 

 

The Blob Storage consists of a storage account with a container residing in it. Containers hold the respective blobs in them.

 

Azure Storage supports three types of blobs; they are,

 

Block Blobs

 

Block blobs are designed to store text and binary data. Block blobs are built with blocks of data that can be managed independently. It has a storage capacity of about 4.75 TiB of data. Larger block blobs are currently in preview and have storage up to 190.7 TiB

 

Append Blobs

 

Append Blobs are created with blocks like block blobs but are enhanced for append operations. Append blobs are preferred for scenarios like logging data from virtual machines.

 

Page Blobs

 

Page Blobs store random access files ranging up to 8 TB in size: these blobs stock virtual hard drive (VHD) files and function as disks for Azure virtual machines.

 

Azure Blob Use case

 

Let us consider an amplified-reality gaming company. The game runs on every mobile platform without any criteria. The scenario here is to add a new feature that allows users to record video clips of their gameplay and further upload them to the servers. Users can either watch the clips directly in-game or through the game website. A log on every upload and viewing can be planned and maintained to benefit analytics and traceability.

 

The user requirement would be a storage solution that can grip thousands of simultaneous uploads, a huge count of video data, and constantly growing log files. There is also a need for viewing functionality in all the mobile apps and website, so the user requires API access from multiple platforms and languages. Azure Blob storage could be an ideal solution for this application

 

Azure Blob Storage was designed to serve specific needs. If the business use case needs to store unstructured data like audio, video, images, etc., then you should probably go with this option. The objects which are being stored in Blob does not necessarily have an extension.

 

The following points describe the use case scenarios,

• Serving images or documents directly to a browser
• Storing Files for distributed access
• Streaming video and audio
• Writing to log Files
• Storing data for backup, restore, disaster recovery and archiving
• Storing data for analysis by an on-premises or Azure-hosted service

Azure Blob Pricing

 

Azure storage offers various access tiers, which allows storing the blob object data in a very cost-effective manner. The available access tiers include,

• Hot- Augmented for storing frequently accessed data.
• Cool- Optimized for storing less frequently accessed data, and the storage period lasts for at least 30 days.
• Archive- Enhanced for storing rarely accessed data and the storage period lasts for at least 180 days with flexible latency requirements

Hot Access Tiers

 

The storage access cost for the hot access tier is comparatively higher than the storage costs of cool and archive tiers. But access cost of hot access tiers is low. Real-time usage for the hot access tier include,

• Data that is in active use or predicted to be accessed often, in detail, the data that is read from and written to often.
• Data that is staged for processing and in eventual migration to the cool access tier come under hot access tiers.

Cool Access Tiers

 

In comparison to hot access tiers, the cool access tier has lower storage costs and higher access costs. This tier is envisioned for data that will stay back in the cool tier for at least 30 days. Real-time usage for the cool access tier include,

• Short-term backup and disaster recovery datasets
• Older media content that is not viewed regularly anymore but is expected to be easily available when accessed
• Large data sets which must be stored cost-effectively when more data is being collected for future processing. For example, long-term storage of scientific data or raw telemetry data from a manufacturing firm

Archive Access Tiers

 

The storage cost of archive access is low, but the data retrieval cost is high when compared to the hot and cool tiers. Data must sustain in the archive tier for at least 180 days or will be subjected to an early deletion charge. The retrieval time of data in an archive access tier might consume more time depending on the priority of the rehydration. Real-time usage for the Archive Access Tier include,

• Long-term backup, secondary backup, and archival datasets
• The preservation of original raw data even after processing it into the final usable form
• Long-time storage of compliance and archival data, which is hardly ever accessed

Data Storage Prices

 

Operations and Data Transfer Prices

 

 

Azure Storage Files

 

Azure Files provide fully managed File shares in the cloud that are approachable via the industry-standard SMB. Azure File shares can be attached parallelly by cloud or on-premises deployments of Windows, Linux, and macOS. It can be cached on Windows servers with Azure File Sync for quicker access. It permits the user to set up highly obtainable network file shares that can be accessed by using the standard Server Message Block (SMB) protocol. Multiple VMs can share similar files with both read and write permissions.

 

The only contrast between Azure Files and files on a corporate file share is, user can access the files from anywhere by using a URL that points to the file and contains a shared access signature (SAS) token. SAS tokens can be generated by the user; they allow specified access to a private asset for a specific time period.

 

 

The storage of file shares is one type of data that can be contained in an Azure Storage account. File shares can be implemented for many real-time scenarios,

• Many on-premises applications rely on file shares. This feature makes it flexible to migrate the applications that share data to Azure. If the file share is mounted to the same drive letter that is used by the on-premises application, the segment of application that accesses the file share should work with minimal, if any, changes.
• Configuration files are secured on a file share and can be accessed from multiple VMs. Tools and utilities consumed by multiple developers in a group can be stored on a file share, making sure that everybody can view it and use the same version.
• Resource logs, metrics, and crash dumps are just three models of data that can be written to a file share and can be handled or examined later.

Azure Storage File Use case

 

When it comes to file sharing, the end user should not be allowed to access the copies of the file from its URI and need to be mapped locally in the computers. This is when Azure File Storage fits customer needs. File Storage can be used if the business use case needs to deal mostly with standard File extensions like *.docx, *.png and *.bak, then you should probably go with this storage option.

 

Let us consider a financial company that is migrating an application to Azure that creates reports and data exports for users and other systems to consume. The company stores these reports and data exports on both NAS devices and Windows file shares. The files are stored this way so they can be easily shared between systems. The company wants to consolidate the storage of these files to a native cloud service. They want to continue to use the Server Message Block (SMB) to access the files securely.

 

The main concern is that they want to reduce the impact on existing applications, systems, and users. The company wants to use a drop-in replacement for their existing SMB protocol shares. They intend that there won't be any code changes required to support the moved data.

 

The following are the solutions provided by Azure Files for the use case scenario,

 

"Lift and shift" applications

 

The foremost challenge faced by the company is shifting its application to Azure. Azure Files ease the "lift and shift "of applications to the cloud, where the file application or user data is stored in the file share.

 

Replace or supplement on-premises File servers

 

After shifting the application to the cloud, the company wants to consolidate the storage of these files to a native cloud service. Azure Files is accustomed to completely replace or supplement any on-premise file servers or NAS devices.

 

Simplify cloud development

 

The company's final requirement is they would like to continue the usage of Server Message Block (SMB) to access its files safely. In Azure Files, user can store development and debugging tools that need to be accessed from many virtual machines.

 

Azure Blob Storage vs File Storage

 

Azure Blob Storage and File Storage, both services have their own defined properties and are implemented in different scenarios. Azure Files provides fully managed and organized cloud file shares that can be accessed from anywhere. Azure Blob Storage permits the storage of unstructured data and it can be accessed at a massive scale

Consider a development environment where every developer needs access to IDE and tools without using the internet to download it. In this situation, Azure Blob Storage would meet the need and using which the developer can only store development tools then give a link to the team to access the Blob location.

 

For implementing a File server in an organization, the user should choose the Azure Files option. A File server is used to share Files across departments in an organization. When it comes to File sharing, end user should not be allowed to access the copies of the file from its URI and need to be mapped locally in the computers. This is when Azure File Storage fits the organization's need.

 

Azure Storage File Pricing

 

Below are prices for Data storage and Operations and Data Transferring in Azure File,

 

Pricing for Data Storage

 

Pricing for Operations and Data Transferring

 

 

Azure Storage Queues

 

Azure Queue storage is an Azure service that implements cloud-based queues. Each queue maintains an inventory of messages. Application components access a queue employing a REST API or an Azure-supplied client library. Typically, you'll have one or more sender components and one or more receiver components. Sender components add messages to the queue. Messages are retrieved from the front of the queue for processing by receiver components. The subsequent illustration shows multiple sender applications adding messages to the Azure Queue and one receiver application retrieving the messages. Storage Queues are part of the Azure Storage infrastructure, feature a simple REST-based GET/PUT/PEEK interface, providing reliable, persistent messaging within and between services.

 

 

Concepts of Queue service

 

URL format

Queues are addressable using the subsequentURL format,

 

https://.queue.core.windows.net/

 

The following URL addresses a queue within the diagram,

 

https://myaccount.queue.core.windows.net/images-to-download

 

Storage account: All access to Azure Storage is completedthrough a storage account. For information about storage account capacity.

Queue: A queue contains a group of messages. The queue name must be all lowercase. For information on naming queues.

Message: A message, in any form, with range up to 64 KB. Before version 2017-07-29, the utmost time-to-live allowed is seven days. For version 2017-07-29 or later, the utmost time-to-live are often any positive number, or -1 indicating that the message doesn't expire. If this parameter is neglected, the default time-to-live is seven days.

 

Azure Storage Queues Use case

 

Imagine the user works as a developer for a major news organization that reports breaking news alerts. The company employs a worldwide network of journalists that are constantly sending updates through a web portal and a mobile app. A middle tier web service layer then takes those alert updates and publishes them online through several channels. However, it's been noticed the system is missing alerts when globally significant events occur.

 

The middle tier provides plenty of capacity to handle normal loads. However, a look at the server logs revealed the system was overloaded when several journalists tried to upload larger breaking stories at the same time. Some writers complained the portal became unresponsive, and others said they lost their stories altogether. The user has spotted a direct correlation between the reported issues and the spike in demand on the middle tier servers.

 

Clearly, user needs a way to handle these unexpected peaks. At such a situation user doesn't want to add more instances of the website and middle tier web service because they are expensive and, under normal conditions, redundant. They could dynamically spin up instances, but this takes time and they would have the issue waiting for new servers to come online.

 

This problem can be solved by using Azure Queue storage. A storage queue is a high-performance message buffer that can act as a broker between the front-end components and the middle tier. The front-end components place a message for each new alert into a queue. The middle tier then retrieves these messages one at a time from the queue for processing. At times of high demand, the queue may grow in length, but no stories will be lost, and the application will remain responsive. When demand drops back to normal levels, the web service will catch up by working through the queue backlog.

 

Azure Storage Queue properties

• Azure Storage Queues can be used when there is a need to store messages larger than the size of 80 GB.
• Azure Storage Queues provide logging of all the occurred transactions in the Storage Queue which can be used for analytics or audit purpose.
• If the application requires load balancing, failure tolerance, and increased scalability then Azure Storage Queues are the best choice.
• Storage Queues provide flexible and reliable delegated access control mechanisms. Users can provide access to Storage Account level or at the entity level.
• When it comes to scalability, the Storage Queues can store up to 200 TB of messages. It is also possible to create an unlimited number of Storage Queues in a Storage Account.
• Allowed characters in Storage Queues names are lowercase alphabets, numbers, and hyphens with length ranging from 3 to 63.
• The maximum message size in Storage Queues is 64 KB. If the messages are base64 encoded, then the maximum message size is 48 KB. Large message sizes are supported by combining queues with blobs, through which messages up to 200 GB can be enqueued as single data.
• Whenever the messages in a Storage Queue are retrieved more than the specified Dequeue Count, the messages are then moved to the configured dead-letter queue.

Azure Storage Queue Pricing

 

Below are prices for Data storage and Operations and Data Transferring in Azure Storage Queue,

 

Pricing for Data Storage

 

Pricing for Operations and Data Transferring

 

 

Azure Table Storage

 

Azure Table storage behaves as a service that stores structured NoSQL data inside thecloud, producing an attribute store with a schema less design. Because Table storage is schema less, it is easyto adapt your data because theneeds of your application evolve. Access to Table storage data is fast and cost-effective for several sorts ofapplications and is usuallylower in cost than traditional SQL for similar volumes of knowledge.

 

• Table: A table is a group of entities. Tables do not enforce a schema on entities, which refer that onetable can contain entities that have different sets of properties.
• Entity: An entity is a collection of properties, alike a database row. An entity in Azure Storage can be up to 1MB in size. An entity in Azure Cosmos DB will be up to 2MB in size.
• Properties: A property is a name-value pair. Each entity can comprise up to 252 properties to store data. Each entity also has three system properties that state a partition key, a row key, and a timestamp. Entities with an equivalentpartition key are oftenqueried more quickly and inserted/updated in atomic operations. An entity's row key'sits unique identifier within a partition

Azure Table Storage Use case

 

Table storage is employed to store flexible datasets like user data for web applications, address books, device information, or other metadata that the service requires. User can store any number of entities in a table, and a storage account may contain any number of tables, up to the capacity limit of the storage account Azure Table Storage Pricing. Azure tables are perfect for storing structured, non-relational data. Real time uses of Table storage include,

• Storing datasets that do not require complex joins, foreign keys, or stored procedures and may be denormalized for fast access
• Quickly querying data using a clustered index
• Accessing data using the OData protocol and LINQ queries with WCF Data Service .NET Libraries

Azure Table Storage Pricing

 

Below are prices for Data storage and Operations and Data Transferring in Azure Storage,

 

Pricing for Data Storage

 

Pricing for Operations and Data Transferring

 

Azure charges $0.00036 per 10,000 transactions for tables. Any kind of operation alongside the storage is counted as a transaction, including reads, writes and deletes.

 

Azure Disk Storage

 

Azure managed disks are block-level storage parts that are managed by Azure and used with Azure Virtual Machines. Managed disks are similar a physical disk in an on-premises server but virtualized. In managed disks, all you must do is specify the disk size, type, and provision the disk. Once the disk is provisioned, Azure handles the rest. Each disk can take one of three roles in a virtual machine,

• OS disk
  One disk in each virtual machine contains the operating system files. When user creates a virtual machine, he/she selects a virtual machine image and that fixes the operating system and the OS disk that's attached to the new machine. The OS disk has a maximum capacity of 2,048 GB.
  
  
• Data disk
  User can add one or more data virtual disks to each virtual machine to store data. For example, database files, website static content, or custom application code should be stored on data disks. The number of data disks that can be added depends on the virtual machine size. Each data disk has a maximum capacity of 32,767 GB.
  
  
• Temporary disk
  Each virtual machine contains a single temporary disk, which is used for short-term storage applications such as page files and swap files. The contents of temporary disks are lost during maintenance events, so do not use these disks for critical data. These disks are local to the server and are not stored in a storage account.

Azure Disk Storage Use case

 

The user manages a healthcare organization, and he is beginning a lift-and-shift migration to the cloud where many of their systems will be running on Azure virtual machines.

 

These systems have a variety of usage and performance profiles which are highly confidential. The user is concerned about the storage and does not want to access that data outside the virtual machine.

 

To address these needs, the organization's got to option is Azure Disk Storage. The Azure Disk Storage is capable of,

• "Lift and shift" of applications that use native file system APIs to read and write data to persistent disks.
• Preserve data that is not required to be accessed from outside the virtual machine to which the disk is attached.

Azure Disk Storage Pricing

 

Azure Managed Disks are the new and recommended disk storage offering to be usedwith Azure virtual machines for persistent storage of knowledge. Customer can use multiple Managed Disks with each virtual machine. Azure offers four varieties of Managed Disks — Ultra Disk, Premium SSD Managed Disks, Standard SSD Managed Disks, Standard HDD Managed Disks. For Azure Managed Disks, user will be billed on an hourly basis.

 

Azure Storage Security

 

Azure Storage accounts provide several high-level security benefits for the data in the cloud,

• Protect the data at rest
• Protect the data in transit
• Support browser cross-domain access
• Control who can access data
• Audit storage access

Encryption at rest

 

All data written to Azure Storage is automatically encrypted by Storage Service Encryption (SSE) with a 256-bit Advanced Encryption Standard (AES) cipher. SSE automatically encrypts data when writing it to Azure Storage. When data is read from Azure Storage, Azure Storage decrypts the data before returning it. This process incurs no additional charges and doesn't degrade performance. It cannot be disabled.

 

Encryption in transit

 

Keep the data secure by enabling transport-level securitybetween Azure and the client. It is advisable to use HTTPS to secure communication over the public internet. When the REST APIs is called to access objects in storage accounts, user can enforce the use of HTTPS by requiring Secure transfer for the storage account. After enabling secure transfer, connections that use HTTP will be refused. This flag will also enforce secure transfer over SMB by requiring SMB 3.0 for all file share mount.

 

CORS support

 

Azure Storage supports cross-domain access through cross-origin resource sharing (CORS). CORS uses HTTP headers so that a web application at one domain can access resources from a server at a different domain. By using CORS, web apps ensure that they load only authorized content from authorized sources. CORS support is an optional flag that can be enabled on Storage accounts.

 

Role-based access control

 

Azure Storage supports Azure Active Directory and role-based access control (RBAC) for both resource management and data operations. To security principals, user can assign RBAC roles that are scoped to the storage account. Use Active Directory to authorize resource management operations, such as configuration. Active Directory is supported for data operations on Blob and Queue storage.

 

Auditing access

 

Auditing is another part of controlling access. User can audit Azure Storage access by using the built-in Storage Analytics service. Storage Analytics logs every operation in real time, and you can search the Storage Analytics logs for specific requests. Filter based on the authentication mechanism, the success of the operation, or the resource that was accessed.

 

Azure Monitoring

 

Azure Monitor maximizes the supplyand performance of the applications and services by delivering a comprehensive solution for collecting, analyzing, and working ontelemetry from cloud and on-premises environments. It helps us understand how the applications are performing and proactively identifies issues affecting them and therefore theresources they depend upon.

 

Azure Storage Account Monitoring

 

Azure Storage Analytics performs logging and offers metrics data for a storage account. User can use this data to trace requests, examine usage trends, and identify issues with the storage account. To use Storage Analytics, user must enable it individually for each service to be monitored. This can be enabled from the Azure portal. 

 

The gathered data is stored in an eminent blob for logging and in renowned tables for metrics, which may be accessed using the Blob service and Table service APIs. Storage Analytics has a 20 TB limit on the quantity of stored data that is independent of total limit for the storage account. 

 

Once the desired Storage Account is configured to the Diagnostics option under the Monitoring section, user will be able to define the type of metrics data to monitor and the retention policy for the data. A default set of metrics is displayed in charts on the Storage Account blade, as well as the individual service blades. Once the metrics are enabled for a service, it may take up to an hour for data to appear in its charts.

 

 

References

 

I have originally published this article here.