Using Azure Queue Storage With Python

Azure Queue Storage is a simple queue storage service in Azure. It is responsible for asynchronously building message queues. The Queue Storage acts as a middle layer in a distributed environment and is exposed using the HTTP protocol. Azure Queue Storage consists of the following components: storage account, queue, and message. A storage account can have multiple queues, and each queue can have multiple messages.

This article will cover how to create a message queue in Azure Queue Storage. Furthermore, it will also explain how to perform other CRUD operations on the file using the Python programming language. 

Prerequisites

Before proceeding with this tutorial, it is recommended to have an active Azure Storage account and Python 3 installed on your machine.

1. Create the Storage Account

Before starting with Queue Storage, you will need the Azure Storage Account. This can be created either through the Azure Storage website, Azure CLI or PowerShell. Head over to Azure Storage website, and create storage named “demoStorage”. 

2. Get the Connection String

The second step is to get the connection string for your Storage Account. You can easily get the key from the “Access Key” under the “Settings” panel of your Azure Storage Account. Copy the connection string. Next, head over to Python code and set the connection string as shown below: 

AZURE_STORAGE_CONNECTION_STRING = "Your_Connection_String" 

3. Install the Azure SDK Storage

Since we are using Python to perform CRUD operations on Azure Queue Storage, it is crucial to install Azure SDK Storage for Python. Use the PIP command to install the required packages as shown below: pip install azure-storage-file-share

4. Create Azure Queue Storage Client

To communicate with the Azure Queue Storage, you will need a Queue Client. The Queue Client requires a queue name and the Storage Account connection string. Execute the code given below to create the Queue Client:

from azure.storage.queue import (
        QueueClient,
        BinaryBase64EncodePolicy,
        BinaryBase64DecodePolicy
)
import os, uuid
# Retrieve the connection string from an environment
# variable named AZURE_STORAGE_CONNECTION_STRING
connect_str = os.getenv("AZURE_STORAGE_CONNECTION_STRING")
# Create a unique name for the queue
q_name = "queue-" + str(uuid.uuid4())
# Instantiate a QueueClient object 
print("Creating queue: " + q_name)
queue_client = QueueClient.from_connection_string(connect_str, q_name)
# Create the queue
queue_client.create_queue()

Azure Queue Storage usually stores messages as texts. However, to store binary data, you can set Base64 encoding and decoding functions. The function will run before you put the messages in the queue.

# Setup Base64 encoding and decoding functions
base64_queue_client = QueueClient.from_connection_string(
                            conn_str=connect_str, queue_name=q_name,
                            message_encode_policy = BinaryBase64EncodePolicy(),
                            message_decode_policy = BinaryBase64DecodePolicy()
                        )

5. Upload the Message on Azure Queue Storage

Now that the Azure Queue Client is ready, you can easily upload the message using the send_message() function. Execute the code given below to store the message in the queue:

message = u"Hello Reader. This is a message."
print("Adding message: " + message)
queue_client.send_message(message)

6. View the Message Stored on the Azure Queue

You can also peek at the messages stored in the queue. The peek function allows users to look at the first message without removing it from the queue. The sample code to view the message in Azure Queue is given below:

# Peek at the first message
messages = queue_client.peek_messages()
for peeked_message in messages:
    print("Peeked message: " + peeked_message.content)

7. Edit the Message Stored on Azure Queue

Similar to other Azure Storage services, Azure Queue also allows the stored content to be edited. For instance, to change the contents of the message in the queue, execute the following code:

messages = queue_client.receive_messages()
list_result = next(messages)
message = queue_client.update_message(
        list_result.id, list_result.pop_receipt,
        visibility_timeout=0, content=u'I am replacing the old message.')
print("Updated message to: " + message.content)

8. Dequeue the Messages Stored on the Azure Queue

After the messages have been processed, it is essential to dequeue the messages stored on the queue. This step is crucial so that the messages are not processed again by mistake. 

messages = queue_client.receive_messages()
for message in messages:
    print("Dequeuing message: " + message.content)
    queue_client.delete_message(message.id, message.pop_receipt)

The receive_messages() function makes the message invisible to the other code for 30 seconds. You must call the delete_message() function to delete that message immediately. The above method deletes one message from the queue. To delete the messages in batch, execute the following code:

messages = queue_client.receive_messages(messages_per_page=5, visibility_timeout=5*60)
for msg_batch in messages.by_page():
   for msg in msg_batch:
      print("Batch dequeue message: " + msg.content)
      queue_client.delete_message(msg)

9. Dequeue the Messages Stored on the Azure Queue

To delete the queue from the Azure Storage account, execute the code following code:

print("Deleting queue: " + queue_client.queue_name)
queue_client.delete_queue()

Conclusion

In conclusion, the Azure File Queue Storage allows users to perform CRUD operations on messages stored in queues using Python and Azure Storage SDK. Hopefully this guide will assist you in setting up your Azure Queue Storage and performing CRUD operations using Python successfully.

Happy coding friends!