Windows Azure Cloud Service And Service Bus Queues

Before explaining the Windows Azure cloud service, let us see what services are offered by the Windows Azure platform.

Windows Azure is an OS that is running in Microsoft data centers and where we can deploy our application and use it as a client.

Windows Azure offers the following five main services:

1. IIAS (Infrastructure as a Service).
   
   
   • Creating a Virtual machine.
     
     
2. PaaS (Platform as a Service).
   
   
   • Cloud Service, where we can deploy our applications.
     
     
3. SaaS (Software as a Service).
   
   
   • Office 365 SharePoint online.
     
     
4. Storage as a Service.
   
   
   • Storing our data in Microsoft data centers using BLOBs and tables.
     
     
5. .Net Services.
   
   
   • Access control service, Service Bus, Caching service.

What is a cloud service?

A cloud service is a platform for deploying our applications to. It offers a hosting environment for us. Previously, it was called hosted services, but it is now called Cloud services.

Simply , a Cloud service might tell us "please give me your code, I will run it for you".

Cloud Service Components

1. Cloud Service Role
   
   
   • It is comprised of application files and configurations.
   
   A cloud service can have the following roles:
   
   
   1. Web Role: Provides a dedicated IIS web server used for hosting front end web applications.
   2. Worker Role: works as a background worker process that requires less user interaction like windows services.
   3. VM Role: As the name itself says it is a Virtual Machine Roll.
      
   
   There are two ways to create virtual machines in the Windows Azure platform, they are:
   
   
   • Creating the VM in IAAS.
   • Creating the VM by using a VM Role.
   
   The virtual machine created as IAAS is stateful, but not VM created by using VM role.
   
   I.e. whenever a VM disappears from a location due to some disaster and may appear in another location, it will not maintain its state (like opened application state).
   
   Also provisioning VM and de-provisioning will be faster in a VM using a VM role compared to an IAAS VM.
   
   But, VM as IAAS, maintains OS and other software updates automatically by app fabric. But in the VM role, the administrator has to maintain all these updates or tasks.
    
2. Role Instance
   
   
   • The role instance is nothing but a virtual machine where the code and configuration files run.
   • The role can have multiple instances defined in the configuration file. A rule can have 20 instances for the default subscription. We can scale out and scale down the instance manually or by using the auto scaling application block library.
    
3. Guest Operating System
   
   
   • Currently the role supports Windows Server 2008 R2 (Enterprise and Standardd) as the operating system.

What are the components required to deploy an application on a cloud?

1. Service Definition file (.csdef) defines the service model and role instances.
2. Service Configuration file (. cscfg) provides configuration settings for the role.
3. Service Package (.cspkg) is a comprised application code as well as resources that will be ready to be deployed.

Application Goal

The demo scenario is: Zipping documents from a container and sending an email to a user after the completion of the job. The application is available for downloads.

I am choosing:

1. Creating a cloud service.
   
   • To deploy our application.
2. Creating Certificate:
   
   • To enable the remote Desktop option for a role.
3. Creating a Service Bus & Queue.
   
   • To communicate between two components.
4. Windows Azure Application:
   
   • Web Role: Front end web application like ASP.Net web application.
   • Worker Role: To perform zipping tasks in a background as background worker.
5. Deployment.

Creating Service Bus

A Service Bus is nothing but a hosted service that provides a secure means of wide-spread communication.

1. Open the management portal.
   
   On the left hand side panel, select service bus.
   
   
   
    
2. Add a namespace and it will check for availability immediately.
   
   
   
   Pick the same region that you intend to deploy your application to.
    
3. A confirmation window appear when the service bus has been created successfully.
   
   
    
4. Once the service bus has been created, you can get an access key as in the following:
   
   
   
   An access key is nothing but a connection string for connection to the service bus.

Before starting the Service Bus Queue, we will see the types of communication mechanisms available in the Azure platform.

Types of Communication Mechanisms Available in Azure

The Windows Azure platform offers two types of communication mechanisms, they are:

1. Windows Azure Queues.
   
   
   • Part of the Windows Azure storage service.
    
2. Service Bus Queues.
   
   
   • Part of the Windows Azure messaging infrastructure.

So, when should we use Windows Azure Queues and Service Bus Queues?

To understand this, we should see the differences between these two they are:

1. The Service Bus Queue offers ordering guaranteed to be first in-first out (FIFO) but not in Azure queues.
2. Service bus queues offers transaction support but not Azure queues.
3. Lock duration between 60 seconds to 5 minutes, but 30 seconds to 7 days in Azure queues.
4. Service bus supports automatic dead lettering but not in Azure queues.
   
   • Whenever a message expires or reaches the queue size limit, that message will be removed from the queue by default. So, those messages will not be available for processing. If you need that message then you must set dead lettering true when the service bus queue is created. These messages will end up in a new queue called dead lettered queue and messages can backside from this queue.
5. One of the best options of the service bus is Automatic duplicate detection, but that is not available in Azure queues.
6. Service bus queues support WCF integration, but not in Azure queues.
7. Maximum message size will be 256kb in service bus queues and 64kb in Azure queues.
8. Maximum queue size is 5GB in service bus queues and 100TB (depends on storage account size) in Azure queues.
9. Message Time-To-Live is unlimited in service buss queues and a maximum of 7 days in Azure queues.
10. The most important thing is that service bus queues support authentication using ACS, but Azure queues support simple storage key authentication.
11. The cost will be the same for both and is $0.01 for 10000 transactions.
12. The storage cost will be extra for Azure queues.

By these differences and based on our need, we can select between service bus queues and Azure queues.

I am selecting service bus queues here because I need automatic duplicate detection.

Before starting service bus queues, let us see the service bus architecture

Service Bus



What is a Service Bus?

Service bus is a hosted service that provide a secure means of wide-spread communication between two components.

The communication scenarios or styles available in the service bus are:

1. Service buss queues.
   
   A simple way of communication between two components . The sender will send a message to a service bus queue that will be received by the receiver and reception of the message will depend on the receiver's availability.
   It is a unidirectional communication.
   
   
   
   The message is nothing but a brokered message class that encapsulates data inside.
    
2. Topics and Subscriptions.
   
   In certain scenarios, the message has to be sent to multiple consumers based on a condition or a filter as in the following. In that case we can use topic and subscription. It is also a unidirectional communication.
   
   
   
    
3. Relays.
   
   
   • In some cases, we need bi-directional communication like a request from the sender and the response from the receiver, so then, we can use relays.
   
   
   
   Relays also support exposure of a WCF service hosted in a corporate network with the cloud.

Creating service bus queues

Queues can be created using Azure Management Portal or by using C#.
Let us see how to create queues and sending messages using queues.
The most important thing while using a service bus in Azure application is, you nust import the service bus assembly into your application.

Import the Service Bus Namespaces

Add the following to the top of any C# file where you want to use Service Bus queues:

using Microsoft.ServiceBus;
using Microsoft.ServiceBus.Messaging;

You are now ready to write code against the Service Bus.

How to Set Up a Service Bus Connection String

The Service Bus uses a connection string to store endpoints and credentials. You can put your connection string in a configuration file, rather than hard-coding it in code:

• When using Windows Azure Cloud Services, it is recommended you store your connection string using the Windows Azure service configuration system (*.csdef and *.cscfg files).
• When using Windows Azure Web Sites or Windows Azure Virtual Machines, it is recommended you store your connection string using the .NET configuration system (e.g. web.config file).

In both cases, you can retrieve your connection string using the "CloudConfigurationManager.GetSetting" method as shown later in this guide.

Configuring your connection string when using Cloud Services

The service configuration mechanism is unique to Windows Azure Cloud Services projects and enables you to dynamically change configuration settings from the Windows Azure Management Portal without redeploying your application. For example, add a Setting to your service definition (*.csdef) file, as in the following:

<ServiceDefinition name="WindowsAzure1">
...
    <WebRole name="MyRole" vmsize="Small">
        <ConfigurationSettings>
            <Setting name="Microsoft.ServiceBus.ConnectionString" />
        </ConfigurationSettings>
    </WebRole>
...
</ServiceDefinition>

You then specify values in the service configuration (*.cscfg) file:

<ServiceConfiguration serviceName="WindowsAzure1">
...
    <Role name="MyRole">
        <ConfigurationSettings>
            <Setting name="Microsoft.ServiceBus.ConnectionString"
                     value="Endpoint=sb://[yourServiceNamespace].servicebus.windows.net/;SharedSecretIssuer=[issuerName];SharedSecretValue=[yourDefaultKey]" />
        </ConfigurationSettings>
    </Role>
...
</ServiceConfiguration>

Use the issuer and key values retrieved from the Management Portal as described in the previous section.

Configuring your connection string when using Web Sites or Virtual Machines

When using Web Sites or Virtual Machines, it is recommended you use the .NET configuration system (e.g. web.config). You store the connection string using the <appSettings> element:

<configuration>
    <appSettings>
        <add key="Microsoft.ServiceBus.ConnectionString"
             value="Endpoint=sb://[yourServiceNamespace].servicebus.windows.net/;SharedSecretIssuer=[issuerName];SharedSecretValue=[yourDefaultKey]" />
    </appSettings>
</configuration>

Use the issuer and key values retrieved from the Management Portal as described in the previous section.

How to Create a Queue

You can perform management operations for Service Bus queues via the NamespaceManager class. TheNamespaceManager class provides methods to create, enumerate, and delete queues.

This example constructs a NamespaceManager object using the Windows AzureCloudConfigurationManager class with a connection string consisting of the base address of a Service Bus service namespace and the appropriate credentials with permissions to manage it. This connection string is of the form:

Endpoint=sb://[yourServiceNamespace].servicebus.windows.net/;SharedSecretIssuer=[issuerName];SharedSecretValue=[yourDefaultKey]

For example, given the configuration settings in the previous section:

// Create the queue if it does not exist already
string connectionString =
    CloudConfigurationManager.GetSetting("Microsoft.ServiceBus.ConnectionString");

var namespaceManager =
    NamespaceManager.CreateFromConnectionString(connectionString);

if (!namespaceManager.QueueExists("TestQueue"))
{
    namespaceManager.CreateQueue("TestQueue");
}

There are overloads of the CreateQueue method that enable you to tune properties of the queue (for example, to set the default "time-to-live" value to be applied to messages sent to the queue). These settings are applied using the QueueDescription class. The following example shows how to create a queue named "TestQueue" with a maximum size of 5GB and a default message time-to-live of 1 minute:

// Configure Queue Settings
QueueDescription qd = new QueueDescription("TestQueue");
qd.MaxSizeInMegabytes = 5120;
qd.DefaultMessageTimeToLive = new TimeSpan(0, 1, 0);

// Create a new Queue with custom settings
string connectionString =
    CloudConfigurationManager.GetSetting("Microsoft.ServiceBus.ConnectionString");

var namespaceManager =
    NamespaceManager.CreateFromConnectionString(connectionString);

if (!namespaceManager.QueueExists("TestQueue"))
{
    namespaceManager.CreateQueue(qd);
}

Note: You can use the QueueExists method on NamespaceManager objects to check if a queue with a specified name already exists within a service namespace.

How to Send Messages to a Queue

To send a message to a Service Bus queue, your application creates a QueueClient object using the connection string.

The code below demonstrates how to create a QueueClient object for the "TestQueue" queue created above using the CreateFromConnectionString API call:

string connectionString =
    CloudConfigurationManager.GetSetting("Microsoft.ServiceBus.ConnectionString");

QueueClient Client =
    QueueClient.CreateFromConnectionString(connectionString, "TestQueue");

Client.Send(new BrokeredMessage());

Messages sent to (and received from) Service Bus queues are instances of the BrokeredMessage class.BrokeredMessage objects have a set of standard properties (such as Label and TimeToLive), a dictionary that is used to hold custom application specific properties, and a body of arbitrary application data. An application can set the body of the message by passing any serializable object into the constructor of the BrokeredMessage, and the appropriate DataContractSerializer will then be used to serialize the object. Alternatively, a System.IO.Stream can be provided.

The following example demonstrates how to send five test messages to the "TestQueue" QueueClientobtained in the code snippet above:

for (int i=0; i<5; i++)
 {
   // Create message, passing a string message for the body
   BrokeredMessage message = new BrokeredMessage("Test message " + i);

   // Set some addtional custom app-specific properties
   message.Properties["TestProperty"] = "TestValue";
   message.Properties["Message number"] = i;  

   // Send message to the queue
   Client.Send(message);
 }

Service Bus queues support a maximum message size of 256 Kb (the header, that includes the standard and custom application properties, can have a maximum size of 64 Kb). There is no limit on the number of messages held in a queue but there is a cap on the total size of the messages held by a queue. This queue size is defined at creation time, with an upper limit of 5 GB.

How to Receive Messages from a Queue

The easiest way to receive messages from a queue is to use a QueueClient object. These objects can work in two different modes: ReceiveAndDelete and PeekLock.

When using the ReceiveAndDelete mode, reception is a single-shot operation; that is, when the Service Bus receives a read request for a message in a queue, it marks the message as consumed, and returns it to the application.

The ReceiveAndDelete mode is the simplest model and works best for scenarios in which an application can tolerate not processing a message in the event of a failure. To understand this, consider a scenario in which the consumer issues the receive request and then crashes before processing it. Because the Service Bus will have marked the message as being consumed, when the application restarts and begins consuming messages again, it will have missed the message that was consumed prior to the crash.

In PeekLock mode (which is the default mode), the receive becomes a two-stage operation, that makes it possible to support applications that cannot tolerate missing messages. When the Service Bus receives a request, it finds the next message to be consumed, locks it to prevent other consumers receiving it, and then returns it to the application. After the application finishes processing the message (or stores it reliably for future processing), it completes the second stage of the receive process by calling Complete on the received message. When the Service Bus sees the Complete call, it marks the message as being consumed and removes it from the queue.

The example below demonstrates how messages can be received and processed using the default PeekLockmode. To specify a different ReceiveMode value, you can use another overload forCreateFromConnectionString. This example creates an infinite loop and processes messages as they arrive in the "TestQueue":

Client.Receive();

// Continuously process messages sent to the "TestQueue"
while (true)
{ 
   BrokeredMessage message = Client.Receive();

   if (message != null)
   {
      try
      {
         Console.WriteLine("Body: " + message.GetBody<string>());
         Console.WriteLine("MessageID: " + message.MessageId);
         Console.WriteLine("Test Property: " +
            message.Properties["TestProperty"]);

         // Remove message from queue
         message.Complete();
      }
      catch (Exception)
      {
         // Indicate a problem, unlock message in queue
         message.Abandon();
      }
   }
}

How to Handle Application Crashes and Unreadable Messages

The Service Bus provides functionality to help you gracefully recover from errors in your application or difficulties processing a message. If a receiver application is unable to process the message for some reason, then it can call the Abandon method on the received message (instead of the Complete method). This will cause the Service Bus to unlock the message within the queue and make it available to be received again, either by the same consuming application or by another consuming application.

There is also a timeout associated with a message locked within the queue, and if the application fails to process the message before the lock timeout expires (for example, if the application crashes), then the Service Bus unlocks the message automatically and makes it available to be received again.

In the event that the application crashes after processing the message but before the Complete request is issued, the message will be redelivered to the application when it restarts. This is often called "At Least Once Processing", that is, each message will be processed at least once but in certain situations the same message may be redelivered. If the scenario cannot tolerate duplicate processing, then application developers should add additional logic to their application to handle duplicate message delivery. This is often done using the MessageId property of the message, that will remain constant across delivery attempts.

Summary

So, based on our need, we can select either service bus queues or Windows Azure queues. There is a demo project associated with this project. You can refer to that. The next article will be about communication styles offered by the Windows Azure service bus.