Azure Event Hub Messaging Using REST

The inception of Azure Cloud Services has not been too long. But the way it has matured - both, in providing new feature at a rapid rate and offering enterprise acceptable SLA - is highly appreciable. There are 100's of services which Azure provide off the box and new services are added every next day. This has been possible as the cloud platform team follows agile practices to delivering new features.

One of the most common requirements which any distributed application, be it enterprise or not enterprise grade, has is to manage to message. Azure has 3 services which offer message delivery platform, collectively covering almost every messaging use case.

• Event Grid (recently release)
• Event Hubs
• Service Bus

In this blog post, I will be talking about how to setup Event Hubs in Azure and use REST to post messages.

Below are the steps for sending a message to Event Hub using REST. 

SETTING EVENT HUB

1. Login to Azure Portal.
2. Select "Create a resource", search Event Hub.
   
3. When you click "Create", a "namespace" is created. A namespace is just like a bigger container under which you can place multiple small containers. In programming terms, you can have multiple classes in a namespace. Also, you can have same named class in the project if they are placed under a different namespace. In event hub worlds the logic remains same, you can have one or many event hubs running under a namespace. The benefit achieved out of this is, billing and resource usage for an Eventhub can be controlled at namespace level which further flows down to each event hub. Below, I have created an event hub's namespace by name blogEventHubns. You can select pricing tier as well as create a new resource group or use the existing.
   
4. Click "Create" which will add a new event hubs namespace.
   
5. Next, we need to add event hub to the event hubs namespace.
   
6. It's simple. I have created a new event hub by the name blogeventhub.
   
7. By default, every event hub is created with a $Default consumer group, which has 2 partitions. As you can see below, the event hub can have multiple partitions which are like pipes where the message are persisted in FIFO (first in first out) flow. By default, each event hub has 2 partitions. Then, you can also set up consumer groups which can be shared with clients who want to read a message.
   
8. The last bit in setting the hub is to check your SAS Policy keys, these keys are token which will authorize the client to access hub messages.
   

Service Bus Explorer

In your journey to work with Azure Event Hubs or Service Bus or Relay or any other Azure messaging platform, you will intensively need Service Bus Explorer. It will act as a debugger tool for you. It is one of the best tools to verify and visualize the messages available. This a tool developed by Azure client experience team, you can find more details and download @Service Bus Explorer

How to set up your service bus explorer,

1. Download the explorer from the link above.
2. Unzip the folder to a local drive.
3. Run ServiceBusExplorer.exe

4. Click the file and select "Connect". Select Enter connection string from the drop-down. Leave the rest as default. Go to Azure SAS Policy as in step 8 above, copy the connection string and past in the box. Click OK.
   
5. Now, you will see that all the entities create with your event hub namespace as in the screenshot below. Also, to view messages in each of the Partition, you will have to create a listener with each of the partitions. The listener will just read the message but it will not be removed from the Event Hub partition.
   
6. Partition Listener looks as below. Also, you have options to select the duration from when you want to view the message, refresh rate etc. Once configured, click the Start button. Select Event Tab as below, as soon as, you send a new message to the hub, it will be displayed in the tab.
   

Making REST request to the hub from IoT client. A lot is already available in Docs for Event Hub. But when I started implementation the docs did not feel very streamline with the implementation. Hence below, I will walk you through in simple steps how you can create your REST request. For this blog purpose, I will be using Postman to make REST request to the hub. In real-world IOT device or any other platform from where you want to make the REST request will work in similar fashion.

1. The first item to look at is POST URL, the URL as stated in MS docs is universal apart from appending you hub namespace.
   
   In our case, it will be,
   https://blogEventHubns.servicebus.windows.net/blogeventhub/messages
2. Next bit is setting the header. Below are the header keys you need to add in each request.
   Content-Type:application/x-www-form-urlencoded
   Host:blogEventHubns.servicebus.windows.net
    
3. The most important element in the header is Authorization key which carries the SAS token. If you go to https://docs.microsoft.com/en-us/rest/api/eventhub/generate-sas-token you can find options to select the language of your choice to generate token. I am using C#, the code as in point 5 has a method with 3 param string resourceUri, string keyName, string key. Details around each of these are as below,
   
   resourceUri 

4. https://blogEventHubns.servicebus.windows.net/blogeventhub/messages
   keyName: RootManageSharedAccessKey (this is available at)
   
   key
   
   Above the red circle, you have an option for 2 keys Primary or Secondary. You can select either. If your key is compromised you have reset option as well.

5. Code explanation for SAS Token.
   
   Line 3,4,5: They generate a timestamp which will be associated with the token i.e your request with this token will only valid till the time specified does not expires
   Line 6: resource url detailed above is encoded with the expiry stamp
   Line 7: Key is encrypted with Hash-based Message Authentication Code (HMAC) by using the SHA256
   Line 8: Converting HASH Key+Resource URL+Expiry into base 64
   Lin 9: Concat all the variable

   private static string createToken(string resourceUri, string keyName, string key) {  
       TimeSpan sinceEpoch = DateTime.UtcNow - new DateTime(1970, 1, 1);  
       var week = 60 * 60 * 24 * 7;  
       var expiry = Convert.ToString((int) sinceEpoch.TotalSeconds + week);  
       string stringToSign = HttpUtility.UrlEncode(resourceUri) + "\n" + expiry;  
       HMACSHA256 hmac = new HMACSHA256(Encoding.UTF8.GetBytes(key));  
       var signature = Convert.ToBase64String(hmac.ComputeHash(Encoding.UTF8.GetBytes(stringToSign)));  
       var sasToken = String.Format(CultureInfo.InvariantCulture, "SharedAccessSignature sr={0}&sig={1}&se={2}&skn={3}", HttpUtility.UrlEncode(resourceUri), HttpUtility.UrlEncode(signature), expiry, keyName);  
       return sasToken;  
   } 

    
6. Once you run the code in point 5, below is the Authorization string you will receive SharedAccessSignature

   sr=https%3a%2f%2fblogEventHubns.servicebus.windows.net%2fblogeventhub%2fmessages&sig=XXXXXXXXXXXXXXXXXXXXX&se=1524543958&skn=RootManageSharedAccessKey

7. Next steps is to open Postman and set the values.
   
8. Setup request type as POST, enter URL as stated in step 3.
   
9. Add header keys as below,
   
   Authorization: as in Step 5,6
   Content-Type:application/x-www-form-urlencoded
   Host:blogEventHubns.servicebus.windows.net
    
10. Set the body of the message.
    
11. Once request complete you will received 201 response which is a success.
    
    If you receive any of below message you need to re-verify Authorization header values
    

Now, let's look at our service bus explored to finally verify the end to end execution. As in the screenshot, you can see Partition 0 received the message as posted in request body in point 10 above.

 

Summary

It's a great milestone to get the message posted to Event Hub. Once the message is persisted in Event hub, you can build a consumer using a custom application or you can connect the hub to Azure Function or Azure Stream Analytics for processing further. In the next post, I will detail about how we can set up an Azure Function to keep a track of new messages. Then, using custom logic, take the incoming message from hub and post to the desired endpoint for processing.