Deploy A Google Action On Azure

Introduction

 

While Alexa still holds the majority of the smart speaker market at 61.1% (as of Jan. 2019), Google is catching up with a 23.9% share. When most people think of Google and smart speakers, Google Home devices come to mind. Voice apps on Alexa are deployed to the Alexa Skill Store. On Google, developers deploy Google Actions to Google Assistant, which is on every Android device. While the Android mobile phone market share is at 44% in the United States, it's at 75% worldwide. This makes Google Actions hard to ignore.

 

This article walks through how to create a Google Action, connect it to Dialogflow, and then create an Azure Function. It is based on the Alexa Space Facts tutorial. In that example, a user requests a random fact that is chosen from a list and read to the user. It uses an Alexa Skill backed by a Lambda hosted on Amazon Web Services. The code in this article uses the same voice commands and facts as the original tutorial but creates a Google Action backed by an Http Trigger hosted in an Azure Function.

 

This article steps should be creating an Azure Function based on the node.js code in the Alexa tutorial. It then sets up a Google Action, connects it to Dialogflow, and sets up a public tunnel for development and testing. Finally, the Azure Function is deployed and the Dialogflow configuration is updated to use it.

 

Create the Space Facts Azure Function

 

The code to support the Action is too large to cover completely in the space of a single article. Download the attached demo and use it locally. These steps cover the key steps taken to create the Azure Function. They apply to Visual Studio 2017, any edition.

1. Create a new project. Select Cloud in the left navigation bar. Select Azure Function.
   
   
   
   
2. Select Http trigger.
   
   
   
   
3. Add the Google.Cloud.Dialogflow.V2 NuGet package. Make sure to check to Include prerelease. Chose the latest beta. At the time of this writing, it's 1.0.0-beta02. This package is used to serialize the Dialogflow WebhookRequest and WebhookResponse messages.
   
   

Inside the Space Facts Azure Function

 

The Azure Function sits at the end of an HTTP endpoint and receives a POST request. It loads the request body text into a string.

[FunctionName("GoogleSpaceFactsFunction")]
public static async Task<IActionResult> Run(
    [HttpTrigger(AuthorizationLevel.Function, "get", "post", Route = null)] HttpRequest req,
    ILogger log)
{
    . . .
    string requestBody = await new StreamReader(req.Body).ReadToEndAsync();
    . . .

The message could match two possible formats. Google Actions can issue either Conversation Webhook requests or Dialogflow Webhook requests. They are two distinctly different formats. If the Google Action uses Dialogflow, then, as expected, the messages will be Dialogflow Webhook requests, otherwise, it'll send a Conversation Webhook message.

 Action Health Check

When in production, Google monitors the Action using a health check request. Even when using Dialogflow, the health check message format is a Conversation message. Surprisingly, I was able to put a Google Action in production without supporting the Google Action Health Check. I immediately started getting email notifications indicating the Google Action is not healthy and would be delisted until it responds to the health check message. 

 

The IsHealthCheck method attempts to deserialize the request into a ConversationRequest class and detect if it is a health check. Since the message could be "Dialogflow Webhook deserialization errors are logged", but don't cause the method to raise an exception. The response to a health check can be a valid Dialogflow Webhook response. The attached project sends the same response to a health check as it does when a user makes a request satisfying the health check. 

public static bool IsHealthCheck(string requestBody, ILogger log)
{
    bool isHealthCheck = false;
    ConversationRequest convReq = null;

    try
    {
        convReq = JsonConvert.DeserializeObject<ConversationRequest>(requestBody);
    }
    catch (Exception ex)
    {
        log.LogInformation(ex, "Web hook request is not a health check. Cannot deserialize request as a health check.");
    }

    var firstInput = convReq?.Inputs?.FirstOrDefault(x => (x.Intent?.Equals(INTENT_MAIN)).GetValueOrDefault(false));

    if (firstInput != null)
    {
        var arg = firstInput.Arguments?.FirstOrDefault(x => (x.Name?.Equals("is_health_check")).GetValueOrDefault(false));

        if (arg != null)
        {
            isHealthCheck = arg.BoolValue;
        }
    }

    return isHealthCheck;
}

UserId Management 

If you've developed an Alexa Skill, then you have come to expect a userId to be provided the Alexa request. Google Action requests (both Dialogflow and Conversation Webhooks) currently include a userId, but that is going away as of May 31st, 2019 (see Anonymous User Identity). It's up to the Azure Function to provide a new userId and place it in a userStorage property on the response. The contents of the property are returned on subsequent requests. The GetUserId method first checks the userStorage property and then falls back to the legacy userId. If no validation is found in either location, it generates a new unique user id.

private static string GetUserId(WebhookRequest request)
      {
          string userId = null;
          Struct intentRequestPayload = request.OriginalDetectIntentRequest?.Payload;

          var userStruct = intentRequestPayload.Fields?["user"];

          string userStorageText = null;
          if ((userStruct?.StructValue?.Fields?.Keys.Contains("userStorage")).GetValueOrDefault(false))
          {
              userStorageText = userStruct?.StructValue?.Fields?["userStorage"]?.StringValue;
          }

          if (!string.IsNullOrWhiteSpace(userStorageText))
          {
              UserStorage userStore = JsonConvert.DeserializeObject<UserStorage>(userStorageText);

              userId  = userStore.UserId;
          }
          else
          {
              if ((userStruct?.StructValue?.Fields?.Keys.Contains("userId")).GetValueOrDefault(false))
              {
                  userId = userStruct?.StructValue?.Fields?["userId"]?.StringValue;
              }

              if (string.IsNullOrWhiteSpace(userId))
              {
                  // The user Id is not provided. Generate a new one and return it.
                  userId = Guid.NewGuid().ToString("N");
              }
          }

          return userId;
      }

Locally Running the Azure Function

 

Start your project in Debug mode. This should result in a command line window that looks similar to,

 

 

By default, Http Trigger Azure Functions launch locally on port 7071. Dialogflow REST API fulfillment requires a publicly accessible endpoint. ngrok is a free utility that creates a public endpoint on port 80 and 443 and maps them to a local port.

 

Go to ngrok.com, sign up for an account, and download the ngrok command line utility. Follow the instructions to connect the ngrok utility to your account. Add it to your PATH in your local environment so it's accessible when you create a command line window. Open a command line window and start ngrok with:

 

ngrok http 7071 

 

This will launch a window that looks similar to:

 

 

This makes the Azure Function available at:

 

https://20b9e3d7.ngrok.io/api/GoogleSpaceFactsFunction 

 

The first portion of the URL is a random string that's regenerated each time ngrok is executed. The endpoint will remain available for over eight hours. This URL will be used in the next.

 

Creating Google Action

 

Open a browser and navigate to console.actions.google.com. If you have a Google Home device, use the same account you used to configure the device. This will let you use the device to test your Action. These steps will configure an Action and point it to Dialogflow.

1. Click Add/import a project and, when prompted, name it Space Facts.
   
   
2. On the next screen, select Conversational in the lower right-hand corner of the dialog. Do not select one of the other template options, like Games & Fun. They use designers that hide access to Dialogflow.
   
   
   
   
3. Give the action a display name. This is what end users will speak to type to activate your Google Action. Invocation names must be unique across all Google Actions. Another Google Action used the name Space Facts and so I selected Space Info. When you create your Google Action, you'll have to pick a unique name.
   
   Select Invocation in the left navigation panel and enter a unique display name. Click Save in the upper right corner.
   
   
   
   4. Select Actions in the left navigation pane. Click Add Action. Leave the default Custom intent selected. Click Build. This will open a new window on the Dialogflow console
   
   
   
   
4. Name the agent Space-Facts. This name does not need to be unique.
   
   
5. Select Intents in the left navigation pane and click Create Intent. Name the new Intent GetNewFactIntent. 
   
   
6. Add the following training phrases:
   
   a fact
   a space fact
   tell me a fact
   tell me a space fact
   give me a fact
   give me a space fact
   tell me trivia
   tell me a space trivia
   give me trivia
   give me a space trivia
   give me some information
   give me some space information
   tell me something
   give me something
   
   
7. Scroll to the bottom of the Intent editing page and expand the last option, Fulfillment. Select the Enable webhook call for this intent option. 
   
   
   
   
8. Click Save at the top of the page.
   
   
9. Navigate to the other two Intents, set the same Fulfillment option in step 7 and save the updates.
   
   
10. Select Fulfillment in the left navigation bar. Enable Webhook Fulfillment and enter the URL generated when creating a tunnel using ngrok. Scroll to the bottom of the page and select Save.
    
    
    
    
11. On the right side of the screen, click see how this works in Google Assistant. This will update the Google Action with the latest Dialogflow updates and return to the Action console.
    
    
12. Navigate to the Test Simulator using the left navigation bar. Type in Talk to {DisplayName}. Use the Display Name of the Google Action you created in step 3. If ngrok is running and the Azure Function project is running in debug mode, then you should get a response.
    
    
    
    
13. Open a browser and navigate to http://localhost:4040. nGrok has an HTTP listener on port 4040 and reports on all traffic. You can see the message that the Google Action generates while using Dialogflow. You can capture these JSON messages and save them for unit testing or manually resubmit an HTTP request with the Replay button.
    
    
    
    
14. Optionally, if you have a Google Home device configured with the same account you used to log into the Action console, you can tell it "Hey Google, talk to {DisplayName}" and see the request come through in the ngrok listener. You can also set a breakpoint in the Azure Function project and hit it while testing.

Deploy to Azure

 

The current Dialogflow configuration points to a development environment, which is perfectly viable while designing and testing. Before submitting an Action for publication, it needs to be hosted in a stable environment. The following steps show how to deploy the Azure Function.

 

These steps assume that you have an active Azure account.

1. In Visual Studio 2017, right-click on the Azure Function project and select Publish.
   
   
2. Leave Create New selected. Click Publish in the lower right corner of the dialog box.
   
   
3. Log into your Azure account if necessary or select your existing Azure account. 
   
   
4. Accept the default values and click Create. This process could take a few minutes.
   
   
   
   
5. Once the deployment completes, open a browser and navigate to portal.azure.com. Log in with your Azure account.
   
   
6. Select Function Apps in the left navigation bar, locate the new Azure Function and select it.
   
   
   
   
7. Navigate to Functions -> GoogleSpaceFactsFunction.
   
   
   
   
8. Click Get function URL
   
   
   
   
9. Copy the URL and save it for use. The Dialogflow Web Fullfillment configuration will be updated to point to this location. 
   
   
   
   The fully-qualified URL includes an API key which is passed on the query string and looks similar to:
   
   https://googledemospacefactsXXXXXXXX.azurewebsites.net/api/GoogleSpaceFactsFunction?code={APIKEY}
   
   
10. Navigate to the Google Actions console and select Actions in the left navigation bar.
    
    
11. Navigate to Dialogflow by selecting Add Action -> Build.
    
    
12. Select Fulfillment in the left navigation bar. Apply the URL copied from the Azure Function in step 9 to the URL entry in the Webhook. Scroll to the bottom of the page and click Save.
    
    
13. Return to the Action Console by clicking See how it works in Google Assistant. Selecting this applied the updated Dialogflow configuration to the Action Console. If you were to return to the Action Console in the browser without performing this step, the Action would still be pointing to the ngrok tunnel endpoint.
    
    
    
    
14. In the Action Console, select Test -> Simulator.
    
    
15. Enter "Talk to {Display Name}" in the test simulator. Use the Display Name entered when you created the Action. This is now invoking the deployed Azure Function.
    

Summary

 

Inspired by a tutorial originally designed for Alexa, developed as a node.js Lambda, and deployed to Amazon Web Services, the Space Facts sample is ported to a Google Action, C#, and Azure.

 

This article demonstrates that an Azure Function can service Dialogflow Webhook requests. It also demonstrates how to use ngrok to test and debug requests originated from Google Home devices. This technique applies to developing any service that supports backend messaging, like Alexa and Twilio. 

 

This is just an introduction. There's plenty more to explore, like how to support conversation, answer re-prompt requests, store user state between sessions, use MP3 audio files, return images in card responses, etc. The attached source includes a working Azure Function and has been deployed to Azure and tested using Dialogflow and a Google Action. It is also maintained on GitHub at Whetstone.Alexa.AdventureSample.