Post
Ask Question

Service Communication: Integration of gRPC using Protocol Buffers

Introduction

gRPC, also known as Remote Procedure Call, was created by Google as an open-source framework for remote procedure calls. It facilitates effective communication between distributed systems, enabling client and server applications to communicate seamlessly, regardless of the programming languages or platforms they are based on.

Features of gRPC

  • Efficiency: gRPC utilizes HTTP/2, incorporating features like multiplexing, header compression, and server push. This leads to decreased latency, enhanced throughput, and more effective network utilization in comparison to traditional HTTP/1.x-based communication protocols.
  • Language-agnostic: gRPC supports a variety of programming languages, such as C++, Java, Python, Go, C#, and others. This language-agnostic approach enables the development of distributed systems with components written in different languages that can seamlessly communicate with each other.
  • Interoperability: gRPC employs Protocol Buffers (protobuf) as its interface definition language (IDL). Protobuf provides a language-neutral and platform-neutral method to define data structure and service methods, facilitating interoperability between diverse systems and simplifying integration with existing infrastructure.
  • Strongly Typed Contracts: By using Protobuf, you can establish strongly typed contracts for your service APIs. This ensures mutual agreement between the client and server regarding the data structure being exchanged, minimizing errors and streamlining integration.
  • Support for Streaming: gRPC supports four types of RPCs: unary, server streaming, client streaming, and bidirectional streaming. This capability allows for the creation of intricate communication patterns where clients and servers can exchange multiple messages within a single RPC call.
  • Modern Ecosystem: As part of the contemporary microservices ecosystem, gRPC seamlessly integrates with technologies like Kubernetes, Istio, Envoy, and more. It offers essential features such as service discovery, load balancing, and health checking, which are crucial for constructing scalable and resilient distributed systems.

The gRPC-based system typically consists of two main components: the client and the server. Let's examine the architecture of each component.

gRPC Schema

Server Architecture

  1. Service Implementation: The server is responsible for implementing the actual functionality of the gRPC service. This entails writing the business logic that handles incoming RPC requests from clients. The server defines one or more gRPC service interfaces, which contain methods that can be remotely invoked by clients.
  2. Service Interface Definition: The service interface is defined using Protocol Buffers (protobuf). It specifies the methods provided by the server and the message types used as input and output parameters for those methods. This interface serves as the contract between the server and its clients.
  3. Code Generation: The service interface definition is compiled using the Protocol Buffers compiler (protoc) along with the gRPC plugin. This compilation process generates server-side stub code in the desired programming language. The generated code acts as the foundation for the server implementation, including classes or structs that handle RPC requests and send responses.
  4. Server Implementation: The generated code is then extended to implement the actual business logic for each RPC method. This involves writing code to process incoming requests, perform necessary computations or data manipulations, and generate appropriate responses.
  5. Server Hosting: The server application runs in a hosting environment, such as a web server like Nginx, or as a standalone service. It listens for incoming gRPC requests on a specified network address and port.
  6. Server Infrastructure: Depending on the application's requirements, the server can be deployed as a single instance or as a cluster of multiple instances behind a load balancer. This deployment strategy ensures scalability and high availability for the system.

Client Architecture

  1. Stub Generation: The client utilizes the Protocol Buffers definition file to generate stub code, which contains methods mirroring those defined in the service interface. This enables the client to call remote methods on the server.
  2. Channel Creation: A gRPC channel is established by the client to connect with the server. This channel serves as a logical connection to the server and manages details like network transport, connection pooling, and load balancing.
  3. Client Implementation: The client application leverages the generated stub code to interact with the server through the gRPC channel. Upon method invocation, the client serializes the request message into binary format and transmits it to the server via the channel. Subsequently, it awaits a response and deserializes the received binary data into the appropriate response message type.
  4. Error Handling and Retry Logic: The client is responsible for managing errors from the server, such as network issues or RPC failures, and may incorporate retry mechanisms to address transient failures.
  5. Client Infrastructure: Similar to the server, the client application can be deployed across various hosting environments based on the application's specific requirements.

gRPC, an open-source remote procedure call (RPC) framework developed by Google, offers several advantages over Windows Communication Foundation (WCF).

  1. Enhanced Performance: gRPC delivers superior performance compared to WCF. By utilizing HTTP/2 as its transport protocol, gRPC supports advanced features like multiplexing, header compression, and server push. This results in reduced latency and improved efficiency, surpassing the capabilities of WCF's reliance on HTTP/1.x.
  2. Language and Platform Interoperability: Designed to be language-agnostic and platform-independent, gRPC supports a wide range of programming languages, including C++, Java, Python, Go, C#, and more. This flexibility simplifies the development of distributed systems where different components are written in different languages.
    gRPC
  3. Strongly Typed Contracts: gRPC employs Protocol Buffers (protobuf) as its interface definition language (IDL). Protobuf provides a strongly typed contract definition, facilitating effective communication between client and server. In contrast, WCF primarily relies on XML-based contracts, which may not offer the same level of efficiency or ease of use.
  4. Comprehensive Streaming Support: gRPC seamlessly supports both unary and streaming RPCs. This means that methods can accept and return multiple messages over a single connection. While WCF also supports streaming, it lacks the seamless integration and comprehensive support found in gRPC.
  5. Integration with Modern Ecosystem: As part of the modern microservices ecosystem, gRPC integrates well with technologies like Kubernetes, Istio, Envoy, and more. It provides essential features such as service discovery, load balancing, and health checking, enabling the development of scalable and resilient distributed systems.
  6. Open Source with Strong Community Support: gRPC is an open-source framework with a thriving community of developers actively contributing to its development and maintenance. This ensures continuous improvement and support for the framework, making it a reliable choice for building robust applications.

In gRPC, there are four types of streaming.

1. Unary RPC

This form of RPC involves the client sending a single request to the server and receiving a single response, similar to traditional synchronous RPC calls.

Protobuf file structure for the above RPC call.

syntax = "proto3";

// Define the service
service MyService {
    // Define the unary RPC method
    rpc MyMethod(RequestMessage) returns (ResponseMessage);
}

// Define the request message type
message RequestMessage {
    // Define the fields of the request message
    string field1 = 1;
    int32 field2 = 2;
    // Add more fields as needed
}

// Define the response message type
message ResponseMessage {
    // Define the fields of the response message
    string result = 1;
    // Add more fields as needed
}

2. Server Streaming RPC

With this type of RPC, the client sends a single request to the server, and in return, the server responds with a stream of messages. The client reads the entire stream until there are no more messages.

Protobuf file presentation for the above call.

syntax = "proto3";

// Define the service
service MyStreamingService {
    // Define the server streaming RPC method
    rpc MyServerStreamingMethod(RequestMessage) returns (stream ResponseMessage);
}

// Define the request message type
message RequestMessage {
    // Define the fields of the request message
    string query = 1;
    // Add more fields as needed
}

// Define the response message type
message ResponseMessage {
    // Define the fields of the response message
    string result = 1;
    // Add more fields as needed
}

3. Client Streaming RPC

In contrast to server streaming, the client sends a stream of messages to the server, and the server responds with a single message. The server processes the entire stream before generating a response.

Protobuf file presentation for the above call.

syntax = "proto3";

// Define the service
service MyClientStreamingService {
    // Define the client streaming RPC method
    rpc MyClientStreamingMethod(stream RequestMessage) returns (ResponseMessage);
}

// Define the request message type
message RequestMessage {
    // Define the fields of the request message
    string data = 1;
    // Add more fields as needed
}

// Define the response message type
message ResponseMessage {
    // Define the fields of the response message
    string result = 1;
    // Add more fields as needed
}

4. Bidirectional Streaming RPC

This RPC type allows both the client and server to independently send a stream of messages. This enables both parties to read and write messages concurrently over the same connection, making it particularly useful for scenarios requiring continuous communication between the client and server.

Protobuf file presentation for the above call.

syntax = "proto3";

// Define the service
service MyStreamingService {
    // Define the bidirectional streaming RPC method
    rpc MyBidirectionalStreamingMethod(stream RequestMessage) returns (stream ResponseMessage);
}

// Define the request message type
message RequestMessage {
    // Define the fields of the request message
    string data = 1;
    // Add more fields as needed
}

// Define the response message type
message ResponseMessage {
    // Define the fields of the response message
    string result = 1;
    // Add more fields as needed
}

What is a Protocol Buffers file?

A Protocol Buffers (protobuf) file is a text file that defines the structure of data messages and, optionally, service interfaces for gRPC-based systems. It uses a language-neutral interface definition language developed by Google for serializing structured data. Protobuf files typically have a .proto extension.

Sample of a protobuf file

syntax = "proto3";

// This defines a gRPC service named MyService. Inside MyService, there's a unary RPC method named MyMethod. This method takes a RequestMessage as input and returns a ResponseMessage.
service MyService {
    // Define the unary RPC method
    rpc MyMethod(RequestMessage) returns (ResponseMessage);
}

// This defines the structure of the request message (RequestMessage) used in the MyMethod RPC method. It has two fields: field1 of type string and field2 of type int32.
message RequestMessage {
    // Define the fields of the request message
    string field1 = 1;
    int32 field2 = 2;
    // Add more fields as needed
}

//This defines the structure of the response message (ResponseMessage) used in the MyMethod RPC method. It has a single field named result of type string with a field number of 1.
message ResponseMessage {
    // Define the fields of the response message
    string result = 1;
    // Add more fields as needed
}

What is a Channel in gRPC?

A channel in gRPC is an abstraction that represents a connection to a gRPC server. It acts as a conduit for communication, enabling clients to send RPC requests to the server and receive responses.

When creating a gRPC channel, you specify the server's address and can also configure various options such as security settings, load balancing policies, and timeouts. Once the channel is established, it takes care of managing the network connections, handling authentication and encryption if needed, and provides a user-friendly API for clients to make RPC calls.

In gRPC, channels are typically created using a channel builder or a channel factory, which abstracts the complexities of channel creation and configuration. This allows clients to focus on making RPC calls without having to worry about the underlying networking details.

Example Project

Step 1. The sample project appears as follows.

Sample project

Step 2. The sample project is illustrated in the following manner.

  • GrpcClientSource: The entirety of the client logic is established within this project.
  • GrpcCommonSource: All files containing proto buffers are defined within this section.
  • GrpcServerSource: The section contains the definition of the logic related to the server.
  • GrpcTestingClient: The project functions as a client.
  • GrpcTestingServer: This project functions in a manner similar to a server.

Step 3. I have segregated the implementation of request and response of protobuf files into distinct files like below.

greet. proto

syntax = "proto3";

import "Protos/HelloRequest.proto";
import "Protos/HelloReply.proto";
option csharp_namespace = "GrpcCommonSource";
package GrpcCommonSource;

// The greeting service definition.
service Greeter {
  // Sends a greeting
  rpc SayHello (HelloRequest) returns (HelloReply);
}

HelloReply.proto

syntax = "proto3";

// The response message containing the greetings.
message HelloReply {
  string message = 1;
}

// HelloRequest.proto
syntax = "proto3";

// The request message containing the user's name.
message HelloRequest {
  string name = 1;
}

Step 4. To start and stop the server

IGrpcServerHelper.cs

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;

namespace GrpcServerSource.Interfaces
{
    public interface IGrpcServerHelper
    {
        void StartServer();
        void StopServer();
        bool CheckServerStatus();
    }
}

GrpcServerHelper.cs

using Grpc.Core;
using Grpc.Core.Interceptors;
using GrpcChatSample.Server;
using GrpcCommonSource;
using GrpcServerSource.Interfaces;
using GrpcServerSource.Services;
using GrpcServerSource.Utilities;
using System.ComponentModel.Composition;
using System.IO;

namespace GrpcServerSource.Implementation
{
    [Export(typeof(IGrpcServerHelper))]
    internal class GrpcServerHelper : IGrpcServerHelper
    {
        #region Fields declaration
        [Import]
        private Logger m_logger = null;
        private Grpc.Core.Server? m_server = null;
        private bool isServerRunning = false;
        [Import]
        private ChatGrpcService? m_service = null;
        [Import]
        private GreetGrpcService? m_greetGrpcservice = null;
        #endregion

        public bool CheckServerStatus()
        {
            return isServerRunning;
        }

        public void StartServer()
        {
            if (!isServerRunning)
            {
                // Locate required files and set true to enable SSL
                var secure = false;

                if (secure)
                {
                    // secure
                    var clientCACert = File.ReadAllText(@"C:\localhost_client.crt");
                    var serverCert = File.ReadAllText(@"C:\localhost_server.crt");
                    var serverKey = File.ReadAllText(@"C:\localhost_serverkey.pem");
                    var keyPair = new KeyCertificatePair(serverCert, serverKey);
                    var credentials = new SslServerCredentials(new[] { keyPair }, clientCACert, SslClientCertificateRequestType.RequestAndRequireAndVerify);

                    // Client authentication is an option. You can remove it as follows if you only need SSL.
                    //var credentials = new SslServerCredentials(new[] { keyPair });

                    m_server = new Grpc.Core.Server
                    {
                        Services =
                        {
                            Chat.BindService(m_service)
                                .Intercept(new ClientIdLogger()) // 2nd
                                .Intercept(new IpAddressAuthenticator()), // 1st
                            Greeter.BindService(m_greetGrpcservice)
                                .Intercept(new ClientIdLogger()) // 2nd
                                .Intercept(new IpAddressAuthenticator()) // 1st
                        },
                        Ports =
                        {
                            new ServerPort("localhost", 50052, credentials)
                        }
                    };
                }
                else
                {
                    // insecure
                    m_server = new Grpc.Core.Server
                    {
                        Services =
                        {
                            Chat.BindService(m_service)
                                .Intercept(new ClientIdLogger()) // 2nd
                                .Intercept(new IpAddressAuthenticator()), // 1st
                            Greeter.BindService(m_greetGrpcservice)
                                .Intercept(new ClientIdLogger()) // 2nd
                                .Intercept(new IpAddressAuthenticator()) // 1st
                        },
                        Ports =
                        {
                            new ServerPort("localhost", 50052, ServerCredentials.Insecure)
                        }
                    };
                }

                m_server.Start();
                m_logger.Info("Started.");
                isServerRunning = true;
            }
        }

        public void StopServer()
        {
            if (m_server != null)
            {
                if (isServerRunning)
                {
                    m_server.ShutdownTask.Wait();
                    isServerRunning = false;
                }
            }
        }
    }
}

Step 5. Client implementation

GreetServiceClient.cs

using Grpc.Core;
using Grpc.Core.Interceptors;
using GrpcCommonSource;
using System.IO;

namespace GrpcClientSource
{
    public class GreetServiceClient : IDisposable
    {
        private readonly Greeter.GreeterClient m_client;
        private readonly Channel? m_channel; // If you create multiple client instances, the Channel should be shared.
        private readonly CancellationTokenSource m_cts = new CancellationTokenSource();
        private bool disposedValue;

        public GreetServiceClient()
        {
            // Locate required files and set true to enable SSL
            var secure = false;

            if (secure)
            {
                // create secure channel
                var serverCACert = File.ReadAllText(@"C:\localhost_server.crt");
                var clientCert = File.ReadAllText(@"C:\localhost_client.crt");
                var clientKey = File.ReadAllText(@"C:\localhost_clientkey.pem");
                var keyPair = new KeyCertificatePair(clientCert, clientKey);
                //var credentials = new SslCredentials(serverCACert, keyPair);

                // Client authentication is an option. You can remove it as follows if you only need SSL.
                var credentials = new SslCredentials(serverCACert);
                m_channel = new Channel("localhost", 50052, credentials);
                m_client = new Greeter.GreeterClient(
                    m_channel
                    .Intercept(new ClientIdInjector()) // 2nd
                    .Intercept(new HeadersInjector())); // 1st
            }
            else
            {
                // create insecure channel
                m_channel = new Channel("localhost", 50052, ChannelCredentials.Insecure);
                m_client = new Greeter.GreeterClient(
                    m_channel
                    .Intercept(new ClientIdInjector()) // 2nd
                    .Intercept(new HeadersInjector())); // 1st
            }
        }

        public HelloReply SayHelloMethod(HelloRequest helloRequest)
        {
            return m_client.SayHello(helloRequest);
        }

        protected virtual void Dispose(bool disposing)
        {
            if (!disposedValue)
            {
                if (disposing)
                {
                    m_cts.Cancel(); // without cancel active calls, ShutdownAsync() never completes...
                    m_channel.ShutdownAsync().Wait();
                }

                disposedValue = true;
            }
        }

        public void Dispose()
        {
            Dispose(disposing: true);
            GC.SuppressFinalize(this);
        }
    }
}

Step 6. Server implementation

GreetGrpcService.cs

using Grpc.Core;
using GrpcCommonSource;
using System.ComponentModel.Composition;

namespace GrpcServerSource.Services
{
    [Export]
    public class GreetGrpcService : Greeter.GreeterBase
    {
        public override Task<HelloReply> SayHello(HelloRequest request, ServerCallContext context)
        {
            return Task.FromResult(new HelloReply
            {
                Message = "Hello " + request.Name
            });
        }
    }
}

Step 7. To execute the project, adhere to the subsequent instructions.

Run GrpcTestingClient and GrpcTestingServer simultaneously like below.

TestingServer

Step 8. Output Appearance

Output Appearence

Repository Path: https://github.com/OmatrixTech/GrpcSampleExample


Similar Articles