Communication Patterns For Distributed Applications

This blog provides information about various communication patterns which we can use in out modern web applications.

Introduction

Today, most cloud-based microservices applications use various communication patterns to enable communication between applications and services. In a distributed applications scenario, allowing applications to connect with other services and applications is essential. Most of the services are now exposed as REST endpoints so that one application can make an http request to the REST endpoint, and the endpoint can send the response back to the consumer application.

Applications can also use a message-based solution to enable communications with other services. A publisher/sender application can send messages to a queue, and one or more subscriber/receiver applications can consume that messages and process it. Applications can also use event-based solutions to build real-time messaging using events and event ingestion services. Publisher applications/services generate events sent to the event ingestion systems. Subscriber applications and services can then consume those events from the ingestion systems.

Http (REST) based communication

This is one of the commonly used communication methods in applications. A service can be exposed as an HTTP endpoint, and one or more clients can send requests to that HTTP endpoint. The REST service accepts the request and sends the response to the client.

The following are the features of REST-based communications.

• The sender and receiver must be online. If the receiver is offline, it sends an error response to the client.
• One-to-one communication.
• Synchronous by default. Optionally, methods can be called asynchronous methods also.
• Sender always gets a response from the receiver. (Success response or an error response).
• The sender knows how the request will be processed and what kind of response can be expected.
• Communication is near real-time.
• We can use an API Gateway to do advanced request-response processing such as SSL offloading, authentication, routing, caching, etc.

Message-based communication

In message-based communication, we use an intermediary component called Message Broker(e.g., RabbitMQ, Azure Service Bus). They commonly follow the AMQP (Advanced Message Queuing Protocol) to provide interoperability between all messaging middleware. When using messaging, applications communicate with each other by sending and receiving messages. This communication is asynchronous.

If a client wants to request a service, it sends a message to the queue, which is then received by the service. If the service needs to reply, it sends a different message to the client. In message-based communication, the client does not expect a reply immediately, so there won't be any response to a request. The application that sends the message is called a Publisher, and the application/service that receives the message is called a Subscriber.

Optionally, you can enable one-to-many communications using topics and subscriptions. Messages can be published to a topic, then forwarded to a virtual subscription queue based on the filter conditions. Subscribers can subscribe only to interested messages from the topic.

The following are the features of message-based communication,

• Asynchronous communication.
• The receiver can be offline. When it comes online, it can consume messages from the queue.
• Typically, no response is expected from the consumer. Consumers can use a reply queue to send responses with correlation IDs if required.
• Not real-time communication.
• The sender can expect how the consumer will process the messages.
• Messages can be sent to all consumers using fanout fashion or to single receivers directly, or only to a group of receivers using a topic.

Event-driven architecture In distributed microservices-based applications, you may need to notify the changes in one service or data to one or more other services. It should be real-time communication, but it should be asynchronous too. The sender (event generator) needs to inform other components or services about the latest changes without expecting a response. REST-based communication is real-time but not asynchronous by default. The client needs to wait for the response from the service. Also, it is one-to-one communication. You may need multiple HTTP requests to notify more than one receiver. A messaging system can be asynchronous but not real-time.

Event-based communications

The event-driven architecture pattern is a popular distributed asynchronous architecture pattern used to produce highly scalable applications. An event is an object that represents what happened in the event source. Its event source generates a stream of events. An event consumer is a service or application that consumes and reacts to events. An event source application uses topics to publish events to the event ingestion system. A topic is an endpoint for publishing an event. It is an endpoint to consume events. Event consumers use event subscriptions to filter and consume events.

An event-driven architecture can use a pub/sub model or an event stream model.

• Pub/sub- When an event is published to the messaging infrastructure, it delivers to each subscriber who has subscribed to it. After the message is delivered, it cannot redeliver the same message. New subscribers will not be able to see previously published messages.
• Event streaming- In the event stream model, events are written to a log, ensuring ordering and durability. Client applications can read from the stream. Clients who joined later can also read messages from the stream.

Event-driven architecture has the following features,

• Real-time processing of requests.
• Asynchronous processing.
• The receiver can be offline.
• One-to-many communication.
• The sender is unaware of how the subscriber (receiver) will handle the event.