An Overview about NATS JetStream Queue Mechanism

Introduction

I hope everyone is doing well. Today we are going to look at the topic called NATS queue mechanism. It is a lightweight and high-performance messaging system that acts as a distributed messaging queue. In the current digital era, we try to de-couple applications from one another, so it is supposed to adapt to plug it in another version or another application easily. 

What is NATS JetStream, and what makes us consider this?

NATS, or the "NATS Messaging System," is an open-source messaging system designed for simplicity, performance, and scalability. It was originally developed in the Go programming language (hence the name GNATS, which stands for Golang NATS), but it now has client libraries for various programming languages, making it versatile and accessible for developers working in different ecosystems. NATS has an active open-source community, providing support, documentation, and a range of client libraries for different programming languages.

Use Case for NATS and NATS Jetstream

• In the case of both the producer and consumer being alive, the system does not require any persistent layer. To fulfill this scenario, NATS is far enough.
• If the system requires a broker to keep or persist data, then NATS won't be sufficient. We have to go for the JetStream option.

NATS JetStream is an extension of NATS core which is designed for use cases that require additional features such as guaranteed message delivery, durability, and persistence.

Features of NATS JetStream

• Guaranteed message delivery, durability, and the ability to replay messages are essential.
• It allows messages to be retained even after being delivered to subscribers. This enables subscribers to catch up on missed messages, providing a form of message persistence.
• Subscribers can acknowledge that they have received and processed a message, and this information can be used for redelivery in case of failures.
• It has the concept of consumer groups, allowing multiple subscribers to work together to process messages from a stream. This facilitates load balancing and fault tolerance.

Comparison between other Message Queue Platforms
 

Queuing System	NATS jetstream	Rabbit MQ	Kafka	SQS
Open Source
Message replay				No built-in capability. However, we can do this using DQL and other options
Event Streaming
Message/event ordering based	Sequence #	Order #	Partition	message grouping/message group Id
Setup Complexity	Easy	Easy	Complex	Easy
Delivery guarantee (exactly once)	It uses acknowledgments to acknowledge the receipt of a message	It supports a feature called publisher confirms.	enable.idempotence = true	using MessageDeduplicationId concept
Scheduled and delayed delivery			Kafka, by default, does not provide native support for delayed message delivery. using these options, we can implement the same Message Timestamp Topic Partitions Intermediate Topic
Horizontally scalable
Cloud Availability
On-Premises
Filters	subject filter wildcard pattern	routing mechanism wildcard pattern	topics and partition	filter and route messages based on attributes

NATS Topology

• Super Cluster
  • Superclusters are made up of NATS clusters that are connected by gateway connections.
• Cluster

  • NATS Clusters are made up of any number of NATS servers that communicate with each other, service, or any number of clients.

  • NATS Cluster provides great fault tolerance and scalability.

• Standalone Server 
  • Single NATS server with clients connected to it. With the ability to process millions of messages a second per server, you’ll find unparalleled efficiency with the NATS server.
• Leaf Node 
  • A leaf node is a single NATS server extended out from a cluster or remote server. Leaf nodes extend clusters via a hub and spoke topology. Leaf nodes allow you to bridge separate security domains.

Persistence Layer options 

The storage mechanism used by NATS JetStream can vary based on the configuration. JetStream supports different storage backends, and you can configure it to use either file-based storage or other storage solutions like SQL databases.

• File Storage: NATS JetStream can be configured to store messages in a file-based storage system. This provides durability and allows messages to survive system restarts.
• SQL Storage: JetStream also supports using SQL databases as a storage backend. This allows you to leverage existing SQL databases for persistence.
• In-memory: While JetStream is designed for persistence, it also optimizes for performance. In some configurations, it may use an in-memory component to enhance message delivery speed.

Implementation steps of NATS Jetstream

Let's look at the implementation steps in another article.

• https://www.c-sharpcorner.com/article/nats-jetstream-implementation-steps/

Conclusion

If you require features like guaranteed delivery and persistence, especially in scenarios where messages need to be retained over time, NATS JetStream would be a more suitable choice. I hope you enjoy reading this article.