Introduction To Saga Pattern

Microservice architecture is an architectural approach where the entire application is build through a collection of individual services. There services make the application

Highly maintainable and loosely coupled
Independently deployable
Highly Testable
Increased scalability

But how does the database looks like when you are running a collection of services? In particular, how are transactions handled in microservices? In this series of posts, we will look into the problems and the solutions for implementing transactions in a microservice architecture.

Transactions in a Monolith

When you think about a Monolith application, most often than not you are dealing with a single database. Different modules/sub-sections in the application usually tend to interact with the same database. The closed nature of the monolith application would enable the interacting modules with inter-dependencies to control the transactions and making it easier to follow ACID properties.

So what exactly is a transaction? A transaction is a sequence of often multiple operations performed on a database, executed as a single unit of work, such that the entire sequence of operations are either be completed wholly or not at all. Each transaction is supposed to follow the ACID (Atomicity, Consistency, Isolation, Durability) Properties to ensure accuracy and state of data.

The Atomicity ensures a transaction is either fully completed or not done at all. Consistency ensures all local constraints ensuring the integrity of data after transaction is completed. Isolation ensures concurrent transactions produces the same results as when they are executed sequentially, and lastly, Durability ensures committed transactions persist their changes even in event of a system failure.

For example, consider the following scenario. In an online e-commerce application, a typical incoming order might include checking the Customer's credit limit and updating his/her purchase history. This might involve sequential updation of multiple tables, each of which has to be rolled back in case of failure. This is quite simple under a monolith which deals with a single database since the scope of transaction is within the same database. However, things change when considering Microservices which often require complex interservice communications.

Database Per Service

Microservices typically follow the Database Per Service design. Each of the databases are exposed only via the API of the particular service and is not accessible to other services directly. This is done to ensure each of the services are loosely coupled and can be independently developed, deployed, and scaled.

Additionally, each of the services could have different data storage requirements, and could opt for optimized database solutions needed for the particular service. Each of the databases can be independently scaled.

Each service, in this case, has its transactions limited to its own database. So how does one manage transactions when the scope of the business operation is not limited to a single database, instead span across multiple services and their databases? Transactions within each individual transaction are ACID, but how does one ensure the cross service transaction follows ACID properties.

One of the common pattern for implementing transaction across different services is the Saga Pattern.

Saga Pattern

Saga allows us to implement business transactions that spans across multiple services. Saga is a series of local transactions, which are executed sequentially.

On successful completion of each local transaction, an event is raised which would trigger the next mini-transaction in the series. On other hand, if the local transaction fails for any reason, it would trigger a series of compensating transactions that undo any of the preceding transactions.

Coordination in a Saga can be done in two implementation approaches

Orchestration
Choreography

Let us look at each of the coordination methods in detail now.

Orchestration

In this approach, a central coordinator (Saga execution controller or SEC) microservices coordinates the participants(services) involved in the transactions and instructs them to execute local transactions based on events and interpretation of state of each subtasks. It tracks all the events in the transaction and handles failure recovery with compensating transactions.

Orchestrator is comparatively easier to build compared to choreography and is suitable when there is control over all the participants in the process. Additionally, due to synchronous processing, the flow of applications is coordinated effectively.

However, it exposes the Saga Execution Controller or SEC as an additional point of failure as it coordinates the workflow and requires additional design complexity due to dependencies between services and coordinator.

Choreography

Choreography, on the other hand, does not require additional co-ordinating controllers and relies on asynchronous messaging pattern to coordinate business operations.

Each microservice participating in the transaction would emit an event (message in the queue), which would be processed by the next microservice(s). Each microservice process its own local transaction and responds to the message queue with a success or failure message. In case of success, the message is then forwarded to services that are interested in the message. In case of failure, the queue could retry the failed operation or execute a compensating transaction.

The lack of point to point communication between the microservices reduces coupling between the microservices. Also, there is no single point of failure due to lack of a centralized coordinator. Choreography is suitable when it is expected to add or remove services, and can be done with minimal changes without disrupting the entire application.

On other hand, the business process is spread out and is more difficult to maintain. Each service is independent but needs to be aware of the events to react to. This arises the complexity of development.

Conclusion

One might wonder why not use a simple monolith application instead of a microservice architecture. Well, that is a topic of its own to discuss and the answer is not simple either. Just as Microservice is not an answer to every problem, Monolith too has its own limitations and benefits. The choice would finally boil down to the application needs. However, I would leave out that discussion out of the scope of this article as it focuses on Saga Pattern.

In this next part of the article, we will look into the implementation of the Saga Pattern using Mass Transist Library.