Microservices and Business Transactions

Blog: Imixs Workflow Blog

Microservices become the new disruptive technology for software development in it’s traditional way. It is the architectural style which brings up a new way to build software systems. The Microservice Architecture evolves very fast and in deed has a lot of success.

But there are also some disadvantages in Microservices. One of the critical parts is the rising complexity within a microservice architecture. It is a problem which is often overlooked in the beginning when all this euphoria lies within a greenfield project. And the problem is often denied for a long time by its advocates. But why does this happen?

Lets take a short look on some lines of code out from the world of outdated hated monolithic software architecture:

try {
    while (orderService.hasMoreOrders()) {
        order=orderService.next();
        invoice=invoiceService.createInvoice(order);
        if (customerService.approveCreditLimit(invoice) {
            messageService.sentInvoice(invoice);
        } else {
           // roll-back transaction
           ....
        }
    }
} catch (OrderException e) {
    messageService.informSalesManager(order);
    loggingService.handleException(e);
}

The example shows a typical code block of business logic. The logic ensures that a new order will not exceed the customer’s credit limit. It’s easy to understand, transactional and there’s no reason to demonize monolithic software. In this short example, we tie the business logic to five different services. All service calls run in the same transaction and thus the data consistency is guaranteed even in case of an exception. Doing the same thing with microservices can become very hard.

The Saga Pattern

The reason why we despair in a microservice architecture with seemingly small problems is because we are trying to transfer code one to one into the new world. But now the business logic, which was tied together in a monolithic software, must be decoupled in a microservice architecture.

The way to do this is called the Microservice Saga Pattern:

A saga is a sequence of local transactions. Each transaction is encapsulated in a service. If a local transaction fails because it violates a business rules then the saga executes a series of compensating transactions that undo the changes that were made by the preceding local transactions.

You can find more information about the Saga Pattern here.

Of course, to implement the Saga Pattern is also a challenge. We need to deal with external service calls, events and compensation strategies. In general there are two ways to coordinate sagas:

The Choreography

In the choreography style of a Saga Pattern each local transaction publishes domain events that trigger local transactions in other services. It is usually the natural approach for the problem. The disadvantage of this method lies in the increasing coupling of the services, since often events have to be managed from a foreign domain.

If a business transaction is very short, a Choreography can be a good solution.

The Orchestration

Implementing a Saga Pattern with the orchestration style is a different approach where an orchestrator tells the participants what local transactions to execute. The orchestrator is also called the Saga Coordinator. This is were the business logic is placed. And this is a little bit similar to our code block at the beginning. A Saga Coordinator calls services based on a declarative business logic and also handles the compensation in case of exceptions.

Describing Business Logic with BPMN

I think the main problem with long-running business transactions within a microservice architecture is the lack of description. In the code example above, this was not necessary because the code block was compact and easy to read. And as typical for monolithic application architecture it was on one place. In the world of microservices we do not have this one place in code.
Thus, we first need a way to describe and document the business logic independent from our services.

The ‘Business Process Model and Notation‘ – BPMN 2.0 – is the common standard to describe a business process. BPMN was initially designed to describe a business process without all the technical details of a software system. As an result, a BPMN diagram is easy to understand and a good starting point to talk about a business process with technician as also with management people.

Within BPMN 2.0 we can extend a model description with additional custom information which is useful for process execution as well for the description of a Saga transaction.

The following example shows how our business case can be modeled with BPMN 2.0:

This example model describes the process flow between the order-, invoice- and customer service. The rectangles show the states our process can take and the circles indicate the events that can occur during the transaction. The triangles symbolize the so-called Signal Events and describe the service calls.

As you can see in the diagram, the service calls of the Invoice-Service and the Customer-Service also provides a compensation in case the credit limit exceeded. Depending on the result of a service call the status of the business transaction changes. In this way we can see and easily understand what happens within our business transaction.

So the next question is how we can implement a Saga Transaction?

How to Implement Saga Transactions?

To execute the business transaction described by our BPMN model we need to implement a service which is able to understand and execute a BPMN 2.0 model. As BPMN 2.0 is a standard XML format we can use a workflow engine which is able to read a BPMN model. In this way the workflow engine becomes the Saga Coordinator which is responsible to call the various services. A major advantage of using a workflow engine is that the orchestration itself can be implemented as a microservice. This increases the flexibility and the service can be reused regardless of the use case for different business transactions. And after all this pattern solves the problem of tight coupling of services.

Saga Transactions with Imixs-Workflow

In the following section I will explain how we at Imixs-Workflow solve the problem of Saga Transactions. Imixs-Workflow is an open source workflow engine based on the BPMN 2.0 standard. Imixs-Workflow is an event-driven workflow engine. This means that the engine is triggered by an event and responsible to persist the status and the business data during the process runtime. This fits perfectly into the concept of the Microservice Saga Pattern.

The process itself is started by an initial event which creates and persists a new process instance. The workflow engine will now wait until a new event is triggered by an external Rest API call or schedules signal events automatically.

Adapter Classes

Signal Events can be bound to a so-called Adapter Class. An Adapter implements a service call by a simple Rest API request or by sending a message to a message broker. Once an Adapter is implemented it can easily reused in different situations within the same model.

Business Data

Each process instance controlled by the Imixs-Workflow engine can contain different kind of business data. This data can consist of initial business information such as the order number and the order total, or contain data from previous service calls – such as the invoice number or an error message. Since the data is automatically persisted within the process instance, it can be used for later service calls. This eliminates the need for a caching mechanism.

Java Enterprise, Microprofile and Docker

Imixs-Workflow is based on the Java Enterprise specification or in the future on Jakarta EE. Together with the Eclipse Microprofile this offers a powerful tool-set to build, extend and run this framework. We already have implemented Imixs-Workflow as a microservice to be run in a micorservice architecture. And we build also Docker images to start the engine in a container based environment. Imixs Workflow is fully open source and I invite you to participate in it.

I will show more examples how to adapt the Saga Pattern soon in my Blog. If you have any questions or ideas just comment or join the Imixs-Microservice project on Github.