Blog Posts Process Management

Microservices and Kafka: A Perfect Match for Enabling Event-driven Architecture and Supercharging Integration

Blog: The Tibco Blog

Reading Time: 2 minutes

In 2003, Gartner defined event-driven architecture (EDA) as the industry best practice for long-running processes. EDA made it possible to design asynchronous processes where an event is posted and the one who posts it is disconnected from the processing of the event — otherwise known as “fire and forget”. Since its conception, more companies are turning to EDA for their projects. However, the rise of cloud native has introduced larger workloads and more advanced capabilities, which required a new solution—microservices and Apache Kafka. 

Today, more organizations are digitally transforming and becoming more cloud-native. With that, SOA has started to hit its limit. Applications have grown to become monolithic, too difficult to maintain, and have limited DevOps capabilities. Every new feature developed requires the entire application to be deployed, often disrupting the current service and reducing SLAs. Scaling is a nightmare, only allowing vertical scalability by adding more memory, more CPUs, or both. 

On the other hand, events represent the way the world works; an IoT sensor measure, a customer check-in, a purchase transaction, a passenger check-in is a “change of state” in a business. An event is an observation about the world that is immutable. It’s not just simply a request, which is why SOA isn’t the best choice when designing modern architectures that require a move to the cloud.

EDAs allow users to create components that react to events as they arrive, it enables subscription and parallel processing models, and reduces the physical and temporal coupling between request-response services. 

With the introduction of microservices in 2011, the realization of adopting a new architectural style became clear. Microservices break down monolithic applications into core functions to be deployed independently, allowing the adoption of DevOps practices, and scale horizontally when needed. Additionally, microservices are loosely coupled and operate in complete isolation so if one piece fails, the whole application doesn’t go down, unlike the monolithic application model.

By combining a high-performing event broker such as Apache Kafka together with microservices, it becomes clear that event-driven architectures are the best fit when moving to the cloud. Because microservices can be deployed in containers, they can be scaled out or in when the load increases or decreases. Kafka can be used as the underlying event broker, enabling horizontal scaling to send concurrent streams from thousands of producers to thousands of consumers or run multiple brokers in a cluster.

By combining a high-performing event broker such as Apache Kafka together with microservices, it becomes clear that event-driven architectures are the best fit when moving to the cloud.
Click To Tweet

Download this whitepaper and watch this on-demand webinar to learn more about how microservices and Apache Kafka can work together to supercharge your integration.  

Leave a Comment

Get the BPI Web Feed

Using the HTML code below, you can display this Business Process Incubator page content with the current filter and sorting inside your web site for FREE.

Copy/Paste this code in your website html code:

<iframe src="" frameborder="0" scrolling="auto" width="100%" height="700">

Customizing your BPI Web Feed

You can click on the Get the BPI Web Feed link on any of our page to create the best possible feed for your site. Here are a few tips to customize your BPI Web Feed.

Customizing the Content Filter
On any page, you can add filter criteria using the MORE FILTERS interface:

Customizing the Content Filter

Customizing the Content Sorting
Clicking on the sorting options will also change the way your BPI Web Feed will be ordered on your site:

Get the BPI Web Feed

Some integration examples