Blog Posts BPMN DMN

Intelligent Kafka message routing using Drools DMN Engine and Apache Camel

Blog: Drools & jBPM Blog

In this post I want to share an interesting use-case of Healthcare message routing, which we implemented using the Drools DMN Engine and Apache Camel, in order to route and dispatch Patient’s Admission-Discharge-Transfer message types to the required Kafka topics and therefore queuing the message to the appropriate sub-system. I believe it is both a very pragmatic and interesting combination of technologies, while it also shows a blueprint and generalised pattern which can be easily replicated to other domains, in order to achieve intelligent message routing for Apache Kafka for many other use-cases.

You can see the tech demo in action in this video and further below for additional details:

The use-case for this tech demo is about specific types of message in the Healthcare domain, which contains information about the Patient’s Admission-Discharge-Transfer (ADT); these types of messages, depending on a set of rules, need to be dispatched to the appropriate Kafka topic(s). This way, further systems and applications downstream can connect to these queues and consume only the relevant messages of interest.

EIP diagram for Intelligent Healthcare message routing

Rules are formalised by the domain expert, in this case the Healthcare professional, who usually is not a software developer, but is knowledgeable and interested in the data contained in the message itself. This makes it a perfect fit for DMN, as the rules can be easily encoded using a DMN Decision Table, so that the decision about the destination queue names can easily be represented in a graphical format –one of the many key advantages of the DMN Standard.

In the example screenshot, the domain expert is interested with the first rule in all ADT messages coming only from the application called “MMS”: these needs to be dispatched to the queue named “MMSAllADT” on Kafka. A second rule prescribe that ADT messages, again only from the application called “MMS”, but pertaining only to a Patient’s discharge (reported as “A03” on HL7), shall also be dispatched to topic “MMSDischarges” on Kafka. For the purpose of simplicity of the demo, the table contains only 2 criteria; naturally this table can be easily extended to accomodate all the specific rules required, as the demonstration video also shows.

The Enterprise Integration Pattern diagram can be revisited, now annotated with the actual technologies used:

For the purpose of this tech demo, the incoming message format is exchanged using the HL7 v2 standard, and provided to a REST Endpoints, which may also serve as a CDS hook. Naturally this could be extended to use more modern standards in the same healthcare domain, such as HL7 FHIR.

It is important to reiterate this “Intelligent message routing” blueprint is very generic, therefore can be analogously architected on top of other business domains, such as FSI, IoT, etc.

Another advantage of this blueprint is the separation of concerns between the business domain logic and the integration code; that is the separation between the rules formalised graphically by means of a DMN Decision Table to decide on which topic name the message shall be queued into, and the actual Apache Camel code implementing the integration between the different systems. This way, the business expert can focus on just and only the decision to be modelled using DMN, while the software developer can offer a very extensible and robust intelligent routing capability.

Taking a little look under the hood, we highlight how the software developer using the Camel DSL only needs to write a minimal amount of code:

I have highlighted visually where the Camel DSL integrates the Drools DMN Engine for the evaluation of the necessary business logic for the rules, in order to decide which Kafka topic(s) the message needs to be queued into, or eventually forwarded to a catch-all queue. This allow to maintain the set of rules and decisions using just and only the DMN model, separately from integration code.

In the video, three scenarios are demonstrated:


In this technical demo, we have seen how we can easily integrate Drools DMN Engine and Apache Camel, to achieve Intelligent message routing on top of Apache Kafka. Specifically we have seen how to formalise the rules for the decision of the appropriate dispatch queue using DMN decision tables, so we can manage separately the business requirements from the actual integration code.

What do you think of this use-case and this demo? Don’t hesitate to let us know in the comments below!

The post Intelligent Kafka message routing using Drools DMN Engine and Apache Camel appeared first on KIE Community.

Leave a Comment

Get the BPI Web Feed

Using the HTML code below, you can display this Business Process Incubator page content with the current filter and sorting inside your web site for FREE.

Copy/Paste this code in your website html code:

<iframe src="" frameborder="0" scrolling="auto" width="100%" height="700">

Customizing your BPI Web Feed

You can click on the Get the BPI Web Feed link on any of our page to create the best possible feed for your site. Here are a few tips to customize your BPI Web Feed.

Customizing the Content Filter
On any page, you can add filter criteria using the MORE FILTERS interface:

Customizing the Content Filter

Customizing the Content Sorting
Clicking on the sorting options will also change the way your BPI Web Feed will be ordered on your site:

Get the BPI Web Feed

Some integration examples