BPM with Apache Kafka

Blog: Imixs Workflow Blog

The Imixs-Workflow project supports now a native integration Adapter for Apache Kafka. With this feature asynchronous messages can be handled within a complex business process.

Apache Kafka is a distributed streaming platform. This means you can publish and subscribe to streams of records, similar to a message queue or enterprise messaging system. Apache Kafka store streams of records in a fault-tolerant durable way and process streams of records as they occur.
Kafka is generally used for two broad classes of applications:

Building real-time streaming data pipelines that reliably get data between systems or applications
Building real-time streaming applications that transform or react to the streams of data

The Imixs-Kafka Adapter integrates Kafka in a native way and allows the producing and consuming of Imixs-Workflow Messages.

Workflow Message Autowire

With Imixs-Kafka you can easily setup a scenario where Workflow Messages are generated automatically during the processing life-cycle. With the Autowire-Function new process instances are send into a Kafka Message Queue so that any consumer interested in workflow events can consume the message and react in various ways.

The Adapter filters Workflow events by the Model Version number so you can control which kind of workflows are send into a message queue.

Workflow Messages based on Business Logic

Another way to send Workflow Messages into a Kafka queue is the Imixs-Adapter Class. This implementation is based on the Imixs-Adapter concept and allows a fine grained modeling of a asynchronous service integration. The Imixs-Kafka Adapter can configured directly in a BPMN 2.0 Model.

This allows you to setup complex workflow scenarios with a asynchronous communication.

Sending Workflow Events With Kafka

The other way to integrate Imixs-Workflow into a Kafka infrastructure is to send Workflow Messages to a Kafka queue to be processed by the Imixs-Workflow Instance. In this way a client can sends a Process Instance to a predefined Message queue.

Imixs-Workflow will automatically consume messages for a predefined topic and process the workflow data. In this way messages can be used to trigger the event-based Imixs-Workflow engine. The following example shows how a client can send a workflow event for a named topic:

String json_workitem = "{"item":[ "
 + " {"name":"type","value":{"@type":"xs:string","$":"workitem"}}, "
 + " {"name":"$modelversion","value":{"@type":"xs:string","$":"1.0.1"}}, "
 + " {"name":"$taskid","value":{"@type":"xs:int","$":"1000"}}, "
 + " {"name":"$eventid","value":{"@type":"xs:int","$":"10"}}, "
 + " {"name":"txtname","value":{"@type":"xs:string","$":"test-json"}}"
 + " ]}";
ProducerRecord<Long, String> record = new ProducerRecord<Long, String>("IN-1.0.1", json_workitem);

Microservices with Kafka an Imixs-Workflow

The Imixs-Kafka Adapter is a powerful adapter to integrate Imixs-Workflow in a Microservice Infrastructure based on Publish-Subscribe Messaging Queues. With Apache Kafka a fault-tolerant and scalable messaging platform can be adapted to Saga Transactions based on a BPMN 2.0 model.

If you have any questions or your need help to integrate Imixs-Workflow into your Microservice Architecture you can join the project on Github.