Blog Posts Case Management Process Management Process Modeling

DMN TCK Turns Two

Blog: Collaborative Planning & Social Business

Attending bpmNEXT conference this week.  It was here, two years ago, that we decided to start the DMN TCK effort.  How far have we come?

The original idea was to simply to back up the written spec with some real running code.  The specification can’t possibly express the full detail necessary for an actual implementation.  Anyone faced with implementing DMN would have to make thousands of small design decisions between, say, behavior X or behavior Y.  Most of those are arbitrary in the sense that the a user could use behavior of either of X or Y, but what matters is that the product is consistent.  Moving from product to product requires that all implementations make the same choices at those levels.  The specification is, by its nature, ambiguous on these many small points.

After the first couple of months we abandoned the idea of creating a reference implementation, and pivoted successfully to the idea of providing a framework for publishing test cases which any successful implementation could use to verify and validate their execution.  The tests would be files constructed in XML: the DMN model in the standard format, and the test data in another XML which we designed containing input values to submit to the decision model, and output values to compare the output to.  To use the tests one needs to write a small “runner” which is code that reads the files, and runs the test.

All of the DMN examples and all the test files are available on GitHub and usable for free with a Creative Commons license.

A Example Test

Here is the XML representation of a DMN decision

This decision simply calculates the yearly amount by multiplying the monthly amount by 12.  This tests the ability for an engine to parse that FEEL expression, and correctly evaluate it.   And here is the XML test file:

As you can see, it puts 10,000 as the monthly amount, and 120,000 as the yearly result to compare to.  Of course the DMN models can be considerably more complex, but the test files generally remains simple because it always is simple a list of inputs and outputs.  It really is that easy to create tests, and because it is easy we hope for widespread adoption.

The results are then collected and listed in this Results Site.

Collecting Tests

Once the basic pieces were in place, we started collecting tests.  The first few tests were models that Bruce Silver had developed while writing his book DMN Method and Style.  The next chunk of tests came from Red Hat during the testing and finalization of their FEEL evaluator.  Camunda added a number of tests.  Finally, in July of 2017 Actico entered the group, and converted hundreds of their internal tests into the standard form that could be used by everyone bringing the total number of tests to 588.

We welcome more tests.  Now that the framework is complete and functional, it is easy to add a DMN model, and a set of input / output values, and that constitutes a new test that anyone can run.  Anyone can contribute who can make a GitHub style pull request.  The team members will review the test, and assure that it runs on at least one existing engine, and then include it in the official set.

Improving the DMN Specification

While it is not the goal for TCK members to discuss modifications to the DMN spec itself, it is necessary to discuss how to implement parts of the DMN spec, and on many occasions we have run into parts that had been interpreted differently by different vendors, or even parts which were impossible to implement.  In many cases the disagreement could be resolved by a careful re-reading of the spec.  In some cases the spec was ambiguous, and so a proposed clarification was submitted to the RTF group maintaining the spec.

For each of these we raised an issue with the RTF to get the spec changed or clarified.

Support

The current effort has received a lot of support from these organizations:

There are seven vendors that have run the tests and contributed results to the results site.  However, the DM Community lists 18 vendors who claim DMN support.  Why are these lists different?  Why have so many vendors not run the tests?  What assurance do you, as a consumer, have that those vendors actually run DMN?  These are questions to ask those vendors directly.

What is Left to Do?

We are waiting for DMN 1.2 to be released so that we can then add some tests that are not compatible with 1.1, and we expect they will be supported by the new release.  When that happens we will create a number of tests around 1.2.

We have not really explored error cases beyond one or two.  Clearly there are many more error cases to check for.  We might also test DMN models with invalid FEEL expressions to make sure that engines react consistently to those.

The main focus, however, will be adding real-world DMN models which have been developed for actual operating business applications.  Those models test not only the execution of the engine in cases that are realistic, but also the suitability of DMN in general to real world problems.

Resources

Leave a Comment

Get the BPI Web Feed

Using the HTML code below, you can display this Business Process Incubator page content with the current filter and sorting inside your web site for FREE.

Copy/Paste this code in your website html code:

<iframe src="https://www.businessprocessincubator.com/content/dmn-tck-turns-two/?wpfpaction=add&postid=76118&feed=html" frameborder="0" scrolling="auto" width="100%" height="700">

Customizing your BPI Web Feed

You can click on the Get the BPI Web Feed link on any of our page to create the best possible feed for your site. Here are a few tips to customize your BPI Web Feed.

Customizing the Content Filter
On any page, you can add filter criteria using the MORE FILTERS interface:

Customizing the Content Filter

Customizing the Content Sorting
Clicking on the sorting options will also change the way your BPI Web Feed will be ordered on your site:

Get the BPI Web Feed

Some integration examples

BPMN.org

XPDL.org

×