A Practitioner’s Review of DMN 1.2
Blog: Lux Magi - Decision Management for Finance
In April 2016, James Taylor and I summarized our experiences of decision modelling in DMN by publishing two wish lists of the features I’d like to see in the next version of the standard. Specifically, I listed new feature suggestions for the way the standard represents decision requirements and decision logic elements.
Now the new version, DMN 1.2, is near release (the OMG committee are due to ratify the proposed changes on June 21). We look at the major new features that made it into this release, how these impact users of the standard and what we think of these changes. If you are new to decision modeling.
DMN 1.2 Update
Over the last month I’ve been modelling the IFRS-17 insurance accounting regulations using the Decision Model and Notation (DMN). I’ve been using the standard to produce a transparent and executable decision service that generates ledger entries that are IFRS-17 compliant. This is complex work which uses most of DMN’s current facilities – so I thought I’d take a look at the new version of the standard, DMN 1.2, announced last week by partners Trisotech, to see what new features might be useful for complex decision models. DMN 1.2 has lots of new features, many of which were on my original April 2016 wish list. Which features will be genuinely useful and which are still missing?
New Feature: Decision Services Can Be Invoked From DRDs
DMN 1.1 supported the definition of a Decision Service, a means of specifying an interface for a Decision defined in a model, via which it can be invoked by an external architecture component (e.g., a BPMS or BRMS). An example Decision Service, Discount Service, is shown in Figure 1. It defines a means of calculating a discount for a customer given their faithfulness, volume of orders and special offers.
However, in DMN 1.1, it wasn’t possible to use Decision Services within the model itself. In this version, decisions can reuse an Business Knowledge Models using a knowledge requirement such as that shown in Figure 2. The Determine Discount knowledge model, which could offer the same business logic as the service, is supplied parameters (e.g., customer history, order volume discount and special offer details – not shown in Figures 2 and 3 for brevity) by its two ‘users’ in lieu of input data. It then provides an outcome just like the service.
DMN 1.2 now allows Decision Services to be invoked inside a model in a similar way. This means that nodes in one Decision Requirement Diagram (DRD) can invoke Decision Services defined in the same DRD, another DRD in the same model or even in a different model. An example invocation is shown in Figure 3. The rounded rectangle is a Decision Service that is being invoked by the decisions Partner Discount and Customer Discount. Just as with the Business Knowledge Model, the invokers provide the necessary parameters (documented as input data nodes outside the service box in the service definition).
Note the expand button on the decision service (the little plus in a box), when selected this toggle ‘expands the service’ giving the ‘internal view’ of the service shown above. This expansion is analogous to the way subprocesses behave in BPMN.
We like this new feature. This ability to invoke decisions in one place that are defined in another is a great step towards diagram scalability. Practitioners can now avoid complex, ‘cat’s cradle’ diagrams by separating the definition of decisions from their use. You should consider this separation of decision use and definition whenever:
- A decision is used in many places
- A decision is independent of the rest of the DRD (i.e., doesn’t share most of the same knowledge or information dependencies
- A decision and the model in which it is used are maintained by distinct parts of your organisation.
We can see this feature will be very useful in refactoring our diagrams, especially in the majority of tools that do not fully support the Decision Requirement Graph (DRG) concept.
New Feature: Ellipsis in DRDs – DRD Diagram Layout Now Maintained in Interchange Files
In DMN each Decision Requirements Diagram (DRD) shows a subset of the components and dependencies represented in the whole Decision Requirements Graph (DRG). So any given DRD in a model is a visual view of part of the DRG.
In our wishlist we argued that modelers should be able to represent (visually) that a component shown in a DRD (e.g., a decision or knowledge model) is involved in other dependencies that are (for brevity) not shown on this DRD. This is now part of DMN 1.2.
Consider an example of two separate DRDs in Figure 4. DRD ‘A’ represents the entire DRG, whereas DRD ‘B’ is a subset of the DRG and the ellipsis in the bottom decision signifies the elided child nodes.
The need for this notational convenience is clarified by a major extension of DMN 1.2’s interchange file structure (the contents of save files in DMN compliant tools). In DMN 1.1, the XML save file format used to transfer decision models from one tool to another only represented the abstract model DRG (e.g., decisions, input data, knowledge sources and their relationships). Now in 1.2, the file preserves both the concrete syntax of diagram layout (e.g., node size, fonts, node positions, color) and the abstract syntax separately. As a consequence, multiple concrete DRDs may be saved which reference objects in the same abstract model DRG.
As a result one DRD may be a partial (simplified) view of another (see example below), used for a specific purpose or to convey a specific facet of a model’s meaning. Notice how the sub-view (Figure 6) is more effective at representing a specific facet of the model (in this example how it depends on a single input data) than the entire DRD (Figure 5). You can click on these diagrams to zoom in.
This use of multiple diagrams to represent multiple perspectives (i.e., different views) of the same underlying graph is frequently pivotal in the development and consensus building of decision models because it allows each perspective to concentrate on clarifying a single business viewpoint (see section 14.3.2 of our book for an explanation of the importance of this). This allows you to customize the model to the audience. This is also a great way of removing the clutter in some diagrams and focusing on the important details – it’s another example of how DMN 1.2 supports scalability in decision modeling.
New Feature: More Powerful Test Expressions in Decision Tables
In our original DMN 1.2 wish list article we called for more powerful conditions to be allowed in decision table cells. We wanted complex logic to be supported without the need to resort to FEEL context entries or boxed expressions. In technical terms, we asked that the unary test restrictions be lifted and that tests not be limited to inequalities and ranges of literal values, as they are in DMN 1.1. We felt that decision table tests should embrace more of FEEL’s expressive power.
We presented a proposal in which any FEEL function could be incorporated into decision table test expressions, providing it yielded a Boolean result (i.e., the success or otherwise of the test). We suggested a reserved symbol could be used to represent the variable being tested within this test expression. The DMN committee have supported this in DMN 1.2 with a facility called generalized unary tests. They have selected the reserved symbol ‘?’. Now, in the input cells of decision tables, it is possible to use FEEL expressions like contains list(?, “NY”), to determine if a list of states includes New York. This extension has interesting consequences for decision tables as the following example shows.
Imagine you are modelling the admission rules for a grueling and dangerous mountain-pass walk. You might impose restrictions on the age, number and experience of persons comprising parties attempting the walk. According to their composition, some parties may be allowed to conduct the walk independently, some may require supervision by one of your staff and others may not be admitted at all. In DMN 1.1 such logic often had to resort to boxed contexts such as that in Figure 7 below.
However by allowing support for generalized unary tests in expressions, it’s possible to reduce this to the logic shown in Figure 8:
The use of these generalized unary tests allows input variables holding collections (in this case collections of ages, qualifications and persons) to be tested directly (as in this example) which was not possible in DMN 1.1. This is a nuanced case because although the second example is more compact and has powerful conditions, it contains some repetition in the conditions (usually this is considered poor practice and it could be prevented in this case with context entries). In addition the syntax may unnerve some business users, especially when used in conjunction with filters (as in the persons.experience conditions).
A common pattern in decision modeling is to use a list of text enumerations to represent the properties of something, for example the issuers of a financial instrument. Then decisions decide on outcomes (e.g., the credit risk of an instrument) according to the membership of this list. In the DMN 1.1 example of Figure 9, we decide the credit risk of an instrument (as a last resort) by looking at the collection Instrument Classes.
Because context entries are needed to check for each important possible member of the collection (e.g., GOVT EMERGING, AP AGENCY), the decision table requires a lot of work if new Instrument Class needs to be handled. It also gets wider and wider as each new instrument class condition is added. In DMN 1.2, we can use generalized unary tests to redesign this table to the much more readable version shown in Figure 10. This has the added benefit that additional instruments classes can be accommodated without next context entries or condition columns.
In our view the introduction of generalized unary tests in decision tables provides a new level of expressive power and scalability. We can easily recall several engagements in which it would have been useful (see example 2 above). It should be used carefully of course. Practitioners should avoid dense logic at the risk of making the decision tables less easy to understand. Nevertheless this feature offers the prospect of more elegant and powerful decision tables.
New Feature: Annotations in Decision Tables
In our wishlist article we argued (as many others did) that, just as DMN Decision Requirement Diagrams benefit from text comments called annotations to clarify their meaning (e.g., to express assumptions, explain goals, denote use of patterns, etc.), there should be a rule level annotation in decision tables with the same intent.
DMN 1.2 now supports this. Decision Table Annotations are (one or more) additional columns of a Decision Table (to the right of the outputs in a rules as rows table or below them in a ‘rules-as-columns’ table) that act like comments or post-execute explanation.
- Document the intent, rationale and idiosyncrasies of each rule (as required)
- Act as a means of supporting after-the-fact behavioral explanations of why specific rules were selected, including the ability to evaluate expressions to assist with diagnostics
- Optionally link a rule with a specific section of a Knowledge Source that it satisfies
We’ve placed an example in Figure 8, above. The Explanation column is a rule level annotation.
Annotations are an effective means of embedding business justifications and other documentation, making your Decision Tables easier to understand. They can also be used in logging decision model behaviour. What is not clear from the DMN 1.2 proposal is the extent to which annotations can evaluate expressions for logging purposes (as is the case with messages in TDM).
Additional Functions in FEEL
Although FEEL incorporates many useful built-in functions for calculating values, manipulating information and aggregating collections of values, we observed that many of the included functions lack mathematically associated functions. For example:
- sum() was supported (by DMN 1.1), but not product()
- mean() was supported, but not median(), mode() or stddev()
It seems the DMN committee were listening! DMN 1.2 incorporates a wealth of new functions, including:
- product(), median(), stddev() and mode() for both list objects and sequences of arguments
- split(string, delimiter) to divide a string into fragments
- abs() to determine the absolute value of a number
- modulo(n, d) for determining the integer remainder after division of n by d
- mathematical functions: sqrt(), log(), exp(), odd(), even()
- Boolean functions: any(), all()
A richer set of the functions reduces the number of functions that modelers have to define themselves, eliminating clutter and repeated work. This is a welcome step. It’s a shame that intersect() is still not supported: we’ve have to add this for about 40% of our decision models for determining what members two (or more) lists have in common. Nevertheless anything that improves the expressive power of FEEL is a welcome development.
Other New Features
The standard included a wealth of other additions which although less impactful are certainly welcome.
Date, Time and Duration Handling
The semantics of addition, subtraction, multiplication and division for dates, times and durations have been clarified after the ambiguities pointed out by Keith Swanson’s blog post last year. In particular the meaning of time zones has been formalized, especially where date calculations feature some dates with a timezone and others without. In addition, the properties of time, date and duration objects have been described in more detail and a new date property, weekday, has been added. This will prove useful for commercial decision models and removes the need for another common user defined function.
New string escape sequences have been added with allow quotes, backslashes, newlines, tabs and Unicode characters to be embedded into strings. These will be useful for text processing and generation.
FEEL iteration statements in DMN 1.2 may iterate over literal ranges instead of just lists. In addition iterations support the reserved word partial which represents a list of the outcome of all previous iterations. Therefore, for example, the following iteration produces the first ten triangular numbers:
For x in 1..10 return if (x=1) then 1 else x+sum(partial)
Although both of these are interesting additions and the use of literal ranges is handy, we haven’t yet come across a use case for partial.
Singleton List Handling
One commonly reoccurring pattern of FEEL expressions is applying a filter to a list which, you know in advance, will yield a single result or in which only the first result is meaningful. Nevertheless, given this list, L, it is still necessary to use the filter L to ‘unwrap’ this single object and use it. For example, if you have a FEEL object student with an attribute dependees representing a list of that student’s financial dependents, you might need the expression…
student.dependees[Relationship Classification(item.relationship) in ("PARTNER", "SPOUSE")]
…to yield the student’s (one and only) partner. DMN 1.2 allows such singleton lists (which only have one entry) to be treated, for convenience, as if they were the singleton object itself. So in the example shown we could omit the . In short, for a list with one item, L, it may behave, where relevant, as if it were L.
While we like the convenience of this, we feel on balance that it is better to be explicit about the difference between lists and items. Such an implicit filter might hide the author’s intent and serve to make FEEL harder to understand which would be unfortunate considering its audience is business analysts and subject matter experts not developers.
Repudiation of S-FEEL
S-FEEL, the simplified subset of FEEL introduced in the earliest drafts of the standard, has been formally renounced. Although still supported by the standard, the committee no longer recommends its use in DMN 1.2 and the implication is that it may be removed in a later version.
S-FEEL was a strict subset of the FEEL expression language introduced with the idea of allowing business analysts to use simple value expressions without overloading their models with the complexity of FEEL full expressions or boxed expressions. It was also a way for tool vendors to offer an intermediate level of support for FEEL.
In practice however, we (and many others) never encountered a single use case for DMN which required S-FEEL. Models can either avoid using the expression language completely and express all their logic using decision tables (it’s surprising how often this can be achieved) or they need expressions, in which case the small subset provided by S-FEEL is never enough. Because of this, we welcome this development.
We feel the latest release of DMN offers many exciting new features and these will help to make decision modeling even more relevant to corporations that need to improve the transparency of their business decisions. Decision services, generalized unary expressions and the extension of the file interchange format will have the most significant impact in our view. It’s good to see that more common functions are being supported (many vendors had supplied these before DMN 1.2) and that the glitches in datetime arithmetic have been ironed out.
One of the most powerful applications of decision modeling is in the integration of predictive analytics, machine learning and AI into business decision making and we were sorry to see no significant advances in this direction. In addition, the continued lack of any glossary integration (e.g., data source and enumeration management) remains a limitation in DMN that is felt by every corporate adopter.