Read Blog Post

Read Blog Post

Contributed on December 2, 2021

48

903

What Makes BPMN and DMN Standards?

Blog: Method & Style (Bruce Silver)

A couple weeks ago my attention was called to a LinkedIn post, one of those clickbait polls: “Is BPMN the standard? Do you use it? Would you expect it from others?” About 65% said Yes, but naturally the haters clogged the comment thread. Of most interest to me was a comment by Alec Sharp, a respected process modeling consultant, who says…

BPMN is what I call a “claimed standard”… in practice rarely followed, at least as intended. I did an informal survey some years back with some of the top business process folks in the world, and none could think of a single case where BPMN contributed to a significant piece of process analysis and design.

That may be true, but this statement betrays a fundamental misunderstanding of what standards are and what users should expect from them.

Item 2. At this year’s Decision Camp, a technical conference for the decision management community, a presentation entitled “DMN On-Ramp: A Pragmatic Approach to DMN Standardization” proposed a new “consortium of practitioners, stakeholders, academics and vendors” that aims to “provide a ‘staircase up the compliance mountain’ for users and vendors; educate, raise awareness about DMN and its business use cases; [and create an] incremental, vendor neutral route to standardization and compliance.” Admittedly, DMN has underperformed in the marketplace, and a community of practice designed to increase adoption is actually a great idea. But with DMN there is no “compliance mountain”; the requirements for Level 1 conformance are barely a molehill. It’s true that I have called Level 1 tools “fake DMN,” but it hasn’t seemed to bother those vendors. The striking thing about this new organization, which is still not off the ground, is that what it dislikes about DMN is everything that makes it a standard.

In the world of software, a standard provides a functional specification for some component or system, defining the name, semantics, and operational behavior of each element.  It is typically filled with SHALLs and MAYs, distinguishing absolute requirements from nice-to-haves.  The specification is developed and maintained not by a single company but by a group of vendors and practitioners representative of that community.  The group’s procedures are governed by strict rules ensuring vendor-neutrality – adoption of some clause is typically by majority vote – and the resulting intellectual property is available free of charge.

Business modeling standards like BPMN and DMN in addition define a graphical notation – standardized shapes and symbols linked to the semantic elements – and a model interchange format in some universal language such as XML. Standardizing the semantics is an absolute must; without that, you cannot legitimately call it a standard.  In business modeling standards from OMG, the semantics are defined by a combination of a UML metamodel (including its companion XML schema) and tables and text in the specification document.  Without adherence to the required notation and semantics, a product cannot rightfully claim conformance to the standard.

Software products conformant to the standard should enjoy some measure of interoperability, benefitting both end users and product vendors:

1. End users have a choice of product vendors conformant to the standard, reducing vendor lock-in risk.  They might start with vendor A and switch later to vendor B with less disruption than starting over completely.  Or business users might use a tool from vendor A and developers a different tool from vendor B.
2. End users have multiple sources of information about the standard and how to use it effectively, in the form of books, online articles, conference presentations, and independent consultants and training.  They are not solely dependent on their product vendor for that.
3. Vendors can more easily incorporate or interact with components from other providers.  They are not obligated to create an entire whole-product ecosystem themselves.
4. In the case of business modeling standards, vendors can more easily engage business users in defining requirements or even full solution implementations.  This benefit has been a particularly significant factor in the adoption of BPMN and DMN.

If you’ve always suspected that standards, while purportedly for the benefit of end users, were actually created by vendors for their own commercial benefit, you’d be right.  Creating standards is a lot of work, as I know from personal experience.  Members of the task force spend several hours a week in conference calls, developing and testing proposals, and reviewing specification text, and someone has to pay their salary for that time.  Actually, the benefits of a standard accrue directly more to end users than to vendors, who typically must modify their existing products to conform to the spec.  But as RFPs and informal product research increasingly add standards compliance as a requirement, vendors are pushed to at least claim support.

Much of the confusion around standards comes from several factors:

1. A standard’s specification is the work of a committee.  As a result, it typically contains some elements of little utility, or none at all, that were put forward by some task force member and managed somehow to be adopted.  Not all parts of the spec are used in practice.
2. The spec is not a teaching document for end users.  It is written for “implementers”, meaning tool vendors.  The smarter ones find a way to ignore the “junk” elements.  Free posters on the web displaying all elements defined in the spec are not “proof” that the standard is too complicated to use; they are just free stuff on the web.  Methodologies like BPMN Method and Style show how to do just about anything you want with a small subset of elements.
3. Standards are based on a particular conceptual framework and set of principles, ideally one already in common but imprecise use.  But adoption of that framework or principles is not the same as conformance to the standard.
4. Standards sometimes claim an intended audience or pattern of use, in particular with regard to empowerment of business users.  Tools based on the standard may or may not choose to support that pattern of use.

In the case of BPMN and DMN specifically, items 3 and 4 above are at the heart of the controversy.

BPMN originated back in 2002-2003 at the dawn of a new technology, web services.  Before then, not only were there no standards for BPM tools, but human workflow and application integration required separate and distinct technologies.  The startup Intalio created the Business Process Modeling Language (BPML) that used web services to combine human and automated tasks in a single process model.  That was revolutionary in itself, but they went further.  BPML would be created graphically by non-programmers using a standardized flowcharting notation called BPMN.  Swimlane flowcharts had been in common use by the process improvement community since the 1980s, and basing BPMN on that conceptual framework was intended to attract the interest of business users.  It worked!  There was even a book written about it, BPM: The Third Wave by Fingar and Smith, that created a mild sensation back in 2003.

But Intalio was actually too early, as the key web service standard WSDL was not yet final, and when it came out, BPML was incompatible with it.  Instead BPML was replaced by a similar process automation language called BPEL from IBM and Microsoft.  Those vendors had no interest in business user empowerment and hence provided no graphical notation.  But BPMN remained popular with business users, so process automation vendors continued to use it, even without BPML.  Increased availability of inexpensive – sometimes free – BPMN tools made it especially popular among business users looking simply to document their processes, with no interest in automation at all.  Over the past 15 years, I’ve trained nearly 10,000 users on process modeling with BPMN, and well over 90% of them have zero interest in automating their processes.

For automation vendors, the original promise of BPMN to business – What You Draw Is What You Execute – was not really true in the BPEL era.  But in 2008-2009, IBM, Oracle, and SAP needed a way to make it true in order to gain business support for large-scale IT projects based on Service Oriented Architecture (SOA).  The result was BPMN 2.0, which made the BPMN notation a true standard, maintained by a real standards organization (OMG), with formal procedures, well-defined semantics and operational behavior, a metamodel and schema, the whole thing.  The semantics and operational behavior had to be well-defined because now BPMN models were automatable on a runtime engine.

Support from the major software vendors increased the availability of BPMN tools, although the vast majority of BPMN users still had no interest in automation.  And the spec was even more complicated than before, still with many junk elements.  That was the birth of BPMN Method and Style, my methodology for getting rid of the junk and creating models that reveal the process logic clearly from the printed diagrams alone.  Two-thirds of the BPMN 2.0 spec concerns details “below the waterline” – not visible in the process diagrams but there to capture implementation details in the XML serialization.  Some of those details are necessary to understand the flow logic, and so Method and Style uses certain conventions to surface them in the diagrams.

So, to get back to Alec Sharp’s misunderstanding, BPMN as a standard has always been about process automation, not process improvement.  It incorporates a conceptual framework – swimlane diagrams – that long predated it, but adds to that strict semantics and operational behavior behind the shapes and symbols.  That semantic clarity is actually a huge improvement over informal swimlane diagrams, but effective use of BPMN for non-executable models requires combining it with something like Method and Style.  Because the BPMN spec provides a standard XML format for the models, Method and Style’s “rules” can be validated in a BPMN tool like Trisotech or ViziModeler, a huge bonus.

In that LinkedIn post, several commenters boasted about how they use BPMN carelessly or even intentionally incorrectly, but what is the point of private semantics?  Several managers have told me of the vast sums wasted by their BPM project teams because their BPMN models could be understood only by the person who created them.  By respecting the defined semantics of the shapes and symbols and adopting conventions like Method and Style, BPMN models become understandable by all members of the team.

Now let’s turn to DMN, which came out in 2015, several years after BPMN 2.0.  Like BPMN, DMN took as its conceptual framework an idea that was becoming increasingly popular, “Decisions First”, meaning decomposing a complex decision, top-down, into a hierarchy of supporting decisions.  This approach stood in contrast to the older “Business Rules” approach based on harvesting rules from a vast array of legacy systems and subject matter experts, and later assembling the rules, bottom-up, into decision logic of some utility.  What DMN calls a Decision Requirements Diagram (DRD) did not originate in DMN, but had been articulated well beforehand in James Taylor’s Decision Management Manifesto, The Decision Model by Goldberg and von Halle, and Knowledge Automation by Alan Fish.  This is important, because just as swimlane diagrams do not imply BPMN compliance, not every tool supporting some form of DRD is DMN-conformant.

DMN distinguishes decision requirements from decision logic.  Decision requirements, captured as DRDs, are intended to be created by business users.  In that diagram, each decision is represented as a rectangle, with incoming arrows representing the inputs to its internal logic, other supporting decisions and input data.  For reasons I don’t pretend to understand, DMN took a step that BPMN dared not even suggest, which is that decision logic should be created by business users as well.  I say I don’t understand it because, looking around today, I see very few DMN vendors that support that notion… Trisotech being the notable exception.

To enable that, DMN defined a set of standard tabular formats called boxed expressions and a new business-friendly expression language called FEEL to go in the cells of those tables.  One type of boxed expression, the decision table, is another example of a popular pre-existing conceptual framework adopted by DMN, but there are several other boxed expression types that are new.  As if that were not enough, the variables used in FEEL are the names of the decisions, input data, and other DRD elements created by business users, which contain spaces and punctuation absolutely forbidden as variable names in almost every other executable language.

There is another key difference between BPMN and DMN.  While most BPMN models are not meant for execution on an automation engine, most DMN models are ultimately intended for execution, so the decision logic is going to be created by someone, often a developer outside of DMN.  While the DMN spec does not insist on use of FEEL and boxed expressions, those elements allow business users to create fully executable models themselves.

Recognizing that many vendors might not want to support FEEL and boxed expressions, DMN defined three levels of conformance.  Conformance Level 1 requires support for

• DRDs (Chapter 6), meaning both the shapes and symbols and their semantics and operational behavior;
• Boxed expressions (Chapter 7), although it is not clear which ones must be supported; and
• Decision tables (Chapter 8)

It does not require that expressions in decision tables and other boxed expressions use FEEL, except to say that if the tool “provides an interpretation” of an expression – I think that means it is executable – it must follow the rules of Chapter 7, boxed expressions.

Conformance Level 2 requires everything from Level 1 plus support for a FEEL subset called S-FEEL in decision tables.

Conformance Level 3 requires everything from Level 1 plus FEEL.

From my observation, most tools claiming DMN conformance are something less than Level 1.  They have DRDs, but without all the elements and not all the correct shapes.  They support decision tables and literal expressions but no other boxed expressions.  And they do not support FEEL.  In other words, they support the pre-existing conceptual frameworks of a decision requirements diagram and decision table, but not the DMN standard.

S-FEEL, required for Level 2, is now deprecated, but basically it means decision table cells must be what are called unary tests in the spec.  Unary tests are extremely simple expressions that allow the table to be verified for completeness.  Basically this means executable decision tables with standard syntax in the cells.

Level 3, to me, is real DMN.  Few vendors support it.  The DMN TCK, sponsored by the Workflow Management Coalition, provides test suites of executable models exploring all the obscure edge and corner cases of FEEL, ensuring interoperability of Level 3-compliant tools, a tremendous benefit.

So let’s return to the “DMN On-Ramp”.  From what I can see, its organizers are fine with DRDs and decision tables, the conceptual frameworks predating DMN.  Their primary complaint, as articulated on their website, is this:

Established vendors already have fully defined expression languages, and ways to manipulate data. For various reasons, it is not feasible for the majority of vendors to replace their implementation with all that DMN defines.

In other words, they just don’t like the DMN part, boxed expressions and FEEL. The question becomes this: If the On-Ramp becomes a reality, what makes the resulting language a standard?

• Are DMN-standard semantics and behaviors still respected?
• What is interoperable between tools?  Is there a standard XML interchange format, and if so, is it compatible with DMN extensions?

If those questions have good answers, the resulting models would meet my definition of a workable standard, and I am all for the effort.  I am not wedded to FEEL.  It is a very good language, powerful yet easy to use.  I have posted previously on its superiority to PowerFX, Microsoft’s expression language for Excel and PowerApps, but I am not opposed to using it in boxed expressions, or plugging in SQL, javascript, or even some vendor’s proprietary rule language so long as it is just an expression language and uses DMN-standard constructs like DRD elements and decision tables, not program statements.

I agree also with the sentiment that DMN Conformance levels are problematic, not that they are an impossible mountain to climb but that they are vague and confusing.  Instead of saying a tool must support all the requirements of Chapters 6, 7, and 8 of the spec, it should be more specific.  At a minimum, a tool should support DRDs containing a specific list of elements, render them with the standard shapes and symbols and endow them with standard semantics and behaviors.  That for me would be Level 1.  For Level 2 I would add decision tables supporting DMN hit policies and unary tests in the input entries.  The On-Ramp people seem to like a lot of metadata used to back up the logic, and if they want to add conformance classes that involve those, I would have no objection.  These things would likely turn fake DMN vendors into real DMN vendors and as a result increase DMN adoption by end users.  That would be a very good thing.