Handling Complex XML Input in DMN
Blog: Method & Style (Bruce Silver)
In a recent post, I showed how DMN could be used to model and execute decision logic on complex XML input data such as loan application information in MISMO format. The MISMO schema was envisioned as an enterprise data dictionary for the mortgage industry, standardizing the names of various information elements for use across any mortgage-related application. As such, however, it is effectively impossible to use MISMO data directly in decision models. For one thing, the elements needed for a particular decision begin five or six levels down from the root and tend to be strewn across a variety of containers. For another, concepts fundamental to a particular mortgage area, such as Borrower, are not in the schema at all. In MISMO, a Borrower is a PARTY with attribute PartyRoleType of “Borrower”. And something as basic as the Borrower’s current income requires a complex join of income items to an Employer of the Borrower. Still, it’s just a data mapping problem.
Trisotech DMN Modeler can automatically create a FEEL item definition (datatype) from XML schema and map XML input data to FEEL on execution. But if we do that directly with the MISMO XSD, we wind up with FEEL elements that – like the original XML data – are effectively impossible to use in decision logic. In the referenced post I solved that problem by mapping MISMO XML to a more DMN-friendly XML schema.
I used XSLT, an XML mapping language, executed as a service in a BPMN model.
Although XSLT is programming, you can generate even complicated mappings graphically in tools like Altova Mapforce, although, to be honest, you need to be pretty familiar with XPath and MISMO to do it. So I can understand why that solution might be unattractive to someone in the mortgage industry who wants to use DMN. They would prefer to create decision models that operate directly on MISMO XML.
It’s not pretty, but you can do it. Even better, you can encapsulate the ugly part in a decision service, created once and reused in any MISMO decision model. My updated solution makes use of Trisotech’s MISMO Accelerator, which lets you import individual container elements of the MISMO schema rather than the whole thing. That helps a bit, but not as much as I’d hoped. In reality, MISMO data is sparse, meaning in a typical instance only a minuscule fraction of the XML elements are present. In the XML instance they are simply omitted, but in the FEEL item definition generated from the XSD they are all present and assigned the value null at runtime. So, even with the Accelerator, the FEEL input data element is huge, over 10 MB, almost all components null.
The scenario is the same as last time. Ken Customer wants to borrow $300,000 on a $340,000 purchase of a single-family home. Our decision logic uses FannieMae’s published manual underwriting rules applied to the data on Ken’s URLA form in MISMO XML format. But unlike last time, now the decision model operates directly on MISMO data.
The model can be deployed to the Trisotech cloud as a decision service that accepts XML input in MISMO format. The model must have a single input data element, and its name must be the name of the MISMO XML root, MESSAGE. But its datatype is not strictly converted from MISMO MESSAGE, which would be the entire MISMO schema. Instead it is manually constructed from the seven container elements selected using the Accelerator: ASSETS, COLLATERALS, EXPENSES, LOAN, LIABILITIES, PARTY, and RELATIONSHIPS, and omitting all the others. In this way, the XML input is valid per the full MISMO schema but contains elements of those seven containers only.
The only decision that operates on MESSAGE is called Mapping, and it simply passes MESSAGE to the decision service MapMISMO. MapMISMO is the DMN equivalent of the XSLT service mentioned earlier. It maps MISMO elements – here in FEEL, not XML – to a more DMN-friendly data structure. MapMISMO is shown below:
The decisions labeled in all caps connected to the input data MESSAGE are the MISMO container elements selected from the Accelerator. They simply drill down from the MISMO root to the selected container. For example, ASSETS is shown here:
The other decisions are the FEEL variables used in the main model decision logic. I will show just a bit of the mapping logic to give an idea of the expressions involved. Borrower1 is a fairly large context. The first several context entries create local variables used in the final result.
theParty selects the MISMO PARTY element representing Borrower1. BorrowerRole selects that PARTY‘s BORROWER container. CurrEmployer1 selects the first EMPLOYER of that BorrowerRole with a status of “Current”. You get the idea… it’s a lot of filter expressions.
To get the Borrower’s current income, itemized by type (Base, Bonus, etc.), we need to create a join using MISMO’s RELATIONSHIPS element. First we get a list of income item ids for the Borrower’s current employer:
The expression above has one tricky part: Because a filter always returns a list, the income items associated with a particular Employer is a list of lists, so we need the FEEL flatten() function to make it a simple list.
We also need to define a function that returns a row of the income item table, including the income type and amount, given its id. This is a context with no final result box, one context entry per column of the table:
To generate the table, we need to iterate over the list of income item ids to call the function:
This gives the flavor of the mapping logic. It’s just FEEL, nothing fancy, filters and occasionally a join, table row context, and iteration. There is a lot of detail to it, but by creating the mapping as a decision service, you can import and reuse it in a wide variety of decision models operating on MISMO data.
Everything you need to know to create models like this is explained in the DMN Basics and DMN Advanced training. Understanding the MISMO data model… that’s up to you.
The post Handling Complex XML Input in DMN appeared first on Method and Style.
Leave a Comment
You must be logged in to post a comment.
