Case Study: Auditing With Process Mining — Part IX: Analysis Answers
This is the 9th article in our case study series on auditing with process mining. The series is written by Jasmine Handler and Andreas Preslmayr from the City of Vienna. You can find an overview of all the articles in the series here.
To answer our analysis questions from Table 1, we used both an explorative and targeted analysis.
During the explorative analysis, we investigated the discovered models looking for unexpected or strange process paths, long waiting times, and other abnormalities in a broader way. With this type of analysis, we were able to address questions No. 1 (“Do the real processes fit the should-process?”) and No. 12 (“Are there any bottlenecks within the process?”).
Questions No. 2 to 11 were answered through a targeted analysis. We translated each analysis question into a customized set of filters based on the definitions we had created in the ‘Analysis questions’ step. Although we had already defined the analysis questions in some detail, we further refined these specifications to ensure we only detected cases that violated the process requirements.
For example, the goal of question No. 2 (“Are all orders released?”) was to find out whether there were cases with a missing order release. The target value for this analysis was 100%. However, there could be legitimate reasons why an order release could be missing. We concluded that only orders that were executed needed a release activity. Furthermore, there might be orders that were never carried out. Thus, we decided to exclude canceled cases from the data basis for this question.
So, we used a combination of three different filters to answer question No. 2 (“Are all orders released?”). First, we excluded all orders that were canceled, which in our data set meant removing cases with the attribute value L or X in the attribute “Löschkennzeichen” (see Figure 13).
Figure 13: Excluding canceled orders
Second, finding out whether an order was really executed was not easy. We had no data that indicated when an order form was sent to the supplier. There was also no data that showed when the contract was locked or when the incoming goods were checked. Therefore, we chose the invoice as a means to measure if an order was actually performed. Figure 14 shows the second filter that includes only orders with an invoice (“Eingangsrechnung erfassen” in German).
Figure 14: Including only orders with an invoice
Third, because we wanted to find violations of question No. 2, we excluded all orders with an order release activity. So, we searched for the opposite of the required behavior in the process. Figure 15 shows this third filter that excludes all cases that contain any of the selected order release activities (“PO – Dokument freigegeben. Stufe 001” – “… 003”).
Figure 15: Excluding orders with an order release activity
As a result of applying these three filters, we found that 17% of all non-canceled cases lacked an order release, although an invoice was recorded. Looking at those cases in more detail, we discovered that all orders referred to the goods group of office supply. The audited party explained that office supplies did not need to go through the regular order release workflow and could be ordered without a prior order release. Thus, these orders also complied with the procurement guidelines despite the irregularities we discovered.
Similar to the example above, we answered each of the remaining analysis questions by translating them into customized filter settings. In addition, we discussed all the discovered irregularities with the audited party to determine whether there was a genuine reason for the deviation or whether we had found a compliance violation within the process.
New parts in this auditing series will appear on this blog every week. Simply come back or sign up to be notified about new blog entries here.
Leave a Comment
You must be logged in to post a comment.
