Case Study: Auditing With Process Mining — Part III: Analysis Questions
This is the 3rd article in our case study series on auditing with process mining. The series is written by Jasmine Handler and Andreas Preslmayr from the City of Vienna. You can find an overview of all the articles in the series here.
We defined the following 12 analysis questions (see Table 1). As shown in Table 1, most of the questions are related to compliance issues (our primary audit objective), only two are related to performance questions, and none are about user experience.
Table 1: Analysis questions
Besides formulating the general analysis questions, we also tried to define them as precisely as possible and make them measurable. Thus, we specified the metric, target value, process scope of interest, and influencing factors for each analysis question. Table 2 shows how we made analysis question No. 2 more concrete by defining these aspects.
Table 2: Analysis question No. 2 in more detail
When we want to answer the question “Are all orders released?” it seems straightforward initially, but it is a good idea to think further about how exactly we can measure the answer to this question. We expected that order releases would be registered within the information system. So, we chose the presence of the release activity as a metric. In addition, we assumed that all orders must be released without exception. So, we set the target value to 100%, which means that a release activity needs to be documented for each order.
Then, we defined the process scope to show which part of the purchase-to-pay process is relevant to find the information needed to answer the analysis question. For question No. 2, the process scope comprised all activities related to the release step in the information system.
Finally, we also collected the influencing factors we needed to consider while performing the data analysis and interpreting the results. Regarding the order release, we assumed that a four-eye principle might be relevant above a specific value limit. Thus we defined the value limit as an influencing factor that needed to be kept in mind for later analysis.
We specified detailed definitions for all analysis questions in a similar manner as shown for question No. 2 above.
Because we defined the analysis questions at a very early stage of the process mining project, we needed to make certain assumptions, especially regarding the metric and the influencing factors. We adapted these analysis questions multiple times in later phases of the project because we kept gaining more insight into the process and the data.
Nevertheless, defining the analysis questions at this early stage of the project was very useful. It helped us to get a good overview of the data we needed. As a result, we could reduce the risk of forgetting certain aspects or data fields during data extraction.
New parts in this auditing series will appear on this blog every week. Simply come back or sign up to be notified about new blog entries here.
Leave a Comment
You must be logged in to post a comment.
