Contributed on October 3, 2013

1.8K

1.3K

Colloquium@TUe

Description

Slides of my presentation at Eindhoven University of Technology, 3 October 2013, Eindhoven, the Netherlands

Transcript

Ghent University & Eindhoven University of Technology
jan.claes@ugent.be – www.janclaes.info
TU/e Colloquium 2013
2 October, 2013
FACULTY OF ECONOMICS AND BUSINESS ADMINISTRATION
The process of process modeling
and process model quality
Jan Claes
Teaching assistant : PhD 2009 – 2015 : Joint PhD
Supervisors : Geert Poels (UGent) and Paul Grefen (TU/e)
Co-supervisors : Frederik Gailly (UGent) and Irene Vanderfeesten (TU/e)
TU/e Colloquium 2013
2/41
Ghent University & Eindhoven University of Technology
jan.claes@ugent.be – www.janclaes.info
Outline
Process of process modeling (PPM)
 PPMChart visualization
 Structured process modeling (SPM)
Future work: preliminary ideas
 Process model quality
 Experiments to link SPM with quality
TU/e Colloquium 2013
3/41
Ghent University & Eindhoven University of Technology
jan.claes@ugent.be – www.janclaes.info
Process of Process Modeling (PPM)
Properties of
textual description
Properties of
modeler
Properties of
modeling process
Properties of
resulting model
PRIMARY RESEARCH FOCUS
Properties of
real process
Properties of
observation process
Properties of software
and modeling language
Properties of
model reader
Properties of
reading process
Properties of
process engine
TU/e Colloquium 2013
4/41
Ghent University & Eindhoven University of Technology
jan.claes@ugent.be – www.janclaes.info
Process of Process Modeling (PPM)
Observational modeling sessions
 People construct models
 Every action on modeling canvas is logged
 Different datasets
• 120 students in Eindhoven 2010
• 14 experts in Berlin 2010
• 14 experts in Eindhoven 2011
• 118 students in Eindhoven 2012
• 146 students in Gent 2013
TU/e Colloquium 2013
5/41
Ghent University & Eindhoven University of Technology
jan.claes@ugent.be – www.janclaes.info
Process of Process Modeling (PPM)
Properties of the modeling process
Activity Timestamp Attributes
Create start event 10:00 Id = 1; x = 10; y = 10
Create activity 10:04 Id = 2; x = 40; y = 10; name = “Receive order”
Create edge 10:05 Id = 3; from = 1, to = 2
Move activity 10:07 Id = 2; x = 15; y = 10
Create gateway 10:08 Id = 4; x = 65; y = 10; type = “XOR”
Create edge 10:09 Id = 5; from = 2, to = 4
Create activity 10:24 Id = 6; x = 80; y = 0; name = “Reject order”
Create activity 10:25 Id = 7; x = 80; y = 20; name = “Prepare order”
Create gateway 10:27 Id = 8; x = 105; y = 10; type = “XOR”
TU/e Colloquium 2013
6/41
Ghent University & Eindhoven University of Technology
jan.claes@ugent.be – www.janclaes.info
 CREATE_ACTIVITY
 CREATE_START_EVENT
 CREATE_END_EVENT
 CREATE_AND
 CREATE_XOR
 CREATE_EDGE
 MOVE_ACTIVITY
 MOVE_START_EVENT
 MOVE_END_EVENT
 MOVE_AND
 MOVE_XOR
 DELETE_ACTIVITY
 DELETE_START_EVENT
 DELETE-END_EVENT
 DELETE_AND
 DELETE_XOR
 DELETE_EDGE
 NAME_ACTIVITY
 RENAME_ACTIVITY
 NAME_EDGE
 RENAME_EDGE
Process of Process Modeling (PPM)
Visualization in PPMChart
time
modelelements
TU/e Colloquium 2013
7/41
Ghent University & Eindhoven University of Technology
jan.claes@ugent.be – www.janclaes.info
Process of Process Modeling (PPM)
9 design principles of visual notations(Moody 2009)
 Visual expressiveness
 Perceptual discriminability
 Graphic economy
 Semantic transparency
 Semiotic clarity
 Dual coding
 Cognitive fit
 Complexity management
 Cognitive integration
TU/e Colloquium 2013
8/41
Ghent University & Eindhoven University of Technology
jan.claes@ugent.be – www.janclaes.info
Process of Process Modeling (PPM)
Visual expressiveness
 Optimal use of graphical variables
 8 graphical variables: shape, size, color, brightness,
orientation, texture, horizontal position and
vertical position(Bertin, 2010)
 Color is most effective(Lohse, 1993; Treisman, 1982; Winn, 1993)
 But can also cause problems (e.g., color blindness,
black-and-white printers)
TU/e Colloquium 2013
9/41
Ghent University & Eindhoven University of Technology
jan.claes@ugent.be – www.janclaes.info
Process of Process Modeling (PPM)
8 graphical variables
 Shape: model element type (   )
 Size: not used
 Color: operation type (   )
 Brightness: model element type (   )
 Orientation: not used
 Texture: not used
 Horizontal position: time
 Vertical position: model element
TU/e Colloquium 2013
10/41
Ghent University & Eindhoven University of Technology
jan.claes@ugent.be – www.janclaes.info
Process of Process Modeling (PPM)
Perceptual discriminability
 Symbols are clearly distinguishable
 The more two concepts differ, the more the
corresponding symbols should differ(Winn, 1990)
 Visual distance is determined by
• Number of different values for the graphical variables
• Size of these differences
 CREATE_ACTIVITY
 CREATE_START_EVENT
 CREATE_END_EVENT
 CREATE_AND
 CREATE_XOR
 CREATE_EDGE
 MOVE_ACTIVITY
 MOVE_START_EVENT
 MOVE_END_EVENT
 MOVE_AND
 MOVE_XOR
 DELETE_ACTIVITY
 DELETE_START_EVENT
 DELETE-END_EVENT
 DELETE_AND
 DELETE_XOR
 DELETE_EDGE
 NAME_ACTIVITY
 RENAME_ACTIVITY
 NAME_EDGE
 RENAME_EDGE
TU/e Colloquium 2013
11/41
Ghent University & Eindhoven University of Technology
jan.claes@ugent.be – www.janclaes.info
Process of Process Modeling (PPM)
Graphic economy
 Limited amount of values for each variable
 Assures cognitive effectiveness(Nordbotten & Crosby, 1999)
 Span of absolute judgment
• Is the amount of distinct observable perceptual values
• Estimated at seven (Miller, 1956)
 Span of attention
• Is the amount of different objects that can be
distinguished at a glance
• Estimated at six objects(Miller, 1956)
TU/e Colloquium 2013
12/41
Ghent University & Eindhoven University of Technology
jan.claes@ugent.be – www.janclaes.info
Process of Process Modeling (PPM)
Semantic transparency
 If a novice would be able to guess the meaning of
each symbol
 Achieved through natural mappings (Norman, 2002)
 Shapes similar to bpmn (   )
 Logical colors (creation, deletion, movement)
 Horizontal timing
TU/e Colloquium 2013
13/41
Ghent University & Eindhoven University of Technology
jan.claes@ugent.be – www.janclaes.info
Process of Process Modeling (PPM)
Semiotic clarity
 Every concept is represented by exactly one symbol
and every symbol represents exactly one concept
(Goodman, 1968)
 Same default symbol for XOR and AND gateway ()
 Same default symbol for start and end event ()
TU/e Colloquium 2013
14/41
Ghent University & Eindhoven University of Technology
jan.claes@ugent.be – www.janclaes.info
Process of Process Modeling (PPM)
Dual coding
 For information processing
 Graphical representation is better than textual
 Combination has highest cognitive effectiveness
(Paivio, 1990)
 Textual line identifiers and time intervals
 No textual code on the dots
 Textual information in pop-up on selected items
TU/e Colloquium 2013
15/41
Ghent University & Eindhoven University of Technology
jan.claes@ugent.be – www.janclaes.info
Process of Process Modeling (PPM)
Cognitive fit
 Optimal representation depends on the task
 Cognitive load is lower for experts(Vessey & Galletta, 1991)
(Optimal representation depends on the modeler)
 View is customizable through various options
 View can be filtered (e.g. hide deleted elements)
TU/e Colloquium 2013
16/41
Ghent University & Eindhoven University of Technology
jan.claes@ugent.be – www.janclaes.info
Process of Process Modeling (PPM)
Complexity management
 Reduce complexity(R. Weber, 1997)
• by modularization (divide the diagram in smaller
subsystems)
• hierarchical structuring (make separate diagrams of the
same information at different levels of abstraction)
 Only one PPM instance at a time
TU/e Colloquium 2013
17/41
Ghent University & Eindhoven University of Technology
jan.claes@ugent.be – www.janclaes.info
Process of Process Modeling (PPM)
Cognitive integration
 Mechanisms to integrate different diagrams
(Hahn & Kim, 1999; Kim, Hahn, & Hahn, 2000)
 Fixed default values for easy comparison
 Line identifiers correspond to model element id’s
 Lines are sorted according to model (start to end)
TU/e Colloquium 2013
18/41
Ghent University & Eindhoven University of Technology
jan.claes@ugent.be – www.janclaes.info
 CREATE_ACTIVITY
 CREATE_START_EVENT
 CREATE_END_EVENT
 CREATE_AND
 CREATE_XOR
 CREATE_EDGE
 MOVE_ACTIVITY
 MOVE_START_EVENT
 MOVE_END_EVENT
 MOVE_AND
 MOVE_XOR
 DELETE_ACTIVITY
 DELETE_START_EVENT
 DELETE-END_EVENT
 DELETE_AND
 DELETE_XOR
 DELETE_EDGE
 NAME_ACTIVITY
 RENAME_ACTIVITY
 NAME_EDGE
 RENAME_EDGE
Process of Process Modeling (PPM)
Visualization in PPMChart
 Start event
 Edge
 Activity
 Gateway
 Edge
 Activity
 Edge
 Edge
 Activity
 Edge
 Gateway
 Edge
7
29
8
9
32
14
30
31
10
33
56
34
time
modelelements
TU/e Colloquium 2013
19/41
Ghent University & Eindhoven University of Technology
jan.claes@ugent.be – www.janclaes.info
Structured Process Modeling (SPM)
UNSTRUCTURED
(rather) chaotic process
TU/e Colloquium 2013
20/41
Ghent University & Eindhoven University of Technology
jan.claes@ugent.be – www.janclaes.info
Structured Process Modeling (SPM)
FLOW-ORIENTED
From start event to end event
TU/e Colloquium 2013
21/41
Ghent University & Eindhoven University of Technology
jan.claes@ugent.be – www.janclaes.info
ASPECT-ORIENTED
Content – structure – lay-out
Structured Process Modeling (SPM)
TU/e Colloquium 2013
22/41
Ghent University & Eindhoven University of Technology
jan.claes@ugent.be – www.janclaes.info
CHUNKING
Work on model part by part
Structured Process Modeling (SPM)
TU/e Colloquium 2013
23/41
Ghent University & Eindhoven University of Technology
jan.claes@ugent.be – www.janclaes.info
Structured Process Modeling (SPM)
Structured process modeling
 Applying a modeling strategy consistently
 Flow-oriented modeling versus aspect-oriented
 First content, then
structure, then lay-out
 Finish aspect before continuing
 Separate vertical zones
 From start to end
(according to the process flow)
 Finish part before continuing
 Diagonal zone in charts
With or within chunks
TU/e Colloquium 2013
24/41
Ghent University & Eindhoven University of Technology
jan.claes@ugent.be – www.janclaes.info
Structured Process Modeling (SPM)
Cognitive aspects
 Cognitive Load Theory (CLT)
limited capacity of working memory
 Cognitive Fit Theory (CFT)
effect increase if task representation fits
TU/e Colloquium 2013
25/41
Ghent University & Eindhoven University of Technology
jan.claes@ugent.be – www.janclaes.info
Structured Process Modeling (SPM)
Conclusion 1
How to construct models?
Apply a modeling style consistently!
Properties of
modeling process
Properties of
resulting model
PRIMARY RESEARCH FOCUS
Structured process modeling
Aspect-oriented
modeling
Flow-oriented
modeling
Chunked
modeling
TU/e Colloquium 2013
26/41
Ghent University & Eindhoven University of Technology
jan.claes@ugent.be – www.janclaes.info
Different studies
 Apply process mining techniques on historical data (Eo-BS)
 Develop PPM visualization (Co-DS)
 Different structuring styles (Ea-BS)
 Link modeling strategy with model quality (Co-BS)
 Develop method/tool to increase model quality (Co-DS)
Different research methods
 Behavioral science (BS) vs. Design science (DS)
 Explorative (Eo) vs. Explanatory (Ea) vs. Confirmative (Co)
Different studies
 Apply process mining techniques on historical data
 Develop PPM visualization
 Different structuring styles
 Link modeling strategy with model quality
 Develop method/tool to increase model quality
Methodology
TU/e Colloquium 2013
27/41
Ghent University & Eindhoven University of Technology
jan.claes@ugent.be – www.janclaes.info
Business Process Modeling (BPM)
Business process model
 Graphical, abstract representation of a process
 Important tool for analysis and improvement
Business process model in BPMN notation
TU/e Colloquium 2013
28/41
Ghent University & Eindhoven University of Technology
jan.claes@ugent.be – www.janclaes.info
Process model quality
 Less nodes
 Less crossing arcs
 Less nested gateways
 …
 More realistic
 More precise
 More complete
 …
Which model is better?
TU/e Colloquium 2013
29/41
Ghent University & Eindhoven University of Technology
jan.claes@ugent.be – www.janclaes.info
Which model is better?
Process model quality
 Less nodes
 Less crossing arcs
 Less nested gateways
 …
 More realistic
 More precise
 More complete
 …
InspectionExecution
TU/e Colloquium 2013
30/41
Ghent University & Eindhoven University of Technology
jan.claes@ugent.be – www.janclaes.info
Execution Inspection
Process model quality
Conclusion 2
Process model quality?
Depends on the goal of the model!
TU/e Colloquium 2013
31/41
Ghent University & Eindhoven University of Technology
jan.claes@ugent.be – www.janclaes.info
Structured Process Modeling (SPM)
How does SPM influence model quality?
Structured process modeling lowers
cognitive efforts and cognitive overload
Less unintentional quality issues in model
 Focus on correctness and completeness
 Focus on understandability and maintainability
But no effect on
 Missing knowledge of domain or model language
 Wrong quality focus
TU/e Colloquium 2013
32/41
Ghent University & Eindhoven University of Technology
jan.claes@ugent.be – www.janclaes.info
Structured Process Modeling (SPM)
Structured process modeling causes
model quality improvement!
 Explanation 1:
• Apply structured modeling style
• Lowers cognitive load
• Results in improved process model quality
 Explanation 2:
• Have a lot of modeling experience
• Results in a consistent modeling style
• And in improved process model quality
Or not?
TU/e Colloquium 2013
33/41
Ghent University & Eindhoven University of Technology
jan.claes@ugent.be – www.janclaes.info
Experiments
How to prove causality?
 Take two identical groups of people
 Give only one group a treatment
 Let both groups make the same exercise
 Check for significant difference of the results
 Difference can only be caused by treatment
TU/e Colloquium 2013
34/41
Ghent University & Eindhoven University of Technology
jan.claes@ugent.be – www.janclaes.info
Experiments
Two identical groups
 Randomized: assign participants randomly to group
 Block randomized: control for secondary variables
• E.g. equal amount of (fe)male participants in each group
 Check with pre-test: check primary variables
• E.g. check difference in experience of both groups
TU/e Colloquium 2013
35/41
Ghent University & Eindhoven University of Technology
jan.claes@ugent.be – www.janclaes.info
Experiments
Give only one group a treatment
 Traditionally pharmaceutical
 In our case learning a technique
 Placebo effect
 Treatment effect (TE) should be verified and
separated from learning effect (LE)
 Treatment group: TE + LE
 Control group: LE
TU/e Colloquium 2013
36/41
Ghent University & Eindhoven University of Technology
jan.claes@ugent.be – www.janclaes.info
Experiments
Triple blinded
 Blind experiment: participants do not know if they
are in the treatment group or the control group
 Double blind: participants and administrators do
not know to which group participants belong
 Triple blind: participants, administrators and data
analysts do not know the group assignment
TU/e Colloquium 2013
37/41
Ghent University & Eindhoven University of Technology
jan.claes@ugent.be – www.janclaes.info
Experiments
Make the same exercise under the same
conditions
 Keep as much variables constant as possible
 Try to control for the others
 Literally the only difference between the two
groups should be the treatment
TU/e Colloquium 2013
38/41
Ghent University & Eindhoven University of Technology
jan.claes@ugent.be – www.janclaes.info
Participants for comparative experiment
 High school students? OK!
 Undergraduates? OK!
 Experienced modelers? OK!
 Non human movie characters? Not OK!
 Cognitive processes in the human mind
Experiments
Participants for comparative experiment
 High school students?
 Undergraduates?
 Experienced modelers?
 Non human movie characters?
TU/e Colloquium 2013
39/41
Ghent University & Eindhoven University of Technology
jan.claes@ugent.be – www.janclaes.info
Experiments with undergraduates
When use students?
 For confirmative, comparative experiments
 No reason to believe that effect is different
(general human cognitive processes)
 Very homogeneous group, large groups
When not to use students?
 For explanatory, observational experiments
 Use real modelers with varying levels of experience
 Representative sample
TU/e Colloquium 2013
40/41
Ghent University & Eindhoven University of Technology
jan.claes@ugent.be – www.janclaes.info
Experiments
Conclusion 3
Use undergraduate students in experiments?
Sometimes no, sometimes yes!
ComparativeExplanatory
TU/e Colloquium 2013
41/41
Ghent University & Eindhoven University of Technology
jan.claes@ugent.be – www.janclaes.info
Contact information
Jan Claes
jan.claes@ugent.be
http://www.janclaes.info
Twitter: @janclaesbelgium
Thanks for your attention!
Do you have feedback on my research plans?
> explanatory theories <
> behavioral science <
> methodology <
> evaluation <