Conformance Checking: Difference between revisions

From QPR ProcessAnalyzer Wiki
Jump to navigation Jump to search
 
(25 intermediate revisions by the same user not shown)
Line 1: Line 1:
This functionality is available in QPR ProcessAnalyzer 2018.3.
== Introduction to Conformance Checking ==
Conformance checking compares a process mining model (i.e., eventlog) to a design model defined using BPMN, and finds which cases don't conform to the design model and which are the actual violations. In the process mining model, conformance can be tested for cases or variations, and each case or variation is either considered as conformant or nonconformant. For nonconformant cases/variations, the conformance checking is also able to identify one or several reasons for the nonconformance. For example the case or variation might contain events that are not in the design model, or the sequence of events doesn't correspond to the defined sequence in the design model.


== Introduction to Conformance Checking ==
The design model can be drawn in a way that the same task (event type name) appears multiple times in the design model. Sometimes its easier to create the design model when each variation/path is drawn as an own sequence (separated by exclusive gateways). In this approach, same tasks may appear multiple time in the design model. Alternatively, the design model may be drawn using more of the gateways to avoid using same tasks multiple times. From the conformance checking viewpoint, both approaches are suitable and it's more a matter of taste which design model is easier to read.
The conformance checking functionality compares the QPR ProcessAnalyzer process mining model with a design model defined using BPMN. In the process mining model, conformance can be tested for Cases or Variations. Each Case or Variation is either considered as conformant or nonconformant. For nonconformant Case or Variation, the conformance checking is also able to identify one or several reasons for the nonconformance. For example the Case or Variation might contain EventTypes that are not in the design model, or the sequence of Events doesn't correspond with the defined sequence in the design model.


There are functions ''AnalyzeConformance'' and ''IsConformant'' available for [[Object_Specific_Properties_and_Functions_in_QPR_ProcessAnalyzer_Expressions#Case|Case]] and [[Object_Specific_Properties_and_Functions_in_QPR_ProcessAnalyzer_Expressions#Variation|Variation]] objects. The IsConformant return true or false depending whether the Case or Variation is conformant or not with theh provided ConformanceModel. AnalyzeConformance function also return reasons for nonconformance for nonconforming Cases or Variations.
The conformance checking calculation is implemented by functions ''AnalyzeConformance'' and ''IsConformant'' that are available for [[QPR_ProcessAnalyzer_Objects_in_Expression_Language#Case|Case]] and [[QPR_ProcessAnalyzer_Objects_in_Expression_Language#Variation|Variation]] objects. In addition, the design model needs to be provided as a parameter. IsConformant function returns true or false depending whether the case or variation is conformant or not. AnalyzeConformance function also return reasons for nonconformance for nonconforming cases or variations.


There are settings ''IgnoreEventTypesMissingInModel'' and ''IgnoreIncompleteCases'' which affect how the conformance checking works. If ''IgnoreEventTypesMissingInModel'' is ''false'' and there are EventTypes in the Case that don't exist in the conformance model (comparing using EventType names), the path does not conform with the model. Then the ''InvalidEventType'' is mentioned as a nonconformance reason.
There are settings ''IgnoreEventTypesMissingInModel'' and ''IgnoreIncompleteCases'' affecting how the conformance checking is done. If ''IgnoreEventTypesMissingInModel'' is ''false'' and there are events in the case that don't exist in the conformance model (comparing using event type names), the path doesn't conform with the model. Then the ''InvalidEventType'' is mentioned as a nonconformance reason.


The conformance checking works as follows:
The conformance checking works as follows:
* The sequence of Events in the checked Case or Variation is traversed one Event at a time as follows:
* The sequence of events in the checked case or variation is traversed one event at a time as follows:
** If the checked EventType does not exist in the design model and ''IgnoreEventTypesMissingInModel'' is set to ''true'', move to the next EventType.
** If the checked event does not exist in the design model and ''IgnoreEventTypesMissingInModel'' is set to ''true'', move to the next event.
** If the checked EventType exists in the design model, find next ConformanceModelTask whose name matches with the EventType name.
** If the checked event exists in the design model, find next ConformanceModelTask whose name matches with the event type name.
** If such ConformanceModelTask is not found, the tested Case or Variation does not conform with the design model. ''InvalidStateChange'' is added as a nonconformance reason.
** If such ConformanceModelTask is not found, the tested case or variation does not conform with the design model. ''InvalidStateChange'' is added as a nonconformance reason.
* After the iteration, if ''IgnoreIncompleteCases'' is ''false'' and the current state in the design is such that any end Event is not reachable without traversing through any additional ConformanceModelTasks, the tested Case or Variation does not conform with the design model. ''NotCompleted'' will be added as nonconformance reason.
* After the iteration, if ''IgnoreIncompleteCases'' is ''false'' and the current state in the design is such that any end event is not reachable without traversing through any additional ConformanceModelTasks, the tested case or variation does not conform to the design model. ''NotCompleted'' will be added as nonconformance reason.


== Conformance Checking Settings ==
== Conformance Checking Settings ==
The following settings are supported by the conformance checking, affecting how the conformance checking works:
The following settings are supported by the conformance checking, affecting the conformance checking behavior:


{| class="wikitable"
{| class="wikitable"
Line 24: Line 24:
||IgnoreEventTypesMissingInModel
||IgnoreEventTypesMissingInModel
||
||
Defines whether the check should ignore any EventTypes that are present in the Case or Variation but not in the design model. Defined as a boolean value (''true'' or ''false''). Default value is ''false''. When ''true'', unknown EventTypes are filtered out before performing conformance check. Defined as a boolean value (''true'' or ''false'').
Defines whether the check should ignore any EventTypes that are present in the case or variation but not in the design model. Defined as a boolean value (''true'' or ''false''). Default value is ''false''. When ''true'', unknown EventTypes are filtered out before performing conformance check. Defined as a boolean value (''true'' or ''false'').
|-
|-
||IgnoreIncompleteCases
||IgnoreIncompleteCases
||
||
Defines whether the check should count unfinished (incomplete) Cases or Variations as conformance violations. Unfinished cases are cases where the last event is reached, but the case should continue to a next event according to the design model. Defined as a boolean value (''true'' or ''false''). Default value is ''false''. When ''false'', incomplete Cases or Variations are counted as nonconformant. When there are pending Cases in the model, it might be useful to set this to ''true''.
Defines whether the check should count unfinished (incomplete) cases or variations as conformance violations. Unfinished cases are cases where the last event is reached, but the case should continue to a next event according to the design model. Defined as a boolean value (''true'' or ''false''). Default value is ''false''. When ''false'', incomplete cases or variations are counted as nonconformant. When there are pending cases in the model, it might be useful to set this to ''true''.
|}
|}


== Identified Reasons for Nonconformance ==
== Types of Nonconformance Reasons ==
The following reasons for nonconformance are identified for a nonconforming Case or Variation. The conformance checking may identify several reasons for a single Case or Variation.
The following reasons for nonconformance are identified for a nonconforming cases or variations. All the InvalidEventType reasons are identified for each case or variation, and for the NotCompleted and InvalidStateChange the first encountered reason in the case or variation is identified.


{| class="wikitable"
{| class="wikitable"
!'''Reason number'''
!'''Reason name'''
!'''Reason name'''
! '''Description'''
! '''Description'''
|-
|-
||1
||InvalidEventType
||InvalidEventType
||
||
Occurs when the tested Case or Variation has EventType names that do not exist in the design model (EventType names should match with the design model task names). This nonconformance reason can be ignored using the ''IgnoreEventTypesMissingInModel'' setting. As an additional information for this type of nonconformance, an array having the following elements will be added after the reason identifier: An array of all the invalid EventType names not found in the model that exist in the object being checked. Corresponding numerical value 1.
Occurs when the tested case or variation has events that do not exist in the design model (event type names should match with the design model task names). This nonconformance reason can be ignored using the ''IgnoreEventTypesMissingInModel'' setting. As an additional information for this type of nonconformance, there is a list of all the invalid EventType names not found in the design model, but occurring in the case or variation.
 
Example:
<pre>
[1, "Sales order changes", "Customer return", "Invoice cancelled"]
</pre>
 
|-
|-
||2
||NotCompleted
||NotCompleted
||Occurs when the Case or Variation ends (the last Event is reached), but according to the compared design model, the Case or Variation should continue, i.e. the end event in the design model is not reached. Checking of this nonconformance reason can be ignored using the ''IgnoreIncompleteCases'' setting. For this nonconformance reason, there is no additional information. Corresponding numerical value 2.
||Occurs when the case or variation ends (the last Event is reached), but according to the design model, the case or variation should continue. Checking this nonconformance reason can be ignored using the ''IgnoreIncompleteCases'' setting. As an additional information, the following elements will be added after the reason identifier:
* Zero based index of the last event.
* Type name of the last event.
 
Example:
<pre>
[2, 3, "Invoice sent"]
</pre>
|-
|-
||3
||InvalidStateChange
||InvalidStateChange
||
||
Occurs when the event sequence in a Case is not possible according to the design model due to missing connections between successive tasks. Corresponding numerical value 3. As an additional information for this type of nonconformance, an array having the following elements will be added after the reason identifier:
Occurs when the event sequence in a case or Variation is not possible according to flow of tasks described by the design model. When this type of nonconformance reason is detected, the conformance check doesn't continue, so there may be further reasons in the case causing nonconformance. Note that this reason doesn't show those invalid transitions, where the target event type is invalid (i.e., is listed as the InvalidEventType reason). As an additional information, the following elements will be added after the reason identifier:
* Index of the first EventType name that was not possible to reach in the design model.
* Zero based index of the detected nonconformant event.
* Name of the previous EventType or null if this was the first event type name in the Case or Variation.
* Name of the previous EventType or null if this was the first event type name in the Case or Variation.
* Name of the nonconformant EventType.
* Name of the nonconformant EventType.
Example:
<pre>
[3, 5, "Invoice sent", "Sales order changed"]
</pre>
|}
|}


== Supported BPMN Elements and Attributes ==
== Supported BPMN Elements and Attributes ==
When creating a design model from a BPMN 2.0 XML file (spec: http://www.omg.org/spec/BPMN/2.0/), the below listed elements are supported. If the file contains any other types, an error is caused when trying to create the design model.
When creating a design model from a BPMN 2.0 XML file (specification: http://www.omg.org/spec/BPMN/2.0/), the below listed elements are supported. If the file contains any other types, an error is caused when trying to create the design model.


{| class="wikitable"
{| class="wikitable"
Line 103: Line 125:
* id
* id
||
||
|}
== Creating Design Model ==
{| class="wikitable"
!'''Function'''
!'''Parameters'''
! '''Description'''
|-
||DesignModelFromXml
||
BPMN 2.0 XML document (String)
||
Creates new ([[Conformance_Checking#DesignModel|DesignModel]] object) from BPMN 2.0 XML document provided as a string. (BPMN 2.0 specification: https://www.omg.org/spec/BPMN/).
In the BPMN 2.0 XML file, the ''<nowiki>http://www.omg.org/spec/BPMN/20100524/DI</nowiki>'' section is not used by the function, and thus it can be omitted to reduce transferred data.
Note that the quotation marks need to be escaped if the BPMN 2.0 XML is provided as a literal in the expression.
Example:
<pre>
let myConformanceModel = DesignModelFromXml(
  "<?xml version=\"1.0\" encoding=\"UTF-8\"?>
    <bpmn:definitions xmlns:xsi=\"http://www.w3.org/2001/XMLSchema-instance\" xmlns:bpmn=\"http://www.omg.org/spec/BPMN/20100524/MODEL\" xmlns:bpmndi=\"http://www.omg.org/spec/BPMN/20100524/DI\">
    <bpmn:process id=\"Process_1\" isExecutable=\"false\">
    <bpmn:startEvent id=\"StartEvent_1\" />
    <bpmn:task id=\"Task_0y5w837\" name=\"abc\">
      <bpmn:outgoing>SequenceFlow_10mpkws</bpmn:outgoing>
    </bpmn:task>
    <bpmn:task id=\"Task_0c16r07\" name=\"def\">
      <bpmn:incoming>SequenceFlow_10mpkws</bpmn:incoming>
    </bpmn:task>
    <bpmn:sequenceFlow id=\"SequenceFlow_10mpkws\" sourceRef=\"Task_0y5w837\" targetRef=\"Task_0c16r07\" />
  </bpmn:process>
  <bpmndi:BPMNDiagram id=\"BPMNDiagram_1\">
    ...
  </bpmndi:BPMNDiagram>
</bpmn:definitions>");
</pre>
|}
|}


Line 129: Line 190:
||StartEvents (DesignModelEvent*)
||StartEvents (DesignModelEvent*)
||Returns an array of all the top-level start DesignModelEvents in the design model.
||Returns an array of all the top-level start DesignModelEvents in the design model.
|-
||Tasks (DesignModelTask*)
||Returns an array of all Tasks in the design model.
|}
|}


Line 212: Line 276:
|}
|}


[[Category: QPR UI]]
[[Category: QPR ProcessAnalyzer]]

Latest revision as of 13:06, 17 April 2023

Introduction to Conformance Checking

Conformance checking compares a process mining model (i.e., eventlog) to a design model defined using BPMN, and finds which cases don't conform to the design model and which are the actual violations. In the process mining model, conformance can be tested for cases or variations, and each case or variation is either considered as conformant or nonconformant. For nonconformant cases/variations, the conformance checking is also able to identify one or several reasons for the nonconformance. For example the case or variation might contain events that are not in the design model, or the sequence of events doesn't correspond to the defined sequence in the design model.

The design model can be drawn in a way that the same task (event type name) appears multiple times in the design model. Sometimes its easier to create the design model when each variation/path is drawn as an own sequence (separated by exclusive gateways). In this approach, same tasks may appear multiple time in the design model. Alternatively, the design model may be drawn using more of the gateways to avoid using same tasks multiple times. From the conformance checking viewpoint, both approaches are suitable and it's more a matter of taste which design model is easier to read.

The conformance checking calculation is implemented by functions AnalyzeConformance and IsConformant that are available for Case and Variation objects. In addition, the design model needs to be provided as a parameter. IsConformant function returns true or false depending whether the case or variation is conformant or not. AnalyzeConformance function also return reasons for nonconformance for nonconforming cases or variations.

There are settings IgnoreEventTypesMissingInModel and IgnoreIncompleteCases affecting how the conformance checking is done. If IgnoreEventTypesMissingInModel is false and there are events in the case that don't exist in the conformance model (comparing using event type names), the path doesn't conform with the model. Then the InvalidEventType is mentioned as a nonconformance reason.

The conformance checking works as follows:

  • The sequence of events in the checked case or variation is traversed one event at a time as follows:
    • If the checked event does not exist in the design model and IgnoreEventTypesMissingInModel is set to true, move to the next event.
    • If the checked event exists in the design model, find next ConformanceModelTask whose name matches with the event type name.
    • If such ConformanceModelTask is not found, the tested case or variation does not conform with the design model. InvalidStateChange is added as a nonconformance reason.
  • After the iteration, if IgnoreIncompleteCases is false and the current state in the design is such that any end event is not reachable without traversing through any additional ConformanceModelTasks, the tested case or variation does not conform to the design model. NotCompleted will be added as nonconformance reason.

Conformance Checking Settings

The following settings are supported by the conformance checking, affecting the conformance checking behavior:

Setting name Description
IgnoreEventTypesMissingInModel

Defines whether the check should ignore any EventTypes that are present in the case or variation but not in the design model. Defined as a boolean value (true or false). Default value is false. When true, unknown EventTypes are filtered out before performing conformance check. Defined as a boolean value (true or false).

IgnoreIncompleteCases

Defines whether the check should count unfinished (incomplete) cases or variations as conformance violations. Unfinished cases are cases where the last event is reached, but the case should continue to a next event according to the design model. Defined as a boolean value (true or false). Default value is false. When false, incomplete cases or variations are counted as nonconformant. When there are pending cases in the model, it might be useful to set this to true.

Types of Nonconformance Reasons

The following reasons for nonconformance are identified for a nonconforming cases or variations. All the InvalidEventType reasons are identified for each case or variation, and for the NotCompleted and InvalidStateChange the first encountered reason in the case or variation is identified.

Reason number Reason name Description
1 InvalidEventType

Occurs when the tested case or variation has events that do not exist in the design model (event type names should match with the design model task names). This nonconformance reason can be ignored using the IgnoreEventTypesMissingInModel setting. As an additional information for this type of nonconformance, there is a list of all the invalid EventType names not found in the design model, but occurring in the case or variation.

Example: 

[1, "Sales order changes", "Customer return", "Invoice cancelled"]
2 NotCompleted Occurs when the case or variation ends (the last Event is reached), but according to the design model, the case or variation should continue. Checking this nonconformance reason can be ignored using the IgnoreIncompleteCases setting. As an additional information, the following elements will be added after the reason identifier:
  • Zero based index of the last event.
  • Type name of the last event.

Example: 

[2, 3, "Invoice sent"]
3 InvalidStateChange

Occurs when the event sequence in a case or Variation is not possible according to flow of tasks described by the design model. When this type of nonconformance reason is detected, the conformance check doesn't continue, so there may be further reasons in the case causing nonconformance. Note that this reason doesn't show those invalid transitions, where the target event type is invalid (i.e., is listed as the InvalidEventType reason). As an additional information, the following elements will be added after the reason identifier:

  • Zero based index of the detected nonconformant event.
  • Name of the previous EventType or null if this was the first event type name in the Case or Variation.
  • Name of the nonconformant EventType.

Example: 

[3, 5, "Invoice sent", "Sales order changed"]

Supported BPMN Elements and Attributes

When creating a design model from a BPMN 2.0 XML file (specification: http://www.omg.org/spec/BPMN/2.0/), the below listed elements are supported. If the file contains any other types, an error is caused when trying to create the design model.

Element type Supported subtypes Used attributes Supported subentities
Flow
  • sequenceFlow
  • id
  • sourceRef
  • targetRef
Events
  • startEvent
  • endEvent
  • id
Tasks
  • sendTask
  • receiveTask
  • userTask
  • manualTask
  • businessRuleTask
  • serviceTask
  • scriptTask
  • id
  • name
  • standardLoopCharacteristics
  • multiInstanceLoopCharacteristics (with and without isSequential attribute)
Gateway
  • parallelGateway
  • exclusiveGateway
  • inclusiveGateway
  • id

Creating Design Model

Function Parameters Description
DesignModelFromXml

BPMN 2.0 XML document (String)

Creates new (DesignModel object) from BPMN 2.0 XML document provided as a string. (BPMN 2.0 specification: https://www.omg.org/spec/BPMN/).

In the BPMN 2.0 XML file, the http://www.omg.org/spec/BPMN/20100524/DI section is not used by the function, and thus it can be omitted to reduce transferred data.

Note that the quotation marks need to be escaped if the BPMN 2.0 XML is provided as a literal in the expression.

Example: 

let myConformanceModel = DesignModelFromXml(
  "<?xml version=\"1.0\" encoding=\"UTF-8\"?>
    <bpmn:definitions xmlns:xsi=\"http://www.w3.org/2001/XMLSchema-instance\" xmlns:bpmn=\"http://www.omg.org/spec/BPMN/20100524/MODEL\" xmlns:bpmndi=\"http://www.omg.org/spec/BPMN/20100524/DI\">
    <bpmn:process id=\"Process_1\" isExecutable=\"false\">
    <bpmn:startEvent id=\"StartEvent_1\" />
    <bpmn:task id=\"Task_0y5w837\" name=\"abc\">
      <bpmn:outgoing>SequenceFlow_10mpkws</bpmn:outgoing>
    </bpmn:task>
    <bpmn:task id=\"Task_0c16r07\" name=\"def\">
      <bpmn:incoming>SequenceFlow_10mpkws</bpmn:incoming>
    </bpmn:task>
    <bpmn:sequenceFlow id=\"SequenceFlow_10mpkws\" sourceRef=\"Task_0y5w837\" targetRef=\"Task_0c16r07\" />
  </bpmn:process>
  <bpmndi:BPMNDiagram id=\"BPMNDiagram_1\">
    ...
  </bpmndi:BPMNDiagram>
</bpmn:definitions>");

Design Model Objects

The design model is is a specific entity DesignModel in the expression language. There are also entities DesignModelFlow, DesignModelNode, DesignModelEvent, DesignModelGateway and DesignModelTask.

DesignModel

DesignModel is an object representing the whole model consisting of nodes and flows. A new design model can be created from a BPMN 2.0 XML file.

DesignModel properties Description
Id (Integer) Returns an unique id for the design model. The id is assigned automatically when a new DesignModel is created.
EndEvents (DesignModelEvent*) Returns an array of all the top-level end DesignModelEvents in the design model.
Flows (DesignModelFlow*) Returns an array of all the top-level DesignModelFlows in the design model.
Nodes (DesignModelNode*) Returns an array of all the top-level DesignModelNodes in the design model.
StartEvents (DesignModelEvent*) Returns an array of all the top-level start DesignModelEvents in the design model.
Tasks (DesignModelTask*) Returns an array of all Tasks in the design model.

DesignModelFlow

DesignModelFlow connects two DesignModelNodes, DesignModelGateways or DesignModelEvents in a DesignModel.

DesignModelFlow properties Description
Id (Integer) Id of the DesignModelFlow.
From (DesignModelNode) Starting node of the DesignModelFlow.
To (DesignModelNode) Ending node of the DesignModelFlow.
Type (String) Type name of the DesignModelFlow SequenceFlow.

DesignModelNode

DesignModelNode is an upper level type defining common properties and functions available for its subtypes. Available subtypes are DesignModelEvent, DesignModelGateway and DesignModelTask.

DesignModelFlow properties Description
Id (Integer) Id of the node. The id is assigned automatically when a new DesignModelNode is created.
IncomingFlows (DesignModelFlow*) Array of incoming DesignModelFlows to this DesignModelNode.
Name (String) Name of the node.
OutgoingFlows (DesignModelFlow*) Array of outgoing DesignModelFlows to this DesignModelNode.
Type (String) Type of the node. Allowed values are Event, Task and Gateway.

DesignModelEvent

DesignModelEvent represents start and end events. It's inherited from DesignModelNode getting all DesignModelNode's properties and functions.

DesignModelEvent properties Description
EventType (String) Type of the event. Allowed values are Start and End.

DesignModelGateway

DesignModelGateway represents a gateway in the DesignModel. It's inherited from DesignModelNode getting all DesignModelNode's properties and functions.

DesignModelGateway properties Description
GatewayType (String) Type of the gateway. Allowed values are Parallel, Exclusive and Inclusive.
BranchType (String) Branching type of gateway. Allowed values are Fork and Join.

DesignModelTask

DesignModelTask represents tasks/activities in the DesignModel. Corresponding object in the process mining model is EventType. When running conformance analysis, DesignModelTask name is matched with EventType name. DesignModelTask object is inherited from DesignModelNode object, getting all DesignModelNode's properties and functions. In addition, there are the following properties for DesignModelTasks:

DesignModelTask properties Description
LoopType (String) Type of looping allowed for the task. Allowed values are null (meaning there is no loop), StandardLoopCharacteristics, MultiInstanceLoopCharacteristics and MultiInstanceSequentialLoopCharacteristics.
TaskType (String) Type of the task. Allowed values are Task, Send, Receive, User, Manual, BusinessRule, Service and Script.
Retrieved from "https://wiki.onqpr.com/pa/index.php?title=Conformance_Checking&oldid=22788"