QPR ProcessAnalyzer Wiki - User contributions [en]

Create Predicted Eventlog

2026-01-20T14:15:37Z

MarHink:

This article has instructions how to install, configure and use eventlog predictions. The prediction creates a new model that contains the source model data and the predictions. It's able to predict case attributes for the generated new cases and event attributes for the predicted events. By default, predictions use [https://en.wikipedia.org/wiki/Transformer_(deep_learning) Transformer] neural network architecture. However, also [https://en.wikipedia.org/wiki/Long_short-term_memory LSTM] and [https://en.wikipedia.org/wiki/Gated_recurrent_unit GRU]-based architectures are supported.

To distinguish the real (source data) and predicted events and cases, there are following attributes in the model:
* Event attribute '''Predicted''' denotes whether the event is from the source data (''false'') or whether it's predicted (''true'').
* Case attribute '''Generated''' denotes whether the case is in the source data (''false'') or whether the prediction generated it as a new case (''true'').

== Prerequisites for prediction ==
Following prerequisites need to be fulfilled to run the eventlog prediction:
* QPR ProcessAnalyzer 2024.8 or later in use
* Snowflake connection is configured
* Source models are stored to Snowflake

== Install prediction to Snowflake ==
To install the eventlog prediction to Snowflake:
# Go to Snowflake, and create a Snowflake-managed stage with name '''DECISION_INTELLIGENCE''' to the same schema configured to QPR ProcessAnalyzer (in the Snowflake connection string). Use settings in the following image: [[File:Create_Snowflake_stage.png]]
# Open the created stage and upload the '''predict.pyz''' file into the stage (ask the file from your QPR representative).
# Create the following procedure to the same schema:
<syntaxhighlight lang="sql">
CREATE OR REPLACE PROCEDURE QPRPA_SP_PREDICTION("CONFIGURATION" OBJECT)
RETURNS OBJECT
LANGUAGE PYTHON
STRICT
RUNTIME_VERSION = '3.11'
PACKAGES = ('nltk','numpy','networkx','pandas','scikit-learn','snowflake-snowpark-python','tensorflow==2.12.0','dill','psutil','prophet','holidays','python-kubernetes','docker-py','cryptography')
HANDLER = 'main'
EXECUTE AS OWNER
AS '
import sys
def main(session, parameters_in: dict) -> dict:
session.file.get(''@decision_intelligence/predict.pyz'', ''/tmp'')
sys.path.append(''/tmp/predict.pyz'')
import predict
return predict.main(session, parameters_in)
';
</syntaxhighlight>

== Create prediction script in QPR ProcessAnalyzer ==
1. Create the following example expression script (e.g., with name '''Create prediction model'''):<syntaxhighlight lang="typescript">
let sourceModel = ProjectByName("<project name>").ModelByName("<model name>");
let completeCaseEventTypeNames = ["<event type name found only in complete cases>", "<another event type name>", "..."];

let targetProject = Project;
let eventTypeColumnName = sourceModel.EventsDataTable.ColumnMappings["EventType"];

_system.ML.GeneratePredictionModel(#{
"Name": "My prediction model", // Name of the PA model to generate to the target project.
"SourceModel": sourceModel, // Snowflake-based PA model used for training the prediction model.
"TargetProject": targetProject, // Target project to create the model into.
"TrainingConfiguration": #{ // Training parameters.
"num_epochs_to_train": 200
},
"GenerationConfiguration": #{ // Model generation parameters.
"cases_to_generate": 1000
},
"TrainingDataFilter": #{
"Items": [
#{
"Type": "IncludeCases",
"Items": [
#{
"Type": "EventAttributeValue",
"Attribute": eventTypeColumnName,
"Values": completeCaseEventTypeNames
}
]
}
]
},
"IncompleteCasesFilter": #{
"Items": [
#{
"Type": "ExcludeCases",
"Items": [
#{
"Type": "EventAttributeValue",
"Attribute": eventTypeColumnName,
"Values": completeCaseEventTypeNames
}
]
}
]
},
"RecreatePredictionModel": true, // Should a prediction model be overwritten if one already exists for this source model and target model name combination.
"TrainingCaseSampleSize": 10000 // Maximum number of cases to use from the source model (random sampled).
});
</syntaxhighlight>2. Configure prediction for the previously created script as instructed in the next chapter. At minimum, replace the tags listed below with some suitable values:
* '''<project name>''': Name of the project in which the source model is located.
* '''<model name>''': Name of the model to be used as source model. This data in this source model will be used to train the prediction model so that it can generate new cases and continuations for incomplete existing cases.
* '''<event type name found only in complete cases>''': This example script has been hard-coded to determine whether a case is complete or incomplete based on the existence of this event type.

== Configure prediction ==
Prediction script has the following settings in the GeneratePredictionModel call:
* '''Name''': Name of the QPR ProcessAnalyzer model that is created to the target project. The model will contain the source model content and the predictions.
* '''SourceModel''': Source model for which the prediction is made. Model can be selected for example based on id with ModelById function or by name with ModelByName function.
* '''TargetProject''': Target project to create the new model into.
* '''RecreatePredictionModel''': When ''true'', a new ML model is trained when the script is run. When ''false'', the prediction is run using possibly pre-existing ML model.
* '''TrainingConfiguration''': Training parameters.
** '''attributes''': Attribute configurations (for more information, see the chapter below).
** '''generate_start_time_trend_images''': If set to true, two images will be generated for each cross validated Prophet-parameter combination and also for the final selected parameters showing the results of plot and plot_components-functions.
*** The images will be generated into stage files with the following path names:
**** plot: @decision_intelligence_testing/{model_name}_st_RMSE={rmse_value or "final"}.png
****plot_components: @decision_intelligence_testing/{model_name}_st_RMSE={rmse_value or "final"}_comp.png
***The default value is false.
** '''max_num_case_clusters''': Set the maximum number of clusters to divide the case attribute values into.
*** The default value is 20.
** '''max_num_traces_in_training''': Set the maximum number of traces used in training.
*** When training, every case of length N will be split into N traces (a.k.a. prefixes) (p_1, ..., p_N), where p_x contains first x events of the all events of the full case.
**** If there are more traces available than this configured value, cases to include will be random sampled so that the maximum is exceeded by at most one case.
**** If null, all the traces will be used, no matter what (may easily lead to running out of memory).
**** The default value is 100000.
** '''num_epochs_to_train''': How many times the training set is used in training. The best performing model out of all the iterations will be selected.
*** The default value is 500.
** '''num_extra_years_to_reserve_in_created_model''': Number of additional years after the year of the last timestamp in the training data to reserve to the capacity of the created ML model, allowing the model to be able to predict timestamps in the range between the minimum timestamp year in the training data and the maximum timestamp year plus this value.
*** The default value is 20.
** '''reserve_extra_sequence_length''': How many extra events to reserve space for in the ML model compared to the number of events the longest case in the training data has.
*** The default value is 5.
** '''samples_per_epoch''': If not null, specifies (approximately) how many traces/prefixes will be used to represent one epoch of data in the training. The actual value used will be made divisible by batch_size using this formula:
***max(floor(samples_per_epoch / batch_size), 1) * batch_size
***If null, every epoch will use all the traces/prefixes in the training data.
***The default value is null
**'''validation_split''': Percentage of traces/prefixes to use to evaluate the loss and any model metrics at the end of each epoch. The model will not be trained on this data.
***If 0, separate validation data will not be used. Instead, all the training data will be used also as validation data.
***The default value is 0.
* '''GenerationConfiguration''': Event generation parameters. When null, no generation is done. For example, following parameters are supported:
** '''avoid_repeated_activities''': Array of activity names that should occur at most once in any case. The probability of selecting any of the activities specified in this configuration more than once is set to be 0.
*** Empty array means that activity generation is not restricted by this setting at all.
*** null value means that there should not be any activities that can occur more than once (shortcut for specifying all the activity names).
*** The default value is an empty array.
** '''cases_to_generate''': Maximum number cases to create. The number of created cases is further limited by the capabilities of the trained model and the ''case_generation_start_time'' and ''case_generation_end_time'' parameters.
*** The default value is such that the number of cases, by itself, is not limited.
** '''case_generation_start_time''': If defined, new cases will be generated after this timestamp (given as string in ISO datetime format).
*** If undefined, the latest start event timestamp used in the training data is used.
*** The default value is undefined.
** '''case_generation_end_time''': If defined, new events and cases will not be generated after this timestamp (given as string in ISO datetime format). E.g., "2015-01-01T00:00:00".
*** The default value is unlimited (only limit comes from the capacity of the trained model)
** '''generate_debug_event_attributes''':
*** If true, additional columns will be added containing, e.g., probabilities of the selected activity and other activities.
*** The default value is false.
** '''max_num_events''':
*** Specifies the maximum number of events to generate for any case.
*** If unspecified (=default), the value equals to ''<the maximum number of events in any case in the training data>''+''<the value of reserve_extra_sequence_length in training>''.
** '''min_prediction_probability ''':
*** The minimum probability of any prediction. If the probability of a prediction is lower than this, it will never be picked.
*** The default value is 0.01.
** '''temperature''':
*** If 0, the generated next activity will always be the one that is the most probable.
*** If 1, the generated next activity is purely based on the probabilities returned by the trained ML model.
*** This behavior is interpolated when using values between 0 and 1.
*** The default value is 0.9.
* '''TrainingDataFilter''': [[Filtering_in_QPR_ProcessAnalyzer_Queries|Filter]] to select specific cases that are used to train the prediction model. This filter is required to train the model only using the completed cases. Uncompleted cases should not be used for the training, so the model doesn't incorrectly learn that cases should end like that.
* '''IncompleteCasesFilter''': Optional [[Filtering_in_QPR_ProcessAnalyzer_Queries|filter]] to select which cases the prediction is made for. To improve performance of the prediction, it's recommended to include only the incomplete cases for which new events might appear, and skip the completed cases for which new events are not expected anymore.
* '''TrainingCaseSampleSize''': Maximum number of cases to take from the source model (cases are selected randomly). Use a lower setting to speed up the ML model training. The greater the value, the more subtle phenomena the prediction can learn from the data.

== Attribute configuration ==
Attribute configuration is used in '''TrainingConfiguration''' (see the chapter above) to configure which event- and case attributes should be used in prediction model and how they are used.

The configuration is in the top level split into two sections: "event" and "case". "Event" is used to configure event attributes, whereas "case" is used for case attributes.

The next level supports one value: "input".

The next level after that, supports the following settings:

* '''categorical_groups''': An array of categorical attribute group configuration objects used to define groups of attributes that will be bundled together in the trained model, either as separate input- or output features. Each attribute group will form its own input- or output vector used in the model training and generation.
** If null, only one group will be created with all the available categorical attributes included.
** The following settings are supported by these objects:
*** '''attributes''': An array of attribute names.
**** If null, all the input attributes are to be included in this group.
*** '''max_num_clusters''': The maximum number of clusters (input- or output vector feature values) to use to represent this group of attributes.
**** Default value: 20
**** NOTE: Clustering is used by default to convert a set of attribute values into an input- or output vector used by the prediction model.
*** '''ignore_values_threshold:''' The minimum percentage of objects having a specific attribute value in order for that attribute value to be taken into account as unique attribute value within this categorical group.
**** Depending on the context, the default value is any one of the following configurations:
***** ignore_values_threshold_for_case_attribute_values
****** Used when clustering case attributes when generating attribute values for generated new cases.
****** Default value is 0.01.
***** ignore_values_threshold_for_case_attributes
****** Used when clustering case attributes.
****** Default value is 0.1.
***** ignore_values_threshold_for_event_attributes
****** Used when clustering event attributes.
****** Default value is 0.1.
* '''columns''': An array of attribute column configuration objects used to define columns in the input data that are to be used as event- or case attributes.
** If null, all the columns will be included as categorical attributes (except case id, event type (only for event) and timestamp (only for event) columns).
** The following settings are supported by these objects:
*** '''label''': Column name.
*** '''type''': Type of the column. Supported types are:
**** '''categorical''': Values can take on one of a limited, and usually fixed, number of possible values.
**** '''numeric''': Value is considered as a continuous numeric value.

==== Example ====
Use all event attributes as input for the prediction model. In addition, additional machine learning input vector for SAP_User-event data column supporting at most 10 unique values.

In addition, for case attributes, only "Region", "Product Group", "Account Manager" and "Customer Group" case data columns are used as categorical attributes and "Cost" as numeric attribute. Furthermore, the four categorical case attributes are grouped into three groups, each of which are used as its own input vector for the prediction model.

When generating, all event attributes will be included for generated events as columns. Generated cases will have only "Region", "Product Group", "Account Manager", "Customer Group", and "Cost" columns.<syntaxhighlight lang="json">
"attributes": #{
"event": #{
"input": #{
"categorical_groups": [
#{
"attributes": None
},
#{
"attributes": ["SAP_User"],
"max_num_clusters": 10
}
],
"columns": None
}
},
"case": #{
"input": #{
"categorical_groups": [#{
"attributes": ["Account Manager"]
}, #{
"attributes": ["Customer Group"]
}, #{
"attributes": ["Region", "Product Group"]
}],
"columns": [
#{ "label": "Region", "type": "categorical" },
#{ "label": "Product Group", "type": "categorical" },
#{ "label": "Account Manager", "type": "categorical" },
#{ "label": "Customer Group", "type": "categorical" },
#{ "label": "Cost", "type": "numeric" }
]
}
}
}
</syntaxhighlight>

== Predicting case attribute values ==
QPR ProcessAnalyzer can also be used to, e.g., predict the final values of case attributes of running cases. The following script gives an example on how to perform this.<syntaxhighlight lang="typescript">
let sourceModel = ProjectByName("<project name>").ModelByName("<model name>");
let caseAttributeToPredict = "<name of the case attribute>";
let resultModelName = "<name of the model to be created/replaced>";
let generateDebugCaseAttributes = false; // Set to true to generate columns for prediction probabilities.
let casesToPredictFilter = "<JSON filter for cases for which the prediction is to be performed>";
let casesToUseForTrainingFilter = "<JSON filter for cases to be used for ML model training>";

let targetProject = Project;

_system.ML.GenerateCaseAttributePredictionModel(#{
"Name": resultModelName, // Name of the PA model to generate ti the target project.
"SourceModel": sourceModel, // Snowflake-based PA model used for training the prediction model.
"TargetProject": targetProject, // Target project to create the model into.
"RecreatePredictionModel": false, // Should a prediction model be overwritten if one already exists for this source model and target model name combination.
"TrainingCaseSampleSize": 10000, // Maximum number of cases to use from the source model (random sampled).
"CommonConfiguration": #{ // Common parameters used by both training and generation.
"output_case_attribute_groups": [#{
"attributes": [caseAttributeToPredict] // Attribute whose value is to be predicted.
}]
},
"TrainingConfiguration": #{ // Training parameters.
"max_num_case_attribute_clusters": 80,
"num_epochs_to_train": 100
},
"GenerationConfiguration": #{ // Case attribute generation parameters.
"generate_debug_case_attributes": generateDebugCaseAttributes // Should probability and probability_all-columns be generated as well as the actual prediction created into a new column named Predicted_<attribute name>
},
"TrainingDataFilter": ParseJson(casesToUseForTrainingFilter), // Filter JSON for events to be used for training.
"IncompleteCasesFilter": ParseJson(casesToPredictFilter) // Filter JSON for events for whose case attribute value is to be predicted.
});

</syntaxhighlight>

Create Predicted Eventlog

2026-01-15T11:28:22Z

MarHink: /* Attribute configuration */

This article has instructions how to install, configure and use eventlog predictions. The prediction creates a new model that contains the source model data and the predictions. It's able to predict case attributes for the generated new cases and event attributes for the predicted events. To distinguish the real (source data) and predicted events and cases, there are following attributes in the model:
* Event attribute '''Predicted''' denotes whether the event is from the source data (''false'') or whether it's predicted (''true'').
* Case attribute '''Generated''' denotes whether the case is in the source data (''false'') or whether the prediction generated it as a new case (''true'').

== Prerequisites for prediction ==
Following prerequisites need to be fulfilled to run the eventlog prediction:
* QPR ProcessAnalyzer 2024.8 or later in use
* Snowflake connection is configured
* Source models are stored to Snowflake

== Install prediction to Snowflake ==
To install the eventlog prediction to Snowflake:
# Go to Snowflake, and create a Snowflake-managed stage with name '''DECISION_INTELLIGENCE''' to the same schema configured to QPR ProcessAnalyzer (in the Snowflake connection string). Use settings in the following image: [[File:Create_Snowflake_stage.png]]
# Open the created stage and upload the '''predict.pyz''' file into the stage (ask the file from your QPR representative).
# Create the following procedure to the same schema:
<syntaxhighlight lang="sql">
CREATE OR REPLACE PROCEDURE QPRPA_SP_PREDICTION("CONFIGURATION" OBJECT)
RETURNS OBJECT
LANGUAGE PYTHON
STRICT
RUNTIME_VERSION = '3.11'
PACKAGES = ('nltk','numpy','networkx','pandas','scikit-learn','snowflake-snowpark-python','tensorflow==2.12.0','dill','psutil','prophet','holidays','python-kubernetes','docker-py','cryptography')
HANDLER = 'main'
EXECUTE AS OWNER
AS '
import sys
def main(session, parameters_in: dict) -> dict:
session.file.get(''@decision_intelligence/predict.pyz'', ''/tmp'')
sys.path.append(''/tmp/predict.pyz'')
import predict
return predict.main(session, parameters_in)
';
</syntaxhighlight>

== Create prediction script in QPR ProcessAnalyzer ==
1. Create the following example expression script (e.g., with name '''Create prediction model'''):<syntaxhighlight lang="typescript">
let sourceModel = ProjectByName("<project name>").ModelByName("<model name>");
let completeCaseEventTypeNames = ["<event type name found only in complete cases>", "<another event type name>", "..."];

let targetProject = Project;
let eventTypeColumnName = sourceModel.EventsDataTable.ColumnMappings["EventType"];

_system.ML.GeneratePredictionModel(#{
"Name": "My prediction model", // Name of the PA model to generate to the target project.
"SourceModel": sourceModel, // Snowflake-based PA model used for training the prediction model.
"TargetProject": targetProject, // Target project to create the model into.
"TrainingConfiguration": #{ // Training parameters.
"num_epochs_to_train": 200
},
"GenerationConfiguration": #{ // Model generation parameters.
"cases_to_generate": 1000
},
"TrainingDataFilter": #{
"Items": [
#{
"Type": "IncludeCases",
"Items": [
#{
"Type": "EventAttributeValue",
"Attribute": eventTypeColumnName,
"Values": completeCaseEventTypeNames
}
]
}
]
},
"IncompleteCasesFilter": #{
"Items": [
#{
"Type": "ExcludeCases",
"Items": [
#{
"Type": "EventAttributeValue",
"Attribute": eventTypeColumnName,
"Values": completeCaseEventTypeNames
}
]
}
]
},
"RecreatePredictionModel": true, // Should a prediction model be overwritten if one already exists for this source model and target model name combination.
"TrainingCaseSampleSize": 10000 // Maximum number of cases to use from the source model (random sampled).
});
</syntaxhighlight>2. Configure prediction for the previously created script as instructed in the next chapter. At minimum, replace the tags listed below with some suitable values:
* '''<project name>''': Name of the project in which the source model is located.
* '''<model name>''': Name of the model to be used as source model. This data in this source model will be used to train the prediction model so that it can generate new cases and continuations for incomplete existing cases.
* '''<event type name found only in complete cases>''': This example script has been hard-coded to determine whether a case is complete or incomplete based on the existence of this event type.

== Configure prediction ==
Prediction script has the following settings in the GeneratePredictionModel call:
* '''Name''': Name of the QPR ProcessAnalyzer model that is created to the target project. The model will contain the source model content and the predictions.
* '''SourceModel''': Source model for which the prediction is made. Model can be selected for example based on id with ModelById function or by name with ModelByName function.
* '''TargetProject''': Target project to create the new model into.
* '''RecreatePredictionModel''': When ''true'', a new ML model is trained when the script is run. When ''false'', the prediction is run using possibly pre-existing ML model.
* '''TrainingConfiguration''': Training parameters.
** '''attributes''': Attribute configurations (for more information, see the chapter below).
** '''generate_start_time_trend_images''': If set to true, two images will be generated for each cross validated Prophet-parameter combination and also for the final selected parameters showing the results of plot and plot_components-functions.
*** The images will be generated into stage files with the following path names:
**** plot: @decision_intelligence_testing/{model_name}_st_RMSE={rmse_value or "final"}.png
****plot_components: @decision_intelligence_testing/{model_name}_st_RMSE={rmse_value or "final"}_comp.png
***The default value is false.
** '''max_num_case_clusters''': Set the maximum number of clusters to divide the case attribute values into.
*** The default value is 20.
** '''max_num_traces_in_training''': Set the maximum number of traces used in training.
*** When training, every case of length N will be split into N traces (a.k.a. prefixes) (p_1, ..., p_N), where p_x contains first x events of the all events of the full case.
**** If there are more traces available than this configured value, cases to include will be random sampled so that the maximum is exceeded by at most one case.
**** If null, all the traces will be used, no matter what (may easily lead to running out of memory).
**** The default value is 100000.
** '''num_epochs_to_train''': How many times the training set is used in training. The best performing model out of all the iterations will be selected.
*** The default value is 500.
** '''num_extra_years_to_reserve_in_created_model''': Number of additional years after the year of the last timestamp in the training data to reserve to the capacity of the created ML model, allowing the model to be able to predict timestamps in the range between the minimum timestamp year in the training data and the maximum timestamp year plus this value.
*** The default value is 20.
** '''reserve_extra_sequence_length''': How many extra events to reserve space for in the ML model compared to the number of events the longest case in the training data has.
*** The default value is 5.
** '''samples_per_epoch''': If not null, specifies (approximately) how many traces/prefixes will be used to represent one epoch of data in the training. The actual value used will be made divisible by batch_size using this formula:
***max(floor(samples_per_epoch / batch_size), 1) * batch_size
***If null, every epoch will use all the traces/prefixes in the training data.
***The default value is null
**'''validation_split''': Percentage of traces/prefixes to use to evaluate the loss and any model metrics at the end of each epoch. The model will not be trained on this data.
***If 0, separate validation data will not be used. Instead, all the training data will be used also as validation data.
***The default value is 0.
* '''GenerationConfiguration''': Event generation parameters. When null, no generation is done. For example, following parameters are supported:
** '''avoid_repeated_activities''': Array of activity names that should occur at most once in any case. The probability of selecting any of the activities specified in this configuration more than once is set to be 0.
*** Empty array means that activity generation is not restricted by this setting at all.
*** null value means that there should not be any activities that can occur more than once (shortcut for specifying all the activity names).
*** The default value is an empty array.
** '''cases_to_generate''': Maximum number cases to create. The number of created cases is further limited by the capabilities of the trained model and the ''case_generation_start_time'' and ''case_generation_end_time'' parameters.
*** The default value is such that the number of cases, by itself, is not limited.
** '''case_generation_start_time''': If defined, new cases will be generated after this timestamp (given as string in ISO datetime format).
*** If undefined, the latest start event timestamp used in the training data is used.
*** The default value is undefined.
** '''case_generation_end_time''': If defined, new events and cases will not be generated after this timestamp (given as string in ISO datetime format). E.g., "2015-01-01T00:00:00".
*** The default value is unlimited (only limit comes from the capacity of the trained model)
** '''generate_debug_event_attributes''':
*** If true, additional columns will be added containing, e.g., probabilities of the selected activity and other activities.
*** The default value is false.
** '''max_num_events''':
*** Specifies the maximum number of events to generate for any case.
*** If unspecified (=default), the value equals to ''<the maximum number of events in any case in the training data>''+''<the value of reserve_extra_sequence_length in training>''.
** '''min_prediction_probability ''':
*** The minimum probability of any prediction. If the probability of a prediction is lower than this, it will never be picked.
*** The default value is 0.01.
** '''temperature''':
*** If 0, the generated next activity will always be the one that is the most probable.
*** If 1, the generated next activity is purely based on the probabilities returned by the trained ML model.
*** This behavior is interpolated when using values between 0 and 1.
*** The default value is 0.9.
* '''TrainingDataFilter''': [[Filtering_in_QPR_ProcessAnalyzer_Queries|Filter]] to select specific cases that are used to train the prediction model. This filter is required to train the model only using the completed cases. Uncompleted cases should not be used for the training, so the model doesn't incorrectly learn that cases should end like that.
* '''IncompleteCasesFilter''': Optional [[Filtering_in_QPR_ProcessAnalyzer_Queries|filter]] to select which cases the prediction is made for. To improve performance of the prediction, it's recommended to include only the incomplete cases for which new events might appear, and skip the completed cases for which new events are not expected anymore.
* '''TrainingCaseSampleSize''': Maximum number of cases to take from the source model (cases are selected randomly). Use a lower setting to speed up the ML model training. The greater the value, the more subtle phenomena the prediction can learn from the data.

== Attribute configuration ==
Attribute configuration is used in '''TrainingConfiguration''' (see the chapter above) to configure which event- and case attributes should be used in prediction model and how they are used.

The configuration is in the top level split into two sections: "event" and "case". "Event" is used to configure event attributes, whereas "case" is used for case attributes.

The next level supports one value: "input".

The next level after that, supports the following settings:

* '''categorical_groups''': An array of categorical attribute group configuration objects used to define groups of attributes that will be bundled together in the trained model, either as separate input- or output features. Each attribute group will form its own input- or output vector used in the model training and generation.
** If null, only one group will be created with all the available categorical attributes included.
** The following settings are supported by these objects:
*** '''attributes''': An array of attribute names.
**** If null, all the input attributes are to be included in this group.
*** '''max_num_clusters''': The maximum number of clusters (input- or output vector feature values) to use to represent this group of attributes.
**** Default value: 20
**** NOTE: Clustering is used by default to convert a set of attribute values into an input- or output vector used by the prediction model.
*** '''ignore_values_threshold:''' The minimum percentage of objects having a specific attribute value in order for that attribute value to be taken into account as unique attribute value within this categorical group.
**** Depending on the context, the default value is any one of the following configurations:
***** ignore_values_threshold_for_case_attribute_values
****** Used when clustering case attributes when generating attribute values for generated new cases.
****** Default value is 0.01.
***** ignore_values_threshold_for_case_attributes
****** Used when clustering case attributes.
****** Default value is 0.1.
***** ignore_values_threshold_for_event_attributes
****** Used when clustering event attributes.
****** Default value is 0.1.
* '''columns''': An array of attribute column configuration objects used to define columns in the input data that are to be used as event- or case attributes.
** If null, all the columns will be included as categorical attributes (except case id, event type (only for event) and timestamp (only for event) columns).
** The following settings are supported by these objects:
*** '''label''': Column name.
*** '''type''': Type of the column. Supported types are:
**** '''categorical''': Values can take on one of a limited, and usually fixed, number of possible values.
**** '''numeric''': Value is considered as a continuous numeric value.

==== Example ====
Use all event attributes as input for the prediction model. In addition, additional machine learning input vector for SAP_User-event data column supporting at most 10 unique values.

In addition, for case attributes, only "Region", "Product Group", "Account Manager" and "Customer Group" case data columns are used as categorical attributes and "Cost" as numeric attribute. Furthermore, the four categorical case attributes are grouped into three groups, each of which are used as its own input vector for the prediction model.

When generating, all event attributes will be included for generated events as columns. Generated cases will have only "Region", "Product Group", "Account Manager", "Customer Group", and "Cost" columns.<syntaxhighlight lang="json">
"attributes": #{
"event": #{
"input": #{
"categorical_groups": [
#{
"attributes": None
},
#{
"attributes": ["SAP_User"],
"max_num_clusters": 10
}
],
"columns": None
}
},
"case": #{
"input": #{
"categorical_groups": [#{
"attributes": ["Account Manager"]
}, #{
"attributes": ["Customer Group"]
}, #{
"attributes": ["Region", "Product Group"]
}],
"columns": [
#{ "label": "Region", "type": "categorical" },
#{ "label": "Product Group", "type": "categorical" },
#{ "label": "Account Manager", "type": "categorical" },
#{ "label": "Customer Group", "type": "categorical" },
#{ "label": "Cost", "type": "numeric" }
]
}
}
}
</syntaxhighlight>

== Predicting case attribute values ==
QPR ProcessAnalyzer can also be used to, e.g., predict the final values of case attributes of running cases. The following script gives an example on how to perform this.<syntaxhighlight lang="typescript">
let sourceModel = ProjectByName("<project name>").ModelByName("<model name>");
let caseAttributeToPredict = "<name of the case attribute>";
let resultModelName = "<name of the model to be created/replaced>";
let generateDebugCaseAttributes = false; // Set to true to generate columns for prediction probabilities.
let casesToPredictFilter = "<JSON filter for cases for which the prediction is to be performed>";
let casesToUseForTrainingFilter = "<JSON filter for cases to be used for ML model training>";

let targetProject = Project;

_system.ML.GenerateCaseAttributePredictionModel(#{
"Name": resultModelName, // Name of the PA model to generate ti the target project.
"SourceModel": sourceModel, // Snowflake-based PA model used for training the prediction model.
"TargetProject": targetProject, // Target project to create the model into.
"RecreatePredictionModel": false, // Should a prediction model be overwritten if one already exists for this source model and target model name combination.
"TrainingCaseSampleSize": 10000, // Maximum number of cases to use from the source model (random sampled).
"CommonConfiguration": #{ // Common parameters used by both training and generation.
"output_case_attribute_groups": [#{
"attributes": [caseAttributeToPredict] // Attribute whose value is to be predicted.
}]
},
"TrainingConfiguration": #{ // Training parameters.
"max_num_case_attribute_clusters": 80,
"num_epochs_to_train": 100
},
"GenerationConfiguration": #{ // Case attribute generation parameters.
"generate_debug_case_attributes": generateDebugCaseAttributes // Should probability and probability_all-columns be generated as well as the actual prediction created into a new column named Predicted_<attribute name>
},
"TrainingDataFilter": ParseJson(casesToUseForTrainingFilter), // Filter JSON for events to be used for training.
"IncompleteCasesFilter": ParseJson(casesToPredictFilter) // Filter JSON for events for whose case attribute value is to be predicted.
});

</syntaxhighlight>

Create Predicted Eventlog

2026-01-15T11:05:58Z

MarHink: /* Attribute configuration */

This article has instructions how to install, configure and use eventlog predictions. The prediction creates a new model that contains the source model data and the predictions. It's able to predict case attributes for the generated new cases and event attributes for the predicted events. To distinguish the real (source data) and predicted events and cases, there are following attributes in the model:
* Event attribute '''Predicted''' denotes whether the event is from the source data (''false'') or whether it's predicted (''true'').
* Case attribute '''Generated''' denotes whether the case is in the source data (''false'') or whether the prediction generated it as a new case (''true'').

== Prerequisites for prediction ==
Following prerequisites need to be fulfilled to run the eventlog prediction:
* QPR ProcessAnalyzer 2024.8 or later in use
* Snowflake connection is configured
* Source models are stored to Snowflake

== Install prediction to Snowflake ==
To install the eventlog prediction to Snowflake:
# Go to Snowflake, and create a Snowflake-managed stage with name '''DECISION_INTELLIGENCE''' to the same schema configured to QPR ProcessAnalyzer (in the Snowflake connection string). Use settings in the following image: [[File:Create_Snowflake_stage.png]]
# Open the created stage and upload the '''predict.pyz''' file into the stage (ask the file from your QPR representative).
# Create the following procedure to the same schema:
<syntaxhighlight lang="sql">
CREATE OR REPLACE PROCEDURE QPRPA_SP_PREDICTION("CONFIGURATION" OBJECT)
RETURNS OBJECT
LANGUAGE PYTHON
STRICT
RUNTIME_VERSION = '3.11'
PACKAGES = ('nltk','numpy','networkx','pandas','scikit-learn','snowflake-snowpark-python','tensorflow==2.12.0','dill','psutil','prophet','holidays','python-kubernetes','docker-py','cryptography')
HANDLER = 'main'
EXECUTE AS OWNER
AS '
import sys
def main(session, parameters_in: dict) -> dict:
session.file.get(''@decision_intelligence/predict.pyz'', ''/tmp'')
sys.path.append(''/tmp/predict.pyz'')
import predict
return predict.main(session, parameters_in)
';
</syntaxhighlight>

== Create prediction script in QPR ProcessAnalyzer ==
1. Create the following example expression script (e.g., with name '''Create prediction model'''):<syntaxhighlight lang="typescript">
let sourceModel = ProjectByName("<project name>").ModelByName("<model name>");
let completeCaseEventTypeNames = ["<event type name found only in complete cases>", "<another event type name>", "..."];

let targetProject = Project;
let eventTypeColumnName = sourceModel.EventsDataTable.ColumnMappings["EventType"];

_system.ML.GeneratePredictionModel(#{
"Name": "My prediction model", // Name of the PA model to generate to the target project.
"SourceModel": sourceModel, // Snowflake-based PA model used for training the prediction model.
"TargetProject": targetProject, // Target project to create the model into.
"TrainingConfiguration": #{ // Training parameters.
"num_epochs_to_train": 200
},
"GenerationConfiguration": #{ // Model generation parameters.
"cases_to_generate": 1000
},
"TrainingDataFilter": #{
"Items": [
#{
"Type": "IncludeCases",
"Items": [
#{
"Type": "EventAttributeValue",
"Attribute": eventTypeColumnName,
"Values": completeCaseEventTypeNames
}
]
}
]
},
"IncompleteCasesFilter": #{
"Items": [
#{
"Type": "ExcludeCases",
"Items": [
#{
"Type": "EventAttributeValue",
"Attribute": eventTypeColumnName,
"Values": completeCaseEventTypeNames
}
]
}
]
},
"RecreatePredictionModel": true, // Should a prediction model be overwritten if one already exists for this source model and target model name combination.
"TrainingCaseSampleSize": 10000 // Maximum number of cases to use from the source model (random sampled).
});
</syntaxhighlight>2. Configure prediction for the previously created script as instructed in the next chapter. At minimum, replace the tags listed below with some suitable values:
* '''<project name>''': Name of the project in which the source model is located.
* '''<model name>''': Name of the model to be used as source model. This data in this source model will be used to train the prediction model so that it can generate new cases and continuations for incomplete existing cases.
* '''<event type name found only in complete cases>''': This example script has been hard-coded to determine whether a case is complete or incomplete based on the existence of this event type.

== Configure prediction ==
Prediction script has the following settings in the GeneratePredictionModel call:
* '''Name''': Name of the QPR ProcessAnalyzer model that is created to the target project. The model will contain the source model content and the predictions.
* '''SourceModel''': Source model for which the prediction is made. Model can be selected for example based on id with ModelById function or by name with ModelByName function.
* '''TargetProject''': Target project to create the new model into.
* '''RecreatePredictionModel''': When ''true'', a new ML model is trained when the script is run. When ''false'', the prediction is run using possibly pre-existing ML model.
* '''TrainingConfiguration''': Training parameters.
** '''attributes''': Attribute configurations (for more information, see the chapter below).
** '''generate_start_time_trend_images''': If set to true, two images will be generated for each cross validated Prophet-parameter combination and also for the final selected parameters showing the results of plot and plot_components-functions.
*** The images will be generated into stage files with the following path names:
**** plot: @decision_intelligence_testing/{model_name}_st_RMSE={rmse_value or "final"}.png
****plot_components: @decision_intelligence_testing/{model_name}_st_RMSE={rmse_value or "final"}_comp.png
***The default value is false.
** '''max_num_case_clusters''': Set the maximum number of clusters to divide the case attribute values into.
*** The default value is 20.
** '''max_num_traces_in_training''': Set the maximum number of traces used in training.
*** When training, every case of length N will be split into N traces (a.k.a. prefixes) (p_1, ..., p_N), where p_x contains first x events of the all events of the full case.
**** If there are more traces available than this configured value, cases to include will be random sampled so that the maximum is exceeded by at most one case.
**** If null, all the traces will be used, no matter what (may easily lead to running out of memory).
**** The default value is 100000.
** '''num_epochs_to_train''': How many times the training set is used in training. The best performing model out of all the iterations will be selected.
*** The default value is 500.
** '''num_extra_years_to_reserve_in_created_model''': Number of additional years after the year of the last timestamp in the training data to reserve to the capacity of the created ML model, allowing the model to be able to predict timestamps in the range between the minimum timestamp year in the training data and the maximum timestamp year plus this value.
*** The default value is 20.
** '''reserve_extra_sequence_length''': How many extra events to reserve space for in the ML model compared to the number of events the longest case in the training data has.
*** The default value is 5.
** '''samples_per_epoch''': If not null, specifies (approximately) how many traces/prefixes will be used to represent one epoch of data in the training. The actual value used will be made divisible by batch_size using this formula:
***max(floor(samples_per_epoch / batch_size), 1) * batch_size
***If null, every epoch will use all the traces/prefixes in the training data.
***The default value is null
**'''validation_split''': Percentage of traces/prefixes to use to evaluate the loss and any model metrics at the end of each epoch. The model will not be trained on this data.
***If 0, separate validation data will not be used. Instead, all the training data will be used also as validation data.
***The default value is 0.
* '''GenerationConfiguration''': Event generation parameters. When null, no generation is done. For example, following parameters are supported:
** '''avoid_repeated_activities''': Array of activity names that should occur at most once in any case. The probability of selecting any of the activities specified in this configuration more than once is set to be 0.
*** Empty array means that activity generation is not restricted by this setting at all.
*** null value means that there should not be any activities that can occur more than once (shortcut for specifying all the activity names).
*** The default value is an empty array.
** '''cases_to_generate''': Maximum number cases to create. The number of created cases is further limited by the capabilities of the trained model and the ''case_generation_start_time'' and ''case_generation_end_time'' parameters.
*** The default value is such that the number of cases, by itself, is not limited.
** '''case_generation_start_time''': If defined, new cases will be generated after this timestamp (given as string in ISO datetime format).
*** If undefined, the latest start event timestamp used in the training data is used.
*** The default value is undefined.
** '''case_generation_end_time''': If defined, new events and cases will not be generated after this timestamp (given as string in ISO datetime format). E.g., "2015-01-01T00:00:00".
*** The default value is unlimited (only limit comes from the capacity of the trained model)
** '''generate_debug_event_attributes''':
*** If true, additional columns will be added containing, e.g., probabilities of the selected activity and other activities.
*** The default value is false.
** '''max_num_events''':
*** Specifies the maximum number of events to generate for any case.
*** If unspecified (=default), the value equals to ''<the maximum number of events in any case in the training data>''+''<the value of reserve_extra_sequence_length in training>''.
** '''min_prediction_probability ''':
*** The minimum probability of any prediction. If the probability of a prediction is lower than this, it will never be picked.
*** The default value is 0.01.
** '''temperature''':
*** If 0, the generated next activity will always be the one that is the most probable.
*** If 1, the generated next activity is purely based on the probabilities returned by the trained ML model.
*** This behavior is interpolated when using values between 0 and 1.
*** The default value is 0.9.
* '''TrainingDataFilter''': [[Filtering_in_QPR_ProcessAnalyzer_Queries|Filter]] to select specific cases that are used to train the prediction model. This filter is required to train the model only using the completed cases. Uncompleted cases should not be used for the training, so the model doesn't incorrectly learn that cases should end like that.
* '''IncompleteCasesFilter''': Optional [[Filtering_in_QPR_ProcessAnalyzer_Queries|filter]] to select which cases the prediction is made for. To improve performance of the prediction, it's recommended to include only the incomplete cases for which new events might appear, and skip the completed cases for which new events are not expected anymore.
* '''TrainingCaseSampleSize''': Maximum number of cases to take from the source model (cases are selected randomly). Use a lower setting to speed up the ML model training. The greater the value, the more subtle phenomena the prediction can learn from the data.

== Attribute configuration ==
Attribute configuration is used in '''TrainingConfiguration''' (see the chapter above) to configure which event- and case attributes should be used in prediction model and how they are used.

The configuration is in the top level split into two sections: "event" and "case". "Event" is used to configure event attributes, whereas "case" is used for case attributes.

The next level supports one value: "input".

The next level after that, supports the following settings:

* '''categorical_groups''': An array of categorical attribute group configuration objects used to define groups of attributes that will be bundled together in the trained model, either as separate input- or output features. Each attribute group will form its own input- or output vector used in the model training and generation.
** If null, only one group will be created with all the available categorical attributes included.
** The following settings are supported by these objects:
*** '''attributes''': An array of attribute names.
**** If null, all the input attributes are to be included in this group.
*** '''max_num_clusters''': The maximum number of clusters (input- or output vector feature values) to use to represent this group of attributes.
**** Default value: 20
**** NOTE: Clustering is used by default to convert a set of attribute values into an input- or output vector used by the prediction model.
*** '''ignore_values_threshold:''' The minimum percentage of objects having a specific attribute value in order for that attribute value to be taken into account as unique attribute value within this categorical group.
**** Depending on the context, the default value is any one of the following configurations:
***** ignore_values_threshold_for_case_attribute_values
***** ignore_values_threshold_for_case_attributes
***** ignore_values_threshold_for_event_attributes
* '''columns''': An array of attribute column configuration objects used to define columns in the input data that are to be used as event- or case attributes.
** If null, all the columns will be included as categorical attributes (except case id, event type (only for event) and timestamp (only for event) columns).
** The following settings are supported by these objects:
*** '''label''': Column name.
*** '''type''': Type of the column. Supported types are:
**** '''categorical''': Values can take on one of a limited, and usually fixed, number of possible values.
**** '''numeric''': Value is considered as a continuous numeric value.

==== Example ====
Use all event attributes as input for the prediction model. In addition, additional machine learning input vector for SAP_User-event data column supporting at most 10 unique values.

In addition, for case attributes, only "Region", "Product Group", "Account Manager" and "Customer Group" case data columns are used as categorical attributes and "Cost" as numeric attribute. Furthermore, the four categorical case attributes are grouped into three groups, each of which are used as its own input vector for the prediction model.

When generating, all event attributes will be included for generated events as columns. Generated cases will have only "Region", "Product Group", "Account Manager", "Customer Group", and "Cost" columns.<syntaxhighlight lang="json">
"attributes": #{
"event": #{
"input": #{
"categorical_groups": [
#{
"attributes": None
},
#{
"attributes": ["SAP_User"],
"max_num_clusters": 10
}
],
"columns": None
}
},
"case": #{
"input": #{
"categorical_groups": [#{
"attributes": ["Account Manager"]
}, #{
"attributes": ["Customer Group"]
}, #{
"attributes": ["Region", "Product Group"]
}],
"columns": [
#{ "label": "Region", "type": "categorical" },
#{ "label": "Product Group", "type": "categorical" },
#{ "label": "Account Manager", "type": "categorical" },
#{ "label": "Customer Group", "type": "categorical" },
#{ "label": "Cost", "type": "numeric" }
]
}
}
}
</syntaxhighlight>

== Predicting case attribute values ==
QPR ProcessAnalyzer can also be used to, e.g., predict the final values of case attributes of running cases. The following script gives an example on how to perform this.<syntaxhighlight lang="typescript">
let sourceModel = ProjectByName("<project name>").ModelByName("<model name>");
let caseAttributeToPredict = "<name of the case attribute>";
let resultModelName = "<name of the model to be created/replaced>";
let generateDebugCaseAttributes = false; // Set to true to generate columns for prediction probabilities.
let casesToPredictFilter = "<JSON filter for cases for which the prediction is to be performed>";
let casesToUseForTrainingFilter = "<JSON filter for cases to be used for ML model training>";

let targetProject = Project;

_system.ML.GenerateCaseAttributePredictionModel(#{
"Name": resultModelName, // Name of the PA model to generate ti the target project.
"SourceModel": sourceModel, // Snowflake-based PA model used for training the prediction model.
"TargetProject": targetProject, // Target project to create the model into.
"RecreatePredictionModel": false, // Should a prediction model be overwritten if one already exists for this source model and target model name combination.
"TrainingCaseSampleSize": 10000, // Maximum number of cases to use from the source model (random sampled).
"CommonConfiguration": #{ // Common parameters used by both training and generation.
"output_case_attribute_groups": [#{
"attributes": [caseAttributeToPredict] // Attribute whose value is to be predicted.
}]
},
"TrainingConfiguration": #{ // Training parameters.
"max_num_case_attribute_clusters": 80,
"num_epochs_to_train": 100
},
"GenerationConfiguration": #{ // Case attribute generation parameters.
"generate_debug_case_attributes": generateDebugCaseAttributes // Should probability and probability_all-columns be generated as well as the actual prediction created into a new column named Predicted_<attribute name>
},
"TrainingDataFilter": ParseJson(casesToUseForTrainingFilter), // Filter JSON for events to be used for training.
"IncompleteCasesFilter": ParseJson(casesToPredictFilter) // Filter JSON for events for whose case attribute value is to be predicted.
});

</syntaxhighlight>

Expression Script Examples

2026-01-12T08:22:24Z

MarHink: /* Create a new object-centric model based on already existing object-centric model containing only filtered objects and events */

This page contains script examples written in the QPR ProcessAnalyzer expression language. See how expression scripts can be created in the [[Managing_Scripts#Creating_Script|Workspace]]. For documentation for the syntax, functions and entities can be found from the main page in the [[QPR_ProcessAnalyzer_Wiki#For_Developers|KPI Expression Language]] section.

== Calling Expression Script from Expression ==
Expression scripts can be called from an expression using the [[QPR_ProcessAnalyzer_Objects_in_Expression_Language#Script|Run]] function with the following syntax:
<syntaxhighlight lang="typescript" line>
let result = ScriptById(123).Run(#{
"parameter1": "value1",
"parameter2": false,
"parameter2": 123.45
})
</syntaxhighlight>

The command waits until the run is completed, and the return value of the called script is returned by the Run function call.

Parameters can be passed to the called script, and the parameters are available as variables in the script. The parameters can contain any type of data.

Expression scripts can also be called from a dashboard. Expressions can be stored to scripts instead of dashboards, which is a way to separate complex expressions from dashboards and allow to reuse expressions across several dashboards.

== Calling SQL Script from Expression ==
SQL script can be called from an expression using the Run function as follows (similar to calling [[#Calling Expression Script from Expression|expression scripts]]):
<syntaxhighlight lang="typescript" line>
let result = ScriptById(123).Run(#{
"parameter1": "value1",
"parameter2": 321
});
let arrayOfAllReports = result.Keys;
let report1 = result.Report1;
let report2 = result.Report2;
</syntaxhighlight>

SQL scripts can return multiple ''reports'', which are combined to a dictionary, where the key is the name of the report ("sheet name") and value is the report data as a DataFrame. See in the above example, how the reports can be accessed by their name.

== Examples ==
=== Call web service===
Contact to a web service, fetch some data, and store it to a datatable.

<syntaxhighlight lang="typescript" line>
let datatableName = "Web Service Data";
let webServiceData = CallWebService(
#{"Address": "https://processanalyzer.onqpr.com/qprpa/api/serverinfo"}
);

let targetDatatable = Project.Datatables.Where(name==datatableName);
if (Count(targetDatatable) == 0) {
targetDatatable = Project.CreateDatatable(datatableName)
.AddColumn("Setting name", "String")
.AddColumn("Setting value", "String")
.AddColumn("Data read", "DateTime");
} else {
targetDatatable = targetDatatable[0];
}

let currentTime = Now;
let dataAsDf = ToDataFrame(
webServiceData.keys.{
let key = _;
[key, webServiceData[key], currentTime];
},
["Setting name", "Setting value", "Data read"]
);
targetDatatable.Import(dataAsDf, #{"Append":true});
WriteLog(`${CountTop(dataAsDf.Rows)} rows written to datatable`);
</syntaxhighlight>

=== Store data to datatable ===
Get all models in the system and store them to a datatable.

<syntaxhighlight lang="typescript" line>
let newDatatable = Project
.CreateDatatable("Models list " + ToString(Now, "dd.MM.yyyy HH:mm:ss"))
.AddColumn("Model name", "String")
.AddColumn("Project name", "String")
.AddColumn("Created time", "DateTime")
.AddColumn("Cases", "Integer");
let startTime = Now;
let modelsData = ToDataFrame(
Models.([Name, Project.Name, CreatedDate, NCases]),
["Model name", "Project name", "Created time", "Cases"]
);
WriteLog(`Listing models took ${(Now - startTime).TotalSeconds.Round(2)} seconds.`);
newDatatable.Import(modelsData);
WriteLog(`Datatable ${newDatatable.Id} created.`);
</syntaxhighlight>

=== Convert datatable column data (Snowflake) ===
Following script converts textual data from the column "TimestampString" to dates to column "Timestamp" by trying different time formats and using the first suitable.

<syntaxhighlight lang="typescript" line>
DatatableById(1)
.UpdateRows(
true,
"Timestamp",
Coalesce(
TryToTimestamp(Column("TimestampString"), "DD.MM.YYYY HH24:MI:SS.FF3"),
TryToTimestamp(Column("TimestampString"), "YYYY-MM-DD HH24:MI:SS.FF3"),
TryToTimestamp(Column("TimestampString"), "YYYY/MM/DD HH24:MI:SS.FF3")
)
);
</syntaxhighlight>

=== Convert datatable column data (in-memory) ===
This script can be used to convert a single column into numerical data type. To use the script, you need to setup the following in the beginning of the script:
* Project name where the datatable is located.
* Datatable name
* Name of the column to be converted

Note that the conversion fails, if there is data that cannot be converted into numerical format. The conversion assumes that period (.) is used as the decimal point.

<syntaxhighlight lang="typescript" line>
let projectName = "New Project";
let datatableName = "qpr processanalyzer events";
let columnName = "Event order in case";

let project = (Projects.Where(Name==projectName))[0];
let datatable = (project.Datatables.Where(Name==datatableName))[0];
DatatableById(datatable.Id).DataFrame
.SetColumns([
columnName: () => {
let data = Column(columnName);
if (data == null) {
null;
} else {
ToFloat(data);
}
}
])
.Persist(datatable.Name, ["ProjectId": project.Id, "Append": false]);
</syntaxhighlight>

Instead of converting to numeric (with the ''ToFloat'' function), data can be converted into string using the ''ToString'' function.

=== Show DataFrame as HTML table ===

This script defines a function to show dataframe as a HTML table, and uses the function for a literal dataframe.

<syntaxhighlight lang="typescript" line>
function dataframeToHtmlTable(df) {
return
`<table>
<tr>
${StringJoin("\r\n\t\t", + df.columns.`<th>${_}</th>`)}
</tr>
${StringJoin("", df.Rows.(
"\r\n\t<tr>" + StringJoin("", _.`\r\n\t\t<td>${ToString(_)}</td>`) + "\r\n\t</tr>"
))}
</table>`
}

let data = ToDataFrame(
[
["one", "two", "three"],
["four", "five", "six"],
["seven", "eight", "nine"]
],
["Column 1", "Column 2", "Column 3"]
);

return dataframeToHtmlTable(data);
</syntaxhighlight>

=== Copy local datatables to Snowflake ===

<syntaxhighlight lang="typescript" line>
// Copies all datatables in a project to another project including datatable contents.
// Usage instructions:
// 1. Create expression script in the project from where you want to copy the datatables.
// 2. Create a new project named as "<name of the project to be moved> - Snowflake". New datatables will be created here. E.g., when moving project named "SAP_OrderToCash", the target project should be named as "SAP_OrderToCash - Snowflake".
// 3. Run the script.
// NOTE: Columns of type "Any" will be created as "String"-columns in Snowflake, thus it is recommended that actual data types are set for the tables prior to the move.

let sourceProject = Project;
let sourceProjectName = Project.Name;
let targetProjectName = `${sourceProjectName} - Snowflake`;
let targetProject = First(Projects.Where(Name == targetProjectName));
if (IsNull(targetProject)) {
WriteLog(`Unable to find target project named "${targetProjectName}". Aborting operation.`);
return;
}
let dts = sourceProject.DataTables;
WriteLog(`Copying all ${CountTop(dts)} data tables found in project "${sourceProject.Name}" (id: ${sourceProject.Id}) to Snowflake in project "${targetProject.Name}" (id: ${targetProject.Id})`);
dts.{
let sourceDt = _;
WriteLog(`Starting to copy data table "${Name}" (id: ${Id}) having ${NRows} rows and ${NColumns} columns.`);
let targetDt;
targetDt = targetProject.DatatableByName(sourceDt.Name);
if (targetDt == null) {
targetDt = targetProject.CreateDataTable(sourceDt.Name, #{"Connection": CreateSnowflakeConnection(#{"ProjectId": targetProject.Id})});
targetDt.Import(sourceDt.SqlDataFrame);
WriteLog(`Finished copying data table "${Name}" (id: ${Id}) to table "${targetDt.Name}" (id: ${targetDt.Id})`);
} else {
WriteLog(`Datatable already exist "${Name}" (id: ${Id}) to table "${targetDt.Name}" (id: ${targetDt.Id})`);
}
}
WriteLog(`Finished copying all the data tables found in project "${sourceProject.Name}" (id: ${sourceProject.Id}) to Snowflake in project "${targetProject.Name}" (id: ${targetProject.Id})`);
</syntaxhighlight>

If you don't need to copy the data but only create the Snowflake datatables with columns, you can change the line 22 to
<syntaxhighlight>
targetDt.Import(sourceDt.SqlDataFrame.head(0));
</syntaxhighlight>

=== Copy single datatable to Snowflake ===
This script creates a copy of a single datatable to Snowflake. Replace the ''<tableId1>'' with the id of the source datatable.
<syntaxhighlight lang="typescript" line>
function CopyDataTableToSnowflake(dataTableId)
{
let sourceDt = DataTableById(dataTableId);
sourceDt.SqlDataFrame.Persist(`${sourceDt.Name} - Snowflake`, #{"Append": false, "Connection": CreateSnowflakeConnection(#{"ProjectId": sourceDt.Project.Id})});
}
CopyDataTableToSnowflake(<tableId1>);
</syntaxhighlight>

=== Create a copy of a data table that has all Any-type columns changed to String-type columns ===
<syntaxhighlight lang="typescript" line>
function ConvertAnyDataTypesToStringsToNewTable(dataTableId)
{
let dt = DataTableById(dataTableId);
let sdf = dt.SqlDataFrame;
let cts = dt.ColumnTypes;
cts.{
let ct = _;
if (ct.DataType == "Any") {
let n = ct.Name;
sdf = sdf.WithColumn(ct.Name, #sql{Cast(Column(Variable("n")), "ShortString")});
}
};
sdf.Persist(`${dt.Name} - Converted`, #{"Append": false, "ProjectId": dt.Project.Id});
}
ConvertAnyDataTypesToStringsToNewTable(<dataTableId>);
</syntaxhighlight>

=== Query number of rows in given data table having a datetime value in given year grouped by month and return resulting table as CSV ===
SqlDataFrame is used in order to prevent loading the whole datatable into memory first. Filtering is performed as first operation in order to minimize the amount of required work for the data source of the data table.<syntaxhighlight lang="typescript" line="1">
DataTableById(<data table id>)
.SqlDataFrame
.Where(#sql{2014 == Year(Column("Start Time"))})
.WithColumn("Month", #sql{Month(Column("Start Time"))})
.GroupBy(["Month"]).Aggregate(["Count"], ["Count"])
.OrderByColumns(["Month"], [true])
.Collect().ToCsv();
</syntaxhighlight>

=== Function for filtering SqlDataFrame by removing rows having, or replacing, the most infrequently occurring column values ===
<syntaxhighlight lang="typescript" line="1">
/***
* @name ColumnWithMinUsage
* @descripion
* Generic function that can be used to filter out the most infrequently occurring attribute values or replace their Values
* with given common value.
* @param df:
* DataFrame to operate on.
* @param columnName:
* Name of the column to be filtered.
* @param newColumnName:
* Name of the column that will contain the new value of the original column after filtering (if includeOthers was applied).
* @param maxNumUniqueValues:
* Maximum number of unique values to include into the comparison for each attribute column. If the amount of unique values for any attribute exceeds this value, only given number of attributes are included that have the highest usage.
* @param minValueUsage:
* Minimum total usage of a value included into the comparison. The number of cases having every returned value should be at least given percentage (a float value between 0.0 and 1.0) of all the compared cases.
* @param includeOthers:
* Should the rest of the attribute values not included due to MinValueUsage or MaxNumUniqueValues filtering be included as an aggregated "Others" value?
* If not empty/null, defines the name used for these other-values.
*/
function ColumnWithMinUsage(df, columnName, newColumnName, maxNumUniqueValues, minValueUsage, includeOthers)
{
let all = df
.GroupBy([])
.Aggregate(["NAllTotal"], ["Count"])
.WithColumn("__Join2", #sql{1});
let minValueUsageEnabled = !IsNullTop(minValueUsage);
let maxNumUniqueValuesEnabled = !IsNullTop(maxNumUniqueValues);
if (minValueUsageEnabled || maxNumUniqueValuesEnabled) {
// Perform column value-based filtering if minValueUsageEnabled or maxNumUniqueValuesEnabled is defined.
let valueColumnName = "__ValueNew";
let filteredValuesColumns = [valueColumnName: columnName];
let filteredValues = df
.GroupBy([columnName]).Aggregate(["Count"], ["Count"]);
if (minValueUsageEnabled) {
filteredValues = filteredValues
.WithColumn("__Join", #sql{1})
.Join(all, ["__Join": "__Join2"], "leftouter")
.WithColumn("Usage", #sql{Column("Count") / Column("NAllTotal")});
filteredValuesColumns = Concat(filteredValuesColumns, ["Usage"]);
}
if (maxNumUniqueValuesEnabled) {
filteredValues = filteredValues
.WithRowNumberColumn("RowNumber", ["Count"], null, [false]);
filteredValuesColumns = Concat(filteredValuesColumns, ["RowNumber"]);
}

filteredValues = filteredValues
.Select(filteredValuesColumns);

// Generate select returning all the accepted values.
let allValues = filteredValues
.(minValueUsageEnabled ? Where(#sql{Column("Usage") >= #expr{minValueUsage}}) : _)
.(maxNumUniqueValuesEnabled ? Where(#sql{Column("RowNumber") <= #expr{maxNumUniqueValues}}) : _)
.Select([valueColumnName, newColumnName: valueColumnName]);

if (!IsNullTop(includeOthers)) {
// If includeOthers is defined, replace original values with the variable defined in includeOthers.
let otherValues = filteredValues
.(minValueUsageEnabled ? Where(#sql{Column("Usage") < #expr{minValueUsage}}) : _)
.(maxNumUniqueValuesEnabled ? Where(#sql{Column("RowNumber") > #expr{maxNumUniqueValues}}) : _)
.WithColumn(newColumnName, #sql{#expr{includeOthers}})
.Select([valueColumnName, newColumnName]);
allValues = allValues.Append(otherValues)
}
df.Join(allValues, [columnName: valueColumnName], "inner")
.RemoveColumns([valueColumnName]);
}
}

// The following example will return only rows containing two of the most common values for Region-column.
//let df = DataTableById(<data table id>).SqlDataFrame;
//df = ColumnWithMinUsage(df, "Region", "_Filtered", 2, null, null);
//df.Collect().ToCsv();

// The following example will return all input rows, but will replace the values of rows whose Region-column
// has a value used by less than 15% of all the rows with a new value: "_Others".
//let df = DataTableById(<data table id>).SqlDataFrame;
//df = ColumnWithMinUsage(df, "Region", "_Filtered", null, 0.15, "_Others");
//df.Collect().ToCsv();
</syntaxhighlight>

=== Export model events and cases ===
<syntaxhighlight lang="typescript" line="1">
function ExportModelEvents(m) {
let attrs = m.EventAttributes;
ToDataFrame(
m.EventLog.Events.Concat(
[Case.Name, Type.Name, ToString(TimeStamp, "yyyy-MM-dd HH:mm:ss.fff")],
{let evt = _; attrs.{let att = _; evt.Attribute(att)}}
),
Concat(
["CaseId", "EventType", "TimeStamp"],
attrs.Name
)
).ToCsv(true);
}

function ExportModelCases(m) {
let attrs = m.CaseAttributes;
ToDataFrame(
m.EventLog.Cases.Concat(
[Name],
{let cas = _; attrs.{let att = _; cas.Attribute(att)}}
),
Concat(
["CaseId"],
attrs.Name
)
).ToCsv(true);
}
</pre>

<pre>
First(Models.Where(Name=="SAP OtC Extended")).EventsDataTable.DataFrame.ToCsv(true)
</pre>

<pre>
First(Models.Where(Name=="SAP OtC Extended")).CasesDataTable.DataFrame.ToCsv(true)
</syntaxhighlight>

=== Calculate all the value usages of a single column for each event in event data table ===
This query could be used, e.g., to find out the maximum resource usage for every resource found in the event data table.<syntaxhighlight lang="typescript" line="1">
function WithUsageColumns(resourceColumn)
{
function WithTotalUsageColumnOfSingleResource(resourceColumn, resourceValue)
{
_
.WithColumn("_Prev", #sql{Lag(Column(resourceColumn), [TimeStamp, EventType], [true, true], [CaseId], 1, null)})
.WithColumn("_UsageDiff", #sql{
CaseWhen(
Column(resourceColumn) == Column("_Prev"), 0,
Column("_Prev") == #expr{resourceValue}, -1,
Column(resourceColumn) == #expr{resourceValue}, 1,
0)
})
.WithColumn(`${resourceValue}_Usage`, #sql{Sum(Column("_UsageDiff"), [TimeStamp, EventType])})
.RemoveColumns(["_Prev", "_UsageDiff"])
}

let sdf = _;
let allValues = sdf.SelectDistinct([resourceColumn]).OrderByColumns([resourceColumn], [true]).Collect().Column(resourceColumn);
allValues.{
let v = _;
sdf = sdf.WithTotalUsageColumnOfSingleResource(resourceColumn, v)
}
sdf
}

let dt = ModelById(<model id>).EventsDataTable;
dt
.SqlDataFrame
.WithUsageColumns(<resource column name>)
.OrderByColumns([dt.ColumnMappings["TimeStamp"]], [true])
.Collect().ToCsv()
</syntaxhighlight>Where:

* <model id> is the id of the model containing event data to be examined.
* <resource column name> is the name of the column in the event data table of the specified model containing the resource being used by that event.

NOTE: This expression uses functionalities that are only supported in Snowflake-based data tables.

=== Create new Snowflake model from filter ===
This script creates a new Snowflake model (and two datatables for cases and events) containing filtered event log from given filter id. The script also works if the model doesn't have a cases datatable.

<syntaxhighlight lang="typescript" line="1">
let filter = FilterById(1); // filter id
let model = filter.model;
let project = model.project;
let nameSuffix = " - " + filter.name + " - " + ToString(Now, "dd-MM-yyyy HH:mm:ss");
let eventsDatatableName = model.EventsDataTable.Name + nameSuffix;
if (eventsDatatableName.length > 440) {
eventsDatatableName = eventsDatatableName.Substring(eventsDatatableName.length - 440);
}
let eventsData = model
.EventsDataTable
.SqlDataFrame
.ApplyFilter(
filter.rules,
model.CasesDataTable?.SqlDataFrame
);
project
.CreateDatatable(eventsDatatableName, #{"Connection": CreateSnowflakeConnection()})
.Import(eventsData);
let modelConfiguration = model.Configuration;
modelConfiguration.DataSource.Events.Set("DataTableName", eventsDatatableName);
if (model.CasesDataTable != null) {
let eventsDataCaseIdColumn = "CaseId_" + ToString(Random());
let casesDatatableName = model.CasesDataTable.Name + nameSuffix;
if (casesDatatableName.length > 440) {
casesDatatableName = casesDatatableName.Substring(casesDatatableName.length - 440);
}
let casesData = model
.CasesDataTable
.SqlDataFrame
.join(
eventsData.SelectDistinct([eventsDataCaseIdColumn: modelConfiguration.DataSource.Events.Columns.CaseId]),
[modelConfiguration.DataSource.Cases.Columns.CaseId: eventsDataCaseIdColumn]
).Select(model.CasesDataTable.ColumnNames);
project
.CreateDatatable(casesDatatableName, #{"Connection": CreateSnowflakeConnection()})
.Import(casesData);
modelConfiguration.DataSource.Cases.Set("DataTableName", casesDatatableName);
}
let modelName = model.Name + nameSuffix;
if (modelName > 440) {
modelName = modelName.Substring(modelName.length - 440);
}
project
.CreateModel(#{
"Name": modelName,
"Description": model.Description,
"Configuration": modelConfiguration
});
return modelName;
</syntaxhighlight>

=== Creating a model consisting of multiple copies of cases in an existing model ===
<syntaxhighlight lang="typescript" line="1">
/**
* @name CreateTestModel
* @description
* Creates a new model (or overwrites an existing) to given target project with given number of
* repetitions of given source model.
* Each repetition will generate "<N>-"-prefix to CaseId-columns, where N equals to the repeat index.
* @param sourceModel
* PA model used for the source data and from where the connection is copied for the target model if a
* new one has to be created.
* @param numRepeats
* Number of times the data in the source model should be repeated in the generated model.
* @param targetProject
* Project in which the target model resides.
* @param targetModelName
* Specifies the name of the test model in the given target project. If a model already exists with
* given name, event and case data in this model will be replaced with the new generated event and
* case data.
* @returns
* Model object of the test model having the newly generated data.
*/
function CreateTestModel(sourceModel, numRepeats, targetProject, targetModelName)
{
let eventsColumnMappings = sourceModel.EventsDataTable.ColumnMappings;
let casesColumnMappings = sourceModel.CasesDataTable.ColumnMappings;
let connection = sourceModel.EventsDataTable.DataSourceConnection;

function CreateResultModel()
{
function GetTable(tableName)
{
let tableConfiguration = #{
"Name": tableName,
"Connection": connection
};
let resultTable = targetProject.DataTableByName(tableName);
if (resultTable == null)
{
resultTable = targetProject.CreateDataTable(tableConfiguration)
.Modify(#{"NameInDataSource": null})
.Synchronize();
}
return resultTable;
}

let eventsTableName = `${targetModelName} - events`;
let casesTableName = `${targetModelName} - cases`;
let targetModel = targetProject.ModelByName(targetModelName);
let eventsTable, casesTable = null;

if (targetModel != null)
{
eventsTable = targetModel.EventsDataTable;
casesTable = targetModel.CasesDataTable;
}
else {
eventsTable = GetTable(eventsTableName);
if (sourceModel.CasesDataTable != null) {
casesTable = GetTable(casesTableName);
}

let timestampMapping = eventsColumnMappings["TimeStamp"];
eventsColumnMappings.Remove("TimeStamp");
eventsColumnMappings.Set("Timestamp", timestampMapping);

let modelConfiguration = #{
"DataSource": #{
"Events":#{
"DataSourceType": "datatable",
"DataTableName": eventsTableName,
"Columns": eventsColumnMappings
}
}
};

if (casesColumnMappings != null) {
modelConfiguration["DataSource"].Set("Cases", #{
"DataSourceType": "datatable",
"DataTableName": casesTableName,
"Columns": casesColumnMappings
});
}

targetModel = targetProject.CreateModel(#{"Name": targetModelName, "Configuration": modelConfiguration});
}

eventsTable.Truncate();
casesTable?.Truncate();

return #{
"TargetModel": targetModel,
"Events": eventsTable,
"Cases": casesTable
};
}

function RepeatNTimes(sourceDf, caseIdColumn, numRepeats)
{
let resultDf = null;
for (let i = 1; i <= numRepeats; ++i) {
let iterationDf = sourceDf
.WithColumn(caseIdColumn, #sql{Concat(#expr{i}, "-", Column(#expr{caseIdColumn}))});
resultDf = resultDf == null ? iterationDf : resultDf.Append(iterationDf);
}
resultDf;
}

let resultModel = CreateResultModel();
let sourceEventDataDf = sourceModel.EventsDataTable.SqlDataFrame;
let resultEventDataDf = RepeatNTimes(sourceEventDataDf, eventsColumnMappings["CaseId"], numRepeats);
resultModel["Events"].Import(resultEventDataDf);

let sourceCaseDataDf = sourceModel.CasesDataTable?.SqlDataFrame;
if (sourceCaseDataDf != null) {
let resultCaseDataDf = RepeatNTimes(sourceCaseDataDf, casesColumnMappings["CaseId"], numRepeats);
resultModel["Cases"].Import(resultCaseDataDf);
}
resultModel["TargetModel"];
}

</syntaxhighlight>Example usage:<blockquote>CreateTestModel(ProjectByName("Project").ModelByName("SAP_OrderToCash - Snowflake"), 3, ProjectByName("TestData"), "TestModel");</blockquote>Creates a new model named "TestModel" (or overwrites old one) into project named "TestData" containing the data from model "SAP_OrderToCash - Snowflake" in project "Project" repeated three times.

=== Analyzing declare patterns found in event log ===

This is an example expression that shows how POSIX-style regular expressions can be used to search for cases in an event log having certain event type patterns [https://www.researchgate.net/publication/277631859_Generating_Event_Logs_Through_the_Simulation_of_Declare_Models declare patterns].

Note: Before use, replace <model id> with a valid model identifier having event data.

<syntaxhighlight lang="typescript" line="1">
let dt = ModelById(<model id>).EventsDataTable;
let sdf = dt.SqlDataFrame.Head(1000);
let caseIdColumn = dt.ColumnMappings["CaseId"], timeStampColumn = dt.ColumnMappings["TimeStamp"], eventTypeColumn = dt.ColumnMappings["EventType"];

let eventTypesDf = sdf
.SelectDistinct([eventTypeColumn])
.OrderByColumns([eventTypeColumn], [true])
.WithRowNumberColumn("Token", [eventTypeColumn])
.WithColumn("Token", #sql{Char(Column("Token") + Unicode("a") - 1)});

sdf = sdf
.Join(eventTypesDf.Select(["_EventType2": eventTypeColumn, "Token"]), [eventTypeColumn: "_EventType2"], "leftouter");

function RespondedExistencePattern(a, b)
{
// [^a]*((a.*b.*)|(b.*a.*))*[^a]*
ToSqlExpression(`Regexp(Column("Trace"), "[^${a}]*((${a}.*${b}.*)|(${b}.*${a}.*))*[^${a}]")`)
}

function ResponsePattern(a, b)
{
// [^a]*(a.*b)*[^a]*
ToSqlExpression(`Regexp(Column("Trace"), "[^${a}]*(${a}.*${b})*[^${a}]")`)
}

function AlternateResponsePattern(a, b)
{
// [^a]*(a[^a]*b[^a]*)*[^a]*
ToSqlExpression(`Regexp(Column("Trace"), "[^${a}]*(${a}[^${a}]*${b}[^${a}]*)*[^${a}]")`)
}

function ChainResponsePattern(a, b)
{
// [^a]*(ab[^a]*)*[^a]*
ToSqlExpression(`Regexp(Column("Trace"), "[^${a}]*(${a}${b}[^${a}]*)*[^${a}]")`)
}

function PrecedencePattern(a, b)
{
// [^b]*(a.*b)*[^b]*
ToSqlExpression(`Regexp(Column("Trace"), "[^${b}]*(${a}.*${b})*[^${b}]")`)
}

let tracesDf = sdf
.GroupBy([caseIdColumn])
.Aggregate(["Trace": "Token"], [#{"Function": "list", "Ordering": [timeStampColumn], "Separator": ""}])
.WithColumn("RespondedExistenceNL", #sql{#expr{RespondedExistencePattern("n", "l")}})
.WithColumn("RespondedExistenceCA", #sql{#expr{RespondedExistencePattern("c", "a")}})
.WithColumn("ResponsePatternNL", #sql{#expr{ResponsePattern("n", "l")}})
.WithColumn("ResponsePatternCA", #sql{#expr{ResponsePattern("c", "a")}})
.WithColumn("AlternateResponsePatternNL", #sql{#expr{AlternateResponsePattern("n", "l")}})
.WithColumn("AlternateResponsePatternCA", #sql{#expr{AlternateResponsePattern("c", "a")}})
.WithColumn("ChainResponsePatternNL", #sql{#expr{ChainResponsePattern("n", "l")}})
.WithColumn("ChainResponsePatternCA", #sql{#expr{ChainResponsePattern("c", "a")}})
.WithColumn("PrecedencePatternNL", #sql{#expr{PrecedencePattern("n", "l")}})
.WithColumn("PrecedencePatternCA", #sql{#expr{PrecedencePattern("c", "a")}})
;

[tracesDf.Collect().ToCsv(), eventTypesDf.Collect().ToCsv()]
</syntaxhighlight>

=== Perform a query and send results as E-mail in a HTML table ===
This example requires two scripts that are both located in the same project:

# Expression-type script to execute (can be, e.g., scheduled to run daily).
# Expression-type script containing the query JSON to use as basis for the e-mail. In this example, this script is named as "Send query as E-mail - query JSON". The contents of this script is just the JSON representation of a query that can be extracted, e.g., from any PA chart view.

Script #1 should contain the following code:<syntaxhighlight lang="typescript">
let replyToAddress = "noreply@test.com";
let recipientsArray = ["testuser@test.com"];
let queryConfiguration = ParseJson(Project.ScriptByName("Send query as E-mail - query JSON").Code)
.Set("IncludeCollect", true);
let resultDf = Query(queryConfiguration);
let mailBodyHtml = resultDf.`
<table>
<caption>Example query</caption>
<thead>
<tr>
${StringJoin("", _.Columns.`
<th>${_}</th>
`)}
</tr>
</thead>
<tbody>
${StringJoin("", _.Rows.`
<tr>
${StringJoin("", _.`
<td>${_}</td>
`)}
</tr>
`)}
</tbody>
</table>
`;

SendEmail(#{
"ReplyTo": [replyToAddress],
"To": recipientsArray,
"Subject": "Example query E-mail",
"IsBodyHtml": true,
"Body": mailBodyHtml
});

</syntaxhighlight>NOTE: QPR ProcessAnalyzer Server has to have SMTP server configured. Also, remember to update the values of replyToAddress and recipientsArray before using.

=== Converting a case-centric model to object-centric model ===
This function serves as an example on how a case-centric model could be converted into an object-centric model having just one object type: "Case".

Note: Before use, replace <model id> with a valid model identifier having event data.<syntaxhighlight lang="typescript">
function ConvertCCModelToOCModel(model, newModelName)
{
let connection = model.EventsDataTable.DataSourceConnection;
let caseIdColumn = model.EventsDataTable.ColumnMappings["CaseId"];
let eventToObjectTableName = `${newModelName} - event-to-object`;
let eventsTableName = `${newModelName} - events`;
let objectsTableName = `${newModelName} - objects`;

let eventsDf = model.EventsDataTable
.SqlDataFrame
.RenameColumns([
"OcelEventType": model.EventsDataTable.ColumnMappings["EventType"],
"OcelEventTime": model.EventsDataTable.ColumnMappings["TimeStamp"]])
.WithColumn("OcelEventId", #sql{Concat("evt-", Cast(RowNumber([Column("OcelEventTime")]), "String"))});

eventsDf
.RenameColumns([
"OcelEventToObjectSourceId": "OcelEventId",
"OcelEventToObjectTargetId": caseIdColumn])
.Select(["OcelEventToObjectSourceId", "OcelEventToObjectTargetId"])
.WithColumn("OcelEventToObjectQualifier", #sql{#expr{caseIdColumn} })
.Persist(eventToObjectTableName, #{"Connection": connection});

eventsDf
.RemoveColumns([caseIdColumn])
.Persist(eventsTableName, #{"Connection": connection});

let casesDt = model.CasesDataTable
.SqlDataFrame
.RenameColumns([
"OcelObjectId": model.CasesDataTable.ColumnMappings["CaseId"]
])
.WithColumn("OcelObjectType", #sql{"Case"})
.Persist(objectsTableName, #{"Connection": connection});

let newConfiguration = #{
"OcelDataSource": #{
"Events": eventsTableName,
"Objects": objectsTableName,
"EventToObject": eventToObjectTableName
}
};

model.Project
.CreateModel(#{
"Name": newModelName,
"Description": model.Description,
"Configuration": newConfiguration
});
}

let ccModel = ModelById(<model id>);
ConvertCCModelToOCModel(ccModel, `ocel - ${ccModel.Name}`);
</syntaxhighlight>

=== Show duplicate rows in datatable ===
This script returns all rows in a datatable appearing more than once (sorted by the most frequent occurrences first). Additionally, the number of occurrences is returned as the last column. Replace the datatable id with the correct one.

<syntaxhighlight lang="typescript">
let datatableId = 1;
let rowCountColumn = "Row count";
let columns = DatatableById(datatableId).Columns.Name;
DatatableById(datatableId)
.SqlDataFrame
.GroupBy(columns)
.Aggregate(
[rowCountColumn: columns[0]],
["Count"]
)
.Where(Column(rowCountColumn) > 1)
.OrderByColumns([rowCountColumn], [false])
.Collect()
.ToCSV();
</syntaxhighlight>

=== Create a new object-centric model based on already existing object-centric model containing only filtered objects and events ===
Script for creating a new object-centric model based on another already existing object-centric model that applies given filter to the original model and creates all the needed tables to store only the relevant rows for all the object-centric data tables.

NOTE: Replace <nowiki><source model id>, <target model name>, <target project id> as well as the value of ocelItems with the applicable values before use.</nowiki><syntaxhighlight lang="typescript">
let newModelName = "<target model name>";
let sourceModelId = <source model id>;
let targetProjectId = <target project id>;

// Replace the value of the ocelItems below with the configuration of object-centric filter to apply.
// Note: Use expression language syntax here, not JSON.
let ocelItems = [#{
"Include": true,
"ObjectAttributeValue": #{
"ObjectType": "Purchase Order",
"Attribute": "po_product",
"Values": [
"0Cows"
]
}
}];

let m = ModelById(sourceModelId);
if (!m.IsOcelModel)
throw `Model ${m.Name} is not an OCPM model`;
if (CountTop(m.CheckModelValidity()) > 0)
throw `Model ${m.Name} is not a valid OCPM model`;

let targetProject = ProjectById(targetProjectId);
if (CountTop(targetProject.Models.Where(Name == newModelName)) > 0)
throw `Model having name ${newModelName} already exists in project ${targetProject.Name}. Rename or delete the existing model before running this script.`;

let newConfiguration = #{
"OcelDataSource": #{
"EventTypes": #{},
"ObjectTypes": #{}
}
};

WriteLog("Creating common tables...");

function PersistDataFrame(dataFrame, configurationName, configurationDict, dataTableName)
{
WriteLog(`Creating filtered table: ${dataTableName}`);
configurationDict.Set(configurationName, dataTableName);
return dataFrame.Persist(dataTableName, #{"ProjectId": targetProjectId, "Append": false});
}

let objectsDf = m.CacheTableSqlDataFrame(#{
"CacheTableType": "Objects",
"Perspective": #{},
"OcelItems": ocelItems
}).SetPrimaryKey(m.OcelObjects.PrimaryKey);
let newObjectsDt = PersistDataFrame(objectsDf, "Objects", newConfiguration.OcelDataSource, `${newModelName} - objects`);

let objectToObjectDf = m.CacheTableSqlDataFrame(#{
"CacheTableType": "ObjectToObject",
"Perspective": #{},
"OcelItems": ocelItems
}).SetPrimaryKey(m.OcelObjectToObject.PrimaryKey)
.Select(["OcelObjectToObjectSourceId", "OcelObjectToObjectTargetId", "OcelObjectToObjectQualifier"]);
let newObjectToObjectDt = PersistDataFrame(objectToObjectDf, "ObjectToObject", newConfiguration.OcelDataSource, `${newModelName} - object-to-object`);

let eventToObjectDf = m.CacheTableSqlDataFrame(#{
"CacheTableType": "EventToObject",
"Perspective": #{},
"OcelItems": ocelItems
}).SetPrimaryKey(m.OcelEventToObject.PrimaryKey)
.Select(["OcelEventToObjectSourceId", "OcelEventToObjectTargetId", "OcelEventToObjectQualifier"]);
let newEventToObjectDt = PersistDataFrame(eventToObjectDf, "EventToObject", newConfiguration.OcelDataSource, `${newModelName} - event-to-object`);

let eventsDf = m.OcelEvents.SqlDataFrame
.Join(eventToObjectDf.SelectDistinct(["OcelEventToObjectSourceId"]), ["OcelEventId": "OcelEventToObjectSourceId"], "inner")
.RemoveColumns(["OcelEventToObjectSourceId"])
.SetPrimaryKey(m.OcelEvents.PrimaryKey);
let newEventsDt = PersistDataFrame(eventsDf, "Events", newConfiguration.OcelDataSource, `${newModelName} - events`);

WriteLog("Creating event type tables...");

m.Configuration.OcelDataSource.EventTypes.ToArray().{
let ar = _;
let eventTypeName = GetContext(ar);
let eventTypeDt = m.OcelEventType(eventTypeName);
let eventTypeDf = eventTypeDt.SqlDataFrame
.Join(newEventsDt.SqlDataFrame.Select(["OcelEventId"]), ["OcelEventTypeEventId": "OcelEventId"], "inner")
.RemoveColumns(["OcelEventId"])
.SetPrimaryKey(eventTypeDt.PrimaryKey);
PersistDataFrame(eventTypeDf, eventTypeName, newConfiguration.OcelDataSource.EventTypes, `${newModelName} - eventtype - ${eventTypeName}`);
};

WriteLog("Creating object type tables...");
let newUniqueObjectsDf = newObjectsDt
.SqlDataFrame
.SelectDistinct(["OcelObjectId"]);

m.Configuration.OcelDataSource.ObjectTypes.ToArray().{
let ar = _;
let objectTypeName = GetContext(ar);
let objectTypeDt = m.OcelObjectType(objectTypeName);
let objectTypeDf = objectTypeDt.SqlDataFrame
.Join(newUniqueObjectsDf, ["OcelObjectTypeObjectId": "OcelObjectId"], "inner")
.RemoveColumns(["OcelObjectId"])
.SetPrimaryKey(objectTypeDt.PrimaryKey);
PersistDataFrame(objectTypeDf, objectTypeName, newConfiguration.OcelDataSource.ObjectTypes, `${newModelName} - objecttype - ${objectTypeName}`);
};

WriteLog(`Creating model ${newModelName}...`);

let newModel = targetProject.CreateModel(#{"Name": newModelName, "Configuration": newConfiguration});

WriteLog(`Model (id=${newModel.Id}) created with configuration:\r\n${ToJson(newConfiguration)}`);
</syntaxhighlight>

Expression Script Examples

2026-01-12T08:18:53Z

MarHink:

This page contains script examples written in the QPR ProcessAnalyzer expression language. See how expression scripts can be created in the [[Managing_Scripts#Creating_Script|Workspace]]. For documentation for the syntax, functions and entities can be found from the main page in the [[QPR_ProcessAnalyzer_Wiki#For_Developers|KPI Expression Language]] section.

== Calling Expression Script from Expression ==
Expression scripts can be called from an expression using the [[QPR_ProcessAnalyzer_Objects_in_Expression_Language#Script|Run]] function with the following syntax:
<syntaxhighlight lang="typescript" line>
let result = ScriptById(123).Run(#{
"parameter1": "value1",
"parameter2": false,
"parameter2": 123.45
})
</syntaxhighlight>

The command waits until the run is completed, and the return value of the called script is returned by the Run function call.

Parameters can be passed to the called script, and the parameters are available as variables in the script. The parameters can contain any type of data.

Expression scripts can also be called from a dashboard. Expressions can be stored to scripts instead of dashboards, which is a way to separate complex expressions from dashboards and allow to reuse expressions across several dashboards.

== Calling SQL Script from Expression ==
SQL script can be called from an expression using the Run function as follows (similar to calling [[#Calling Expression Script from Expression|expression scripts]]):
<syntaxhighlight lang="typescript" line>
let result = ScriptById(123).Run(#{
"parameter1": "value1",
"parameter2": 321
});
let arrayOfAllReports = result.Keys;
let report1 = result.Report1;
let report2 = result.Report2;
</syntaxhighlight>

SQL scripts can return multiple ''reports'', which are combined to a dictionary, where the key is the name of the report ("sheet name") and value is the report data as a DataFrame. See in the above example, how the reports can be accessed by their name.

== Examples ==
=== Call web service===
Contact to a web service, fetch some data, and store it to a datatable.

<syntaxhighlight lang="typescript" line>
let datatableName = "Web Service Data";
let webServiceData = CallWebService(
#{"Address": "https://processanalyzer.onqpr.com/qprpa/api/serverinfo"}
);

let targetDatatable = Project.Datatables.Where(name==datatableName);
if (Count(targetDatatable) == 0) {
targetDatatable = Project.CreateDatatable(datatableName)
.AddColumn("Setting name", "String")
.AddColumn("Setting value", "String")
.AddColumn("Data read", "DateTime");
} else {
targetDatatable = targetDatatable[0];
}

let currentTime = Now;
let dataAsDf = ToDataFrame(
webServiceData.keys.{
let key = _;
[key, webServiceData[key], currentTime];
},
["Setting name", "Setting value", "Data read"]
);
targetDatatable.Import(dataAsDf, #{"Append":true});
WriteLog(`${CountTop(dataAsDf.Rows)} rows written to datatable`);
</syntaxhighlight>

=== Store data to datatable ===
Get all models in the system and store them to a datatable.

<syntaxhighlight lang="typescript" line>
let newDatatable = Project
.CreateDatatable("Models list " + ToString(Now, "dd.MM.yyyy HH:mm:ss"))
.AddColumn("Model name", "String")
.AddColumn("Project name", "String")
.AddColumn("Created time", "DateTime")
.AddColumn("Cases", "Integer");
let startTime = Now;
let modelsData = ToDataFrame(
Models.([Name, Project.Name, CreatedDate, NCases]),
["Model name", "Project name", "Created time", "Cases"]
);
WriteLog(`Listing models took ${(Now - startTime).TotalSeconds.Round(2)} seconds.`);
newDatatable.Import(modelsData);
WriteLog(`Datatable ${newDatatable.Id} created.`);
</syntaxhighlight>

=== Convert datatable column data (Snowflake) ===
Following script converts textual data from the column "TimestampString" to dates to column "Timestamp" by trying different time formats and using the first suitable.

<syntaxhighlight lang="typescript" line>
DatatableById(1)
.UpdateRows(
true,
"Timestamp",
Coalesce(
TryToTimestamp(Column("TimestampString"), "DD.MM.YYYY HH24:MI:SS.FF3"),
TryToTimestamp(Column("TimestampString"), "YYYY-MM-DD HH24:MI:SS.FF3"),
TryToTimestamp(Column("TimestampString"), "YYYY/MM/DD HH24:MI:SS.FF3")
)
);
</syntaxhighlight>

=== Convert datatable column data (in-memory) ===
This script can be used to convert a single column into numerical data type. To use the script, you need to setup the following in the beginning of the script:
* Project name where the datatable is located.
* Datatable name
* Name of the column to be converted

Note that the conversion fails, if there is data that cannot be converted into numerical format. The conversion assumes that period (.) is used as the decimal point.

<syntaxhighlight lang="typescript" line>
let projectName = "New Project";
let datatableName = "qpr processanalyzer events";
let columnName = "Event order in case";

let project = (Projects.Where(Name==projectName))[0];
let datatable = (project.Datatables.Where(Name==datatableName))[0];
DatatableById(datatable.Id).DataFrame
.SetColumns([
columnName: () => {
let data = Column(columnName);
if (data == null) {
null;
} else {
ToFloat(data);
}
}
])
.Persist(datatable.Name, ["ProjectId": project.Id, "Append": false]);
</syntaxhighlight>

Instead of converting to numeric (with the ''ToFloat'' function), data can be converted into string using the ''ToString'' function.

=== Show DataFrame as HTML table ===

This script defines a function to show dataframe as a HTML table, and uses the function for a literal dataframe.

<syntaxhighlight lang="typescript" line>
function dataframeToHtmlTable(df) {
return
`<table>
<tr>
${StringJoin("\r\n\t\t", + df.columns.`<th>${_}</th>`)}
</tr>
${StringJoin("", df.Rows.(
"\r\n\t<tr>" + StringJoin("", _.`\r\n\t\t<td>${ToString(_)}</td>`) + "\r\n\t</tr>"
))}
</table>`
}

let data = ToDataFrame(
[
["one", "two", "three"],
["four", "five", "six"],
["seven", "eight", "nine"]
],
["Column 1", "Column 2", "Column 3"]
);

return dataframeToHtmlTable(data);
</syntaxhighlight>

=== Copy local datatables to Snowflake ===

<syntaxhighlight lang="typescript" line>
// Copies all datatables in a project to another project including datatable contents.
// Usage instructions:
// 1. Create expression script in the project from where you want to copy the datatables.
// 2. Create a new project named as "<name of the project to be moved> - Snowflake". New datatables will be created here. E.g., when moving project named "SAP_OrderToCash", the target project should be named as "SAP_OrderToCash - Snowflake".
// 3. Run the script.
// NOTE: Columns of type "Any" will be created as "String"-columns in Snowflake, thus it is recommended that actual data types are set for the tables prior to the move.

let sourceProject = Project;
let sourceProjectName = Project.Name;
let targetProjectName = `${sourceProjectName} - Snowflake`;
let targetProject = First(Projects.Where(Name == targetProjectName));
if (IsNull(targetProject)) {
WriteLog(`Unable to find target project named "${targetProjectName}". Aborting operation.`);
return;
}
let dts = sourceProject.DataTables;
WriteLog(`Copying all ${CountTop(dts)} data tables found in project "${sourceProject.Name}" (id: ${sourceProject.Id}) to Snowflake in project "${targetProject.Name}" (id: ${targetProject.Id})`);
dts.{
let sourceDt = _;
WriteLog(`Starting to copy data table "${Name}" (id: ${Id}) having ${NRows} rows and ${NColumns} columns.`);
let targetDt;
targetDt = targetProject.DatatableByName(sourceDt.Name);
if (targetDt == null) {
targetDt = targetProject.CreateDataTable(sourceDt.Name, #{"Connection": CreateSnowflakeConnection(#{"ProjectId": targetProject.Id})});
targetDt.Import(sourceDt.SqlDataFrame);
WriteLog(`Finished copying data table "${Name}" (id: ${Id}) to table "${targetDt.Name}" (id: ${targetDt.Id})`);
} else {
WriteLog(`Datatable already exist "${Name}" (id: ${Id}) to table "${targetDt.Name}" (id: ${targetDt.Id})`);
}
}
WriteLog(`Finished copying all the data tables found in project "${sourceProject.Name}" (id: ${sourceProject.Id}) to Snowflake in project "${targetProject.Name}" (id: ${targetProject.Id})`);
</syntaxhighlight>

If you don't need to copy the data but only create the Snowflake datatables with columns, you can change the line 22 to
<syntaxhighlight>
targetDt.Import(sourceDt.SqlDataFrame.head(0));
</syntaxhighlight>

=== Copy single datatable to Snowflake ===
This script creates a copy of a single datatable to Snowflake. Replace the ''<tableId1>'' with the id of the source datatable.
<syntaxhighlight lang="typescript" line>
function CopyDataTableToSnowflake(dataTableId)
{
let sourceDt = DataTableById(dataTableId);
sourceDt.SqlDataFrame.Persist(`${sourceDt.Name} - Snowflake`, #{"Append": false, "Connection": CreateSnowflakeConnection(#{"ProjectId": sourceDt.Project.Id})});
}
CopyDataTableToSnowflake(<tableId1>);
</syntaxhighlight>

=== Create a copy of a data table that has all Any-type columns changed to String-type columns ===
<syntaxhighlight lang="typescript" line>
function ConvertAnyDataTypesToStringsToNewTable(dataTableId)
{
let dt = DataTableById(dataTableId);
let sdf = dt.SqlDataFrame;
let cts = dt.ColumnTypes;
cts.{
let ct = _;
if (ct.DataType == "Any") {
let n = ct.Name;
sdf = sdf.WithColumn(ct.Name, #sql{Cast(Column(Variable("n")), "ShortString")});
}
};
sdf.Persist(`${dt.Name} - Converted`, #{"Append": false, "ProjectId": dt.Project.Id});
}
ConvertAnyDataTypesToStringsToNewTable(<dataTableId>);
</syntaxhighlight>

=== Query number of rows in given data table having a datetime value in given year grouped by month and return resulting table as CSV ===
SqlDataFrame is used in order to prevent loading the whole datatable into memory first. Filtering is performed as first operation in order to minimize the amount of required work for the data source of the data table.<syntaxhighlight lang="typescript" line="1">
DataTableById(<data table id>)
.SqlDataFrame
.Where(#sql{2014 == Year(Column("Start Time"))})
.WithColumn("Month", #sql{Month(Column("Start Time"))})
.GroupBy(["Month"]).Aggregate(["Count"], ["Count"])
.OrderByColumns(["Month"], [true])
.Collect().ToCsv();
</syntaxhighlight>

=== Function for filtering SqlDataFrame by removing rows having, or replacing, the most infrequently occurring column values ===
<syntaxhighlight lang="typescript" line="1">
/***
* @name ColumnWithMinUsage
* @descripion
* Generic function that can be used to filter out the most infrequently occurring attribute values or replace their Values
* with given common value.
* @param df:
* DataFrame to operate on.
* @param columnName:
* Name of the column to be filtered.
* @param newColumnName:
* Name of the column that will contain the new value of the original column after filtering (if includeOthers was applied).
* @param maxNumUniqueValues:
* Maximum number of unique values to include into the comparison for each attribute column. If the amount of unique values for any attribute exceeds this value, only given number of attributes are included that have the highest usage.
* @param minValueUsage:
* Minimum total usage of a value included into the comparison. The number of cases having every returned value should be at least given percentage (a float value between 0.0 and 1.0) of all the compared cases.
* @param includeOthers:
* Should the rest of the attribute values not included due to MinValueUsage or MaxNumUniqueValues filtering be included as an aggregated "Others" value?
* If not empty/null, defines the name used for these other-values.
*/
function ColumnWithMinUsage(df, columnName, newColumnName, maxNumUniqueValues, minValueUsage, includeOthers)
{
let all = df
.GroupBy([])
.Aggregate(["NAllTotal"], ["Count"])
.WithColumn("__Join2", #sql{1});
let minValueUsageEnabled = !IsNullTop(minValueUsage);
let maxNumUniqueValuesEnabled = !IsNullTop(maxNumUniqueValues);
if (minValueUsageEnabled || maxNumUniqueValuesEnabled) {
// Perform column value-based filtering if minValueUsageEnabled or maxNumUniqueValuesEnabled is defined.
let valueColumnName = "__ValueNew";
let filteredValuesColumns = [valueColumnName: columnName];
let filteredValues = df
.GroupBy([columnName]).Aggregate(["Count"], ["Count"]);
if (minValueUsageEnabled) {
filteredValues = filteredValues
.WithColumn("__Join", #sql{1})
.Join(all, ["__Join": "__Join2"], "leftouter")
.WithColumn("Usage", #sql{Column("Count") / Column("NAllTotal")});
filteredValuesColumns = Concat(filteredValuesColumns, ["Usage"]);
}
if (maxNumUniqueValuesEnabled) {
filteredValues = filteredValues
.WithRowNumberColumn("RowNumber", ["Count"], null, [false]);
filteredValuesColumns = Concat(filteredValuesColumns, ["RowNumber"]);
}

filteredValues = filteredValues
.Select(filteredValuesColumns);

// Generate select returning all the accepted values.
let allValues = filteredValues
.(minValueUsageEnabled ? Where(#sql{Column("Usage") >= #expr{minValueUsage}}) : _)
.(maxNumUniqueValuesEnabled ? Where(#sql{Column("RowNumber") <= #expr{maxNumUniqueValues}}) : _)
.Select([valueColumnName, newColumnName: valueColumnName]);

if (!IsNullTop(includeOthers)) {
// If includeOthers is defined, replace original values with the variable defined in includeOthers.
let otherValues = filteredValues
.(minValueUsageEnabled ? Where(#sql{Column("Usage") < #expr{minValueUsage}}) : _)
.(maxNumUniqueValuesEnabled ? Where(#sql{Column("RowNumber") > #expr{maxNumUniqueValues}}) : _)
.WithColumn(newColumnName, #sql{#expr{includeOthers}})
.Select([valueColumnName, newColumnName]);
allValues = allValues.Append(otherValues)
}
df.Join(allValues, [columnName: valueColumnName], "inner")
.RemoveColumns([valueColumnName]);
}
}

// The following example will return only rows containing two of the most common values for Region-column.
//let df = DataTableById(<data table id>).SqlDataFrame;
//df = ColumnWithMinUsage(df, "Region", "_Filtered", 2, null, null);
//df.Collect().ToCsv();

// The following example will return all input rows, but will replace the values of rows whose Region-column
// has a value used by less than 15% of all the rows with a new value: "_Others".
//let df = DataTableById(<data table id>).SqlDataFrame;
//df = ColumnWithMinUsage(df, "Region", "_Filtered", null, 0.15, "_Others");
//df.Collect().ToCsv();
</syntaxhighlight>

=== Export model events and cases ===
<syntaxhighlight lang="typescript" line="1">
function ExportModelEvents(m) {
let attrs = m.EventAttributes;
ToDataFrame(
m.EventLog.Events.Concat(
[Case.Name, Type.Name, ToString(TimeStamp, "yyyy-MM-dd HH:mm:ss.fff")],
{let evt = _; attrs.{let att = _; evt.Attribute(att)}}
),
Concat(
["CaseId", "EventType", "TimeStamp"],
attrs.Name
)
).ToCsv(true);
}

function ExportModelCases(m) {
let attrs = m.CaseAttributes;
ToDataFrame(
m.EventLog.Cases.Concat(
[Name],
{let cas = _; attrs.{let att = _; cas.Attribute(att)}}
),
Concat(
["CaseId"],
attrs.Name
)
).ToCsv(true);
}
</pre>

<pre>
First(Models.Where(Name=="SAP OtC Extended")).EventsDataTable.DataFrame.ToCsv(true)
</pre>

<pre>
First(Models.Where(Name=="SAP OtC Extended")).CasesDataTable.DataFrame.ToCsv(true)
</syntaxhighlight>

=== Calculate all the value usages of a single column for each event in event data table ===
This query could be used, e.g., to find out the maximum resource usage for every resource found in the event data table.<syntaxhighlight lang="typescript" line="1">
function WithUsageColumns(resourceColumn)
{
function WithTotalUsageColumnOfSingleResource(resourceColumn, resourceValue)
{
_
.WithColumn("_Prev", #sql{Lag(Column(resourceColumn), [TimeStamp, EventType], [true, true], [CaseId], 1, null)})
.WithColumn("_UsageDiff", #sql{
CaseWhen(
Column(resourceColumn) == Column("_Prev"), 0,
Column("_Prev") == #expr{resourceValue}, -1,
Column(resourceColumn) == #expr{resourceValue}, 1,
0)
})
.WithColumn(`${resourceValue}_Usage`, #sql{Sum(Column("_UsageDiff"), [TimeStamp, EventType])})
.RemoveColumns(["_Prev", "_UsageDiff"])
}

let sdf = _;
let allValues = sdf.SelectDistinct([resourceColumn]).OrderByColumns([resourceColumn], [true]).Collect().Column(resourceColumn);
allValues.{
let v = _;
sdf = sdf.WithTotalUsageColumnOfSingleResource(resourceColumn, v)
}
sdf
}

let dt = ModelById(<model id>).EventsDataTable;
dt
.SqlDataFrame
.WithUsageColumns(<resource column name>)
.OrderByColumns([dt.ColumnMappings["TimeStamp"]], [true])
.Collect().ToCsv()
</syntaxhighlight>Where:

* <model id> is the id of the model containing event data to be examined.
* <resource column name> is the name of the column in the event data table of the specified model containing the resource being used by that event.

NOTE: This expression uses functionalities that are only supported in Snowflake-based data tables.

=== Create new Snowflake model from filter ===
This script creates a new Snowflake model (and two datatables for cases and events) containing filtered event log from given filter id. The script also works if the model doesn't have a cases datatable.

<syntaxhighlight lang="typescript" line="1">
let filter = FilterById(1); // filter id
let model = filter.model;
let project = model.project;
let nameSuffix = " - " + filter.name + " - " + ToString(Now, "dd-MM-yyyy HH:mm:ss");
let eventsDatatableName = model.EventsDataTable.Name + nameSuffix;
if (eventsDatatableName.length > 440) {
eventsDatatableName = eventsDatatableName.Substring(eventsDatatableName.length - 440);
}
let eventsData = model
.EventsDataTable
.SqlDataFrame
.ApplyFilter(
filter.rules,
model.CasesDataTable?.SqlDataFrame
);
project
.CreateDatatable(eventsDatatableName, #{"Connection": CreateSnowflakeConnection()})
.Import(eventsData);
let modelConfiguration = model.Configuration;
modelConfiguration.DataSource.Events.Set("DataTableName", eventsDatatableName);
if (model.CasesDataTable != null) {
let eventsDataCaseIdColumn = "CaseId_" + ToString(Random());
let casesDatatableName = model.CasesDataTable.Name + nameSuffix;
if (casesDatatableName.length > 440) {
casesDatatableName = casesDatatableName.Substring(casesDatatableName.length - 440);
}
let casesData = model
.CasesDataTable
.SqlDataFrame
.join(
eventsData.SelectDistinct([eventsDataCaseIdColumn: modelConfiguration.DataSource.Events.Columns.CaseId]),
[modelConfiguration.DataSource.Cases.Columns.CaseId: eventsDataCaseIdColumn]
).Select(model.CasesDataTable.ColumnNames);
project
.CreateDatatable(casesDatatableName, #{"Connection": CreateSnowflakeConnection()})
.Import(casesData);
modelConfiguration.DataSource.Cases.Set("DataTableName", casesDatatableName);
}
let modelName = model.Name + nameSuffix;
if (modelName > 440) {
modelName = modelName.Substring(modelName.length - 440);
}
project
.CreateModel(#{
"Name": modelName,
"Description": model.Description,
"Configuration": modelConfiguration
});
return modelName;
</syntaxhighlight>

=== Creating a model consisting of multiple copies of cases in an existing model ===
<syntaxhighlight lang="typescript" line="1">
/**
* @name CreateTestModel
* @description
* Creates a new model (or overwrites an existing) to given target project with given number of
* repetitions of given source model.
* Each repetition will generate "<N>-"-prefix to CaseId-columns, where N equals to the repeat index.
* @param sourceModel
* PA model used for the source data and from where the connection is copied for the target model if a
* new one has to be created.
* @param numRepeats
* Number of times the data in the source model should be repeated in the generated model.
* @param targetProject
* Project in which the target model resides.
* @param targetModelName
* Specifies the name of the test model in the given target project. If a model already exists with
* given name, event and case data in this model will be replaced with the new generated event and
* case data.
* @returns
* Model object of the test model having the newly generated data.
*/
function CreateTestModel(sourceModel, numRepeats, targetProject, targetModelName)
{
let eventsColumnMappings = sourceModel.EventsDataTable.ColumnMappings;
let casesColumnMappings = sourceModel.CasesDataTable.ColumnMappings;
let connection = sourceModel.EventsDataTable.DataSourceConnection;

function CreateResultModel()
{
function GetTable(tableName)
{
let tableConfiguration = #{
"Name": tableName,
"Connection": connection
};
let resultTable = targetProject.DataTableByName(tableName);
if (resultTable == null)
{
resultTable = targetProject.CreateDataTable(tableConfiguration)
.Modify(#{"NameInDataSource": null})
.Synchronize();
}
return resultTable;
}

let eventsTableName = `${targetModelName} - events`;
let casesTableName = `${targetModelName} - cases`;
let targetModel = targetProject.ModelByName(targetModelName);
let eventsTable, casesTable = null;

if (targetModel != null)
{
eventsTable = targetModel.EventsDataTable;
casesTable = targetModel.CasesDataTable;
}
else {
eventsTable = GetTable(eventsTableName);
if (sourceModel.CasesDataTable != null) {
casesTable = GetTable(casesTableName);
}

let timestampMapping = eventsColumnMappings["TimeStamp"];
eventsColumnMappings.Remove("TimeStamp");
eventsColumnMappings.Set("Timestamp", timestampMapping);

let modelConfiguration = #{
"DataSource": #{
"Events":#{
"DataSourceType": "datatable",
"DataTableName": eventsTableName,
"Columns": eventsColumnMappings
}
}
};

if (casesColumnMappings != null) {
modelConfiguration["DataSource"].Set("Cases", #{
"DataSourceType": "datatable",
"DataTableName": casesTableName,
"Columns": casesColumnMappings
});
}

targetModel = targetProject.CreateModel(#{"Name": targetModelName, "Configuration": modelConfiguration});
}

eventsTable.Truncate();
casesTable?.Truncate();

return #{
"TargetModel": targetModel,
"Events": eventsTable,
"Cases": casesTable
};
}

function RepeatNTimes(sourceDf, caseIdColumn, numRepeats)
{
let resultDf = null;
for (let i = 1; i <= numRepeats; ++i) {
let iterationDf = sourceDf
.WithColumn(caseIdColumn, #sql{Concat(#expr{i}, "-", Column(#expr{caseIdColumn}))});
resultDf = resultDf == null ? iterationDf : resultDf.Append(iterationDf);
}
resultDf;
}

let resultModel = CreateResultModel();
let sourceEventDataDf = sourceModel.EventsDataTable.SqlDataFrame;
let resultEventDataDf = RepeatNTimes(sourceEventDataDf, eventsColumnMappings["CaseId"], numRepeats);
resultModel["Events"].Import(resultEventDataDf);

let sourceCaseDataDf = sourceModel.CasesDataTable?.SqlDataFrame;
if (sourceCaseDataDf != null) {
let resultCaseDataDf = RepeatNTimes(sourceCaseDataDf, casesColumnMappings["CaseId"], numRepeats);
resultModel["Cases"].Import(resultCaseDataDf);
}
resultModel["TargetModel"];
}

</syntaxhighlight>Example usage:<blockquote>CreateTestModel(ProjectByName("Project").ModelByName("SAP_OrderToCash - Snowflake"), 3, ProjectByName("TestData"), "TestModel");</blockquote>Creates a new model named "TestModel" (or overwrites old one) into project named "TestData" containing the data from model "SAP_OrderToCash - Snowflake" in project "Project" repeated three times.

=== Analyzing declare patterns found in event log ===

This is an example expression that shows how POSIX-style regular expressions can be used to search for cases in an event log having certain event type patterns [https://www.researchgate.net/publication/277631859_Generating_Event_Logs_Through_the_Simulation_of_Declare_Models declare patterns].

Note: Before use, replace <model id> with a valid model identifier having event data.

<syntaxhighlight lang="typescript" line="1">
let dt = ModelById(<model id>).EventsDataTable;
let sdf = dt.SqlDataFrame.Head(1000);
let caseIdColumn = dt.ColumnMappings["CaseId"], timeStampColumn = dt.ColumnMappings["TimeStamp"], eventTypeColumn = dt.ColumnMappings["EventType"];

let eventTypesDf = sdf
.SelectDistinct([eventTypeColumn])
.OrderByColumns([eventTypeColumn], [true])
.WithRowNumberColumn("Token", [eventTypeColumn])
.WithColumn("Token", #sql{Char(Column("Token") + Unicode("a") - 1)});

sdf = sdf
.Join(eventTypesDf.Select(["_EventType2": eventTypeColumn, "Token"]), [eventTypeColumn: "_EventType2"], "leftouter");

function RespondedExistencePattern(a, b)
{
// [^a]*((a.*b.*)|(b.*a.*))*[^a]*
ToSqlExpression(`Regexp(Column("Trace"), "[^${a}]*((${a}.*${b}.*)|(${b}.*${a}.*))*[^${a}]")`)
}

function ResponsePattern(a, b)
{
// [^a]*(a.*b)*[^a]*
ToSqlExpression(`Regexp(Column("Trace"), "[^${a}]*(${a}.*${b})*[^${a}]")`)
}

function AlternateResponsePattern(a, b)
{
// [^a]*(a[^a]*b[^a]*)*[^a]*
ToSqlExpression(`Regexp(Column("Trace"), "[^${a}]*(${a}[^${a}]*${b}[^${a}]*)*[^${a}]")`)
}

function ChainResponsePattern(a, b)
{
// [^a]*(ab[^a]*)*[^a]*
ToSqlExpression(`Regexp(Column("Trace"), "[^${a}]*(${a}${b}[^${a}]*)*[^${a}]")`)
}

function PrecedencePattern(a, b)
{
// [^b]*(a.*b)*[^b]*
ToSqlExpression(`Regexp(Column("Trace"), "[^${b}]*(${a}.*${b})*[^${b}]")`)
}

let tracesDf = sdf
.GroupBy([caseIdColumn])
.Aggregate(["Trace": "Token"], [#{"Function": "list", "Ordering": [timeStampColumn], "Separator": ""}])
.WithColumn("RespondedExistenceNL", #sql{#expr{RespondedExistencePattern("n", "l")}})
.WithColumn("RespondedExistenceCA", #sql{#expr{RespondedExistencePattern("c", "a")}})
.WithColumn("ResponsePatternNL", #sql{#expr{ResponsePattern("n", "l")}})
.WithColumn("ResponsePatternCA", #sql{#expr{ResponsePattern("c", "a")}})
.WithColumn("AlternateResponsePatternNL", #sql{#expr{AlternateResponsePattern("n", "l")}})
.WithColumn("AlternateResponsePatternCA", #sql{#expr{AlternateResponsePattern("c", "a")}})
.WithColumn("ChainResponsePatternNL", #sql{#expr{ChainResponsePattern("n", "l")}})
.WithColumn("ChainResponsePatternCA", #sql{#expr{ChainResponsePattern("c", "a")}})
.WithColumn("PrecedencePatternNL", #sql{#expr{PrecedencePattern("n", "l")}})
.WithColumn("PrecedencePatternCA", #sql{#expr{PrecedencePattern("c", "a")}})
;

[tracesDf.Collect().ToCsv(), eventTypesDf.Collect().ToCsv()]
</syntaxhighlight>

=== Perform a query and send results as E-mail in a HTML table ===
This example requires two scripts that are both located in the same project:

# Expression-type script to execute (can be, e.g., scheduled to run daily).
# Expression-type script containing the query JSON to use as basis for the e-mail. In this example, this script is named as "Send query as E-mail - query JSON". The contents of this script is just the JSON representation of a query that can be extracted, e.g., from any PA chart view.

Script #1 should contain the following code:<syntaxhighlight lang="typescript">
let replyToAddress = "noreply@test.com";
let recipientsArray = ["testuser@test.com"];
let queryConfiguration = ParseJson(Project.ScriptByName("Send query as E-mail - query JSON").Code)
.Set("IncludeCollect", true);
let resultDf = Query(queryConfiguration);
let mailBodyHtml = resultDf.`
<table>
<caption>Example query</caption>
<thead>
<tr>
${StringJoin("", _.Columns.`
<th>${_}</th>
`)}
</tr>
</thead>
<tbody>
${StringJoin("", _.Rows.`
<tr>
${StringJoin("", _.`
<td>${_}</td>
`)}
</tr>
`)}
</tbody>
</table>
`;

SendEmail(#{
"ReplyTo": [replyToAddress],
"To": recipientsArray,
"Subject": "Example query E-mail",
"IsBodyHtml": true,
"Body": mailBodyHtml
});

</syntaxhighlight>NOTE: QPR ProcessAnalyzer Server has to have SMTP server configured. Also, remember to update the values of replyToAddress and recipientsArray before using.

=== Converting a case-centric model to object-centric model ===
This function serves as an example on how a case-centric model could be converted into an object-centric model having just one object type: "Case".

Note: Before use, replace <model id> with a valid model identifier having event data.<syntaxhighlight lang="typescript">
function ConvertCCModelToOCModel(model, newModelName)
{
let connection = model.EventsDataTable.DataSourceConnection;
let caseIdColumn = model.EventsDataTable.ColumnMappings["CaseId"];
let eventToObjectTableName = `${newModelName} - event-to-object`;
let eventsTableName = `${newModelName} - events`;
let objectsTableName = `${newModelName} - objects`;

let eventsDf = model.EventsDataTable
.SqlDataFrame
.RenameColumns([
"OcelEventType": model.EventsDataTable.ColumnMappings["EventType"],
"OcelEventTime": model.EventsDataTable.ColumnMappings["TimeStamp"]])
.WithColumn("OcelEventId", #sql{Concat("evt-", Cast(RowNumber([Column("OcelEventTime")]), "String"))});

eventsDf
.RenameColumns([
"OcelEventToObjectSourceId": "OcelEventId",
"OcelEventToObjectTargetId": caseIdColumn])
.Select(["OcelEventToObjectSourceId", "OcelEventToObjectTargetId"])
.WithColumn("OcelEventToObjectQualifier", #sql{#expr{caseIdColumn} })
.Persist(eventToObjectTableName, #{"Connection": connection});

eventsDf
.RemoveColumns([caseIdColumn])
.Persist(eventsTableName, #{"Connection": connection});

let casesDt = model.CasesDataTable
.SqlDataFrame
.RenameColumns([
"OcelObjectId": model.CasesDataTable.ColumnMappings["CaseId"]
])
.WithColumn("OcelObjectType", #sql{"Case"})
.Persist(objectsTableName, #{"Connection": connection});

let newConfiguration = #{
"OcelDataSource": #{
"Events": eventsTableName,
"Objects": objectsTableName,
"EventToObject": eventToObjectTableName
}
};

model.Project
.CreateModel(#{
"Name": newModelName,
"Description": model.Description,
"Configuration": newConfiguration
});
}

let ccModel = ModelById(<model id>);
ConvertCCModelToOCModel(ccModel, `ocel - ${ccModel.Name}`);
</syntaxhighlight>

=== Show duplicate rows in datatable ===
This script returns all rows in a datatable appearing more than once (sorted by the most frequent occurrences first). Additionally, the number of occurrences is returned as the last column. Replace the datatable id with the correct one.

<syntaxhighlight lang="typescript">
let datatableId = 1;
let rowCountColumn = "Row count";
let columns = DatatableById(datatableId).Columns.Name;
DatatableById(datatableId)
.SqlDataFrame
.GroupBy(columns)
.Aggregate(
[rowCountColumn: columns[0]],
["Count"]
)
.Where(Column(rowCountColumn) > 1)
.OrderByColumns([rowCountColumn], [false])
.Collect()
.ToCSV();
</syntaxhighlight>

=== Create a new object-centric model based on already existing object-centric model containing only filtered objects and events ===
Script for creating a new OCPM-model based on another already existing model in PA that applies given filter to the original model and creates all the needed tables to store only the relevant rows for all the OCPM data tables.<syntaxhighlight lang="typescript">
let newModelName = "<target model name>";
let sourceModelId = <source model id>;
let targetProjectId = <target project id>;

// Replace the value of the ocelItems below with the configuration of object-centric filter to apply.
// Note: Use expression language syntax here, not JSON.
let ocelItems = [#{
"Include": true,
"ObjectAttributeValue": #{
"ObjectType": "Purchase Order",
"Attribute": "po_product",
"Values": [
"0Cows"
]
}
}];

let m = ModelById(sourceModelId);
if (!m.IsOcelModel)
throw `Model ${m.Name} is not an OCPM model`;
if (CountTop(m.CheckModelValidity()) > 0)
throw `Model ${m.Name} is not a valid OCPM model`;

let targetProject = ProjectById(targetProjectId);
if (CountTop(targetProject.Models.Where(Name == newModelName)) > 0)
throw `Model having name ${newModelName} already exists in project ${targetProject.Name}. Rename or delete the existing model before running this script.`;

let newConfiguration = #{
"OcelDataSource": #{
"EventTypes": #{},
"ObjectTypes": #{}
}
};

WriteLog("Creating common tables...");

function PersistDataFrame(dataFrame, configurationName, configurationDict, dataTableName)
{
WriteLog(`Creating filtered table: ${dataTableName}`);
configurationDict.Set(configurationName, dataTableName);
return dataFrame.Persist(dataTableName, #{"ProjectId": targetProjectId, "Append": false});
}

let objectsDf = m.CacheTableSqlDataFrame(#{
"CacheTableType": "Objects",
"Perspective": #{},
"OcelItems": ocelItems
}).SetPrimaryKey(m.OcelObjects.PrimaryKey);
let newObjectsDt = PersistDataFrame(objectsDf, "Objects", newConfiguration.OcelDataSource, `${newModelName} - objects`);

let objectToObjectDf = m.CacheTableSqlDataFrame(#{
"CacheTableType": "ObjectToObject",
"Perspective": #{},
"OcelItems": ocelItems
}).SetPrimaryKey(m.OcelObjectToObject.PrimaryKey)
.Select(["OcelObjectToObjectSourceId", "OcelObjectToObjectTargetId", "OcelObjectToObjectQualifier"]);
let newObjectToObjectDt = PersistDataFrame(objectToObjectDf, "ObjectToObject", newConfiguration.OcelDataSource, `${newModelName} - object-to-object`);

let eventToObjectDf = m.CacheTableSqlDataFrame(#{
"CacheTableType": "EventToObject",
"Perspective": #{},
"OcelItems": ocelItems
}).SetPrimaryKey(m.OcelEventToObject.PrimaryKey)
.Select(["OcelEventToObjectSourceId", "OcelEventToObjectTargetId", "OcelEventToObjectQualifier"]);
let newEventToObjectDt = PersistDataFrame(eventToObjectDf, "EventToObject", newConfiguration.OcelDataSource, `${newModelName} - event-to-object`);

let eventsDf = m.OcelEvents.SqlDataFrame
.Join(eventToObjectDf.SelectDistinct(["OcelEventToObjectSourceId"]), ["OcelEventId": "OcelEventToObjectSourceId"], "inner")
.RemoveColumns(["OcelEventToObjectSourceId"])
.SetPrimaryKey(m.OcelEvents.PrimaryKey);
let newEventsDt = PersistDataFrame(eventsDf, "Events", newConfiguration.OcelDataSource, `${newModelName} - events`);

WriteLog("Creating event type tables...");

m.Configuration.OcelDataSource.EventTypes.ToArray().{
let ar = _;
let eventTypeName = GetContext(ar);
let eventTypeDt = m.OcelEventType(eventTypeName);
let eventTypeDf = eventTypeDt.SqlDataFrame
.Join(newEventsDt.SqlDataFrame.Select(["OcelEventId"]), ["OcelEventTypeEventId": "OcelEventId"], "inner")
.RemoveColumns(["OcelEventId"])
.SetPrimaryKey(eventTypeDt.PrimaryKey);
PersistDataFrame(eventTypeDf, eventTypeName, newConfiguration.OcelDataSource.EventTypes, `${newModelName} - eventtype - ${eventTypeName}`);
};

WriteLog("Creating object type tables...");
let newUniqueObjectsDf = newObjectsDt
.SqlDataFrame
.SelectDistinct(["OcelObjectId"]);

m.Configuration.OcelDataSource.ObjectTypes.ToArray().{
let ar = _;
let objectTypeName = GetContext(ar);
let objectTypeDt = m.OcelObjectType(objectTypeName);
let objectTypeDf = objectTypeDt.SqlDataFrame
.Join(newUniqueObjectsDf, ["OcelObjectTypeObjectId": "OcelObjectId"], "inner")
.RemoveColumns(["OcelObjectId"])
.SetPrimaryKey(objectTypeDt.PrimaryKey);
PersistDataFrame(objectTypeDf, objectTypeName, newConfiguration.OcelDataSource.ObjectTypes, `${newModelName} - objecttype - ${objectTypeName}`);
};

WriteLog(`Creating model ${newModelName}...`);

let newModel = targetProject.CreateModel(#{"Name": newModelName, "Configuration": newConfiguration});

WriteLog(`Model (id=${newModel.Id}) created with configuration:\r\n${ToJson(newConfiguration)}`);
</syntaxhighlight>

Expression Script Examples

2026-01-12T08:17:08Z

MarHink:

This page contains script examples written in the QPR ProcessAnalyzer expression language. See how expression scripts can be created in the [[Managing_Scripts#Creating_Script|Workspace]]. For documentation for the syntax, functions and entities can be found from the main page in the [[QPR_ProcessAnalyzer_Wiki#For_Developers|KPI Expression Language]] section.

== Calling Expression Script from Expression ==
Expression scripts can be called from an expression using the [[QPR_ProcessAnalyzer_Objects_in_Expression_Language#Script|Run]] function with the following syntax:
<syntaxhighlight lang="typescript" line>
let result = ScriptById(123).Run(#{
"parameter1": "value1",
"parameter2": false,
"parameter2": 123.45
})
</syntaxhighlight>

The command waits until the run is completed, and the return value of the called script is returned by the Run function call.

Parameters can be passed to the called script, and the parameters are available as variables in the script. The parameters can contain any type of data.

Expression scripts can also be called from a dashboard. Expressions can be stored to scripts instead of dashboards, which is a way to separate complex expressions from dashboards and allow to reuse expressions across several dashboards.

== Calling SQL Script from Expression ==
SQL script can be called from an expression using the Run function as follows (similar to calling [[#Calling Expression Script from Expression|expression scripts]]):
<syntaxhighlight lang="typescript" line>
let result = ScriptById(123).Run(#{
"parameter1": "value1",
"parameter2": 321
});
let arrayOfAllReports = result.Keys;
let report1 = result.Report1;
let report2 = result.Report2;
</syntaxhighlight>

SQL scripts can return multiple ''reports'', which are combined to a dictionary, where the key is the name of the report ("sheet name") and value is the report data as a DataFrame. See in the above example, how the reports can be accessed by their name.

== Examples ==
=== Call web service===
Contact to a web service, fetch some data, and store it to a datatable.

<syntaxhighlight lang="typescript" line>
let datatableName = "Web Service Data";
let webServiceData = CallWebService(
#{"Address": "https://processanalyzer.onqpr.com/qprpa/api/serverinfo"}
);

let targetDatatable = Project.Datatables.Where(name==datatableName);
if (Count(targetDatatable) == 0) {
targetDatatable = Project.CreateDatatable(datatableName)
.AddColumn("Setting name", "String")
.AddColumn("Setting value", "String")
.AddColumn("Data read", "DateTime");
} else {
targetDatatable = targetDatatable[0];
}

let currentTime = Now;
let dataAsDf = ToDataFrame(
webServiceData.keys.{
let key = _;
[key, webServiceData[key], currentTime];
},
["Setting name", "Setting value", "Data read"]
);
targetDatatable.Import(dataAsDf, #{"Append":true});
WriteLog(`${CountTop(dataAsDf.Rows)} rows written to datatable`);
</syntaxhighlight>

=== Store data to datatable ===
Get all models in the system and store them to a datatable.

<syntaxhighlight lang="typescript" line>
let newDatatable = Project
.CreateDatatable("Models list " + ToString(Now, "dd.MM.yyyy HH:mm:ss"))
.AddColumn("Model name", "String")
.AddColumn("Project name", "String")
.AddColumn("Created time", "DateTime")
.AddColumn("Cases", "Integer");
let startTime = Now;
let modelsData = ToDataFrame(
Models.([Name, Project.Name, CreatedDate, NCases]),
["Model name", "Project name", "Created time", "Cases"]
);
WriteLog(`Listing models took ${(Now - startTime).TotalSeconds.Round(2)} seconds.`);
newDatatable.Import(modelsData);
WriteLog(`Datatable ${newDatatable.Id} created.`);
</syntaxhighlight>

=== Convert datatable column data (Snowflake) ===
Following script converts textual data from the column "TimestampString" to dates to column "Timestamp" by trying different time formats and using the first suitable.

<syntaxhighlight lang="typescript" line>
DatatableById(1)
.UpdateRows(
true,
"Timestamp",
Coalesce(
TryToTimestamp(Column("TimestampString"), "DD.MM.YYYY HH24:MI:SS.FF3"),
TryToTimestamp(Column("TimestampString"), "YYYY-MM-DD HH24:MI:SS.FF3"),
TryToTimestamp(Column("TimestampString"), "YYYY/MM/DD HH24:MI:SS.FF3")
)
);
</syntaxhighlight>

=== Convert datatable column data (in-memory) ===
This script can be used to convert a single column into numerical data type. To use the script, you need to setup the following in the beginning of the script:
* Project name where the datatable is located.
* Datatable name
* Name of the column to be converted

Note that the conversion fails, if there is data that cannot be converted into numerical format. The conversion assumes that period (.) is used as the decimal point.

<syntaxhighlight lang="typescript" line>
let projectName = "New Project";
let datatableName = "qpr processanalyzer events";
let columnName = "Event order in case";

let project = (Projects.Where(Name==projectName))[0];
let datatable = (project.Datatables.Where(Name==datatableName))[0];
DatatableById(datatable.Id).DataFrame
.SetColumns([
columnName: () => {
let data = Column(columnName);
if (data == null) {
null;
} else {
ToFloat(data);
}
}
])
.Persist(datatable.Name, ["ProjectId": project.Id, "Append": false]);
</syntaxhighlight>

Instead of converting to numeric (with the ''ToFloat'' function), data can be converted into string using the ''ToString'' function.

=== Show DataFrame as HTML table ===

This script defines a function to show dataframe as a HTML table, and uses the function for a literal dataframe.

<syntaxhighlight lang="typescript" line>
function dataframeToHtmlTable(df) {
return
`<table>
<tr>
${StringJoin("\r\n\t\t", + df.columns.`<th>${_}</th>`)}
</tr>
${StringJoin("", df.Rows.(
"\r\n\t<tr>" + StringJoin("", _.`\r\n\t\t<td>${ToString(_)}</td>`) + "\r\n\t</tr>"
))}
</table>`
}

let data = ToDataFrame(
[
["one", "two", "three"],
["four", "five", "six"],
["seven", "eight", "nine"]
],
["Column 1", "Column 2", "Column 3"]
);

return dataframeToHtmlTable(data);
</syntaxhighlight>

=== Copy local datatables to Snowflake ===

<syntaxhighlight lang="typescript" line>
// Copies all datatables in a project to another project including datatable contents.
// Usage instructions:
// 1. Create expression script in the project from where you want to copy the datatables.
// 2. Create a new project named as "<name of the project to be moved> - Snowflake". New datatables will be created here. E.g., when moving project named "SAP_OrderToCash", the target project should be named as "SAP_OrderToCash - Snowflake".
// 3. Run the script.
// NOTE: Columns of type "Any" will be created as "String"-columns in Snowflake, thus it is recommended that actual data types are set for the tables prior to the move.

let sourceProject = Project;
let sourceProjectName = Project.Name;
let targetProjectName = `${sourceProjectName} - Snowflake`;
let targetProject = First(Projects.Where(Name == targetProjectName));
if (IsNull(targetProject)) {
WriteLog(`Unable to find target project named "${targetProjectName}". Aborting operation.`);
return;
}
let dts = sourceProject.DataTables;
WriteLog(`Copying all ${CountTop(dts)} data tables found in project "${sourceProject.Name}" (id: ${sourceProject.Id}) to Snowflake in project "${targetProject.Name}" (id: ${targetProject.Id})`);
dts.{
let sourceDt = _;
WriteLog(`Starting to copy data table "${Name}" (id: ${Id}) having ${NRows} rows and ${NColumns} columns.`);
let targetDt;
targetDt = targetProject.DatatableByName(sourceDt.Name);
if (targetDt == null) {
targetDt = targetProject.CreateDataTable(sourceDt.Name, #{"Connection": CreateSnowflakeConnection(#{"ProjectId": targetProject.Id})});
targetDt.Import(sourceDt.SqlDataFrame);
WriteLog(`Finished copying data table "${Name}" (id: ${Id}) to table "${targetDt.Name}" (id: ${targetDt.Id})`);
} else {
WriteLog(`Datatable already exist "${Name}" (id: ${Id}) to table "${targetDt.Name}" (id: ${targetDt.Id})`);
}
}
WriteLog(`Finished copying all the data tables found in project "${sourceProject.Name}" (id: ${sourceProject.Id}) to Snowflake in project "${targetProject.Name}" (id: ${targetProject.Id})`);
</syntaxhighlight>

If you don't need to copy the data but only create the Snowflake datatables with columns, you can change the line 22 to
<syntaxhighlight>
targetDt.Import(sourceDt.SqlDataFrame.head(0));
</syntaxhighlight>

=== Copy single datatable to Snowflake ===
This script creates a copy of a single datatable to Snowflake. Replace the ''<tableId1>'' with the id of the source datatable.
<syntaxhighlight lang="typescript" line>
function CopyDataTableToSnowflake(dataTableId)
{
let sourceDt = DataTableById(dataTableId);
sourceDt.SqlDataFrame.Persist(`${sourceDt.Name} - Snowflake`, #{"Append": false, "Connection": CreateSnowflakeConnection(#{"ProjectId": sourceDt.Project.Id})});
}
CopyDataTableToSnowflake(<tableId1>);
</syntaxhighlight>

=== Create a copy of a data table that has all Any-type columns changed to String-type columns ===
<syntaxhighlight lang="typescript" line>
function ConvertAnyDataTypesToStringsToNewTable(dataTableId)
{
let dt = DataTableById(dataTableId);
let sdf = dt.SqlDataFrame;
let cts = dt.ColumnTypes;
cts.{
let ct = _;
if (ct.DataType == "Any") {
let n = ct.Name;
sdf = sdf.WithColumn(ct.Name, #sql{Cast(Column(Variable("n")), "ShortString")});
}
};
sdf.Persist(`${dt.Name} - Converted`, #{"Append": false, "ProjectId": dt.Project.Id});
}
ConvertAnyDataTypesToStringsToNewTable(<dataTableId>);
</syntaxhighlight>

=== Query number of rows in given data table having a datetime value in given year grouped by month and return resulting table as CSV ===
SqlDataFrame is used in order to prevent loading the whole datatable into memory first. Filtering is performed as first operation in order to minimize the amount of required work for the data source of the data table.<syntaxhighlight lang="typescript" line="1">
DataTableById(<data table id>)
.SqlDataFrame
.Where(#sql{2014 == Year(Column("Start Time"))})
.WithColumn("Month", #sql{Month(Column("Start Time"))})
.GroupBy(["Month"]).Aggregate(["Count"], ["Count"])
.OrderByColumns(["Month"], [true])
.Collect().ToCsv();
</syntaxhighlight>

=== Function for filtering SqlDataFrame by removing rows having, or replacing, the most infrequently occurring column values ===
<syntaxhighlight lang="typescript" line="1">
/***
* @name ColumnWithMinUsage
* @descripion
* Generic function that can be used to filter out the most infrequently occurring attribute values or replace their Values
* with given common value.
* @param df:
* DataFrame to operate on.
* @param columnName:
* Name of the column to be filtered.
* @param newColumnName:
* Name of the column that will contain the new value of the original column after filtering (if includeOthers was applied).
* @param maxNumUniqueValues:
* Maximum number of unique values to include into the comparison for each attribute column. If the amount of unique values for any attribute exceeds this value, only given number of attributes are included that have the highest usage.
* @param minValueUsage:
* Minimum total usage of a value included into the comparison. The number of cases having every returned value should be at least given percentage (a float value between 0.0 and 1.0) of all the compared cases.
* @param includeOthers:
* Should the rest of the attribute values not included due to MinValueUsage or MaxNumUniqueValues filtering be included as an aggregated "Others" value?
* If not empty/null, defines the name used for these other-values.
*/
function ColumnWithMinUsage(df, columnName, newColumnName, maxNumUniqueValues, minValueUsage, includeOthers)
{
let all = df
.GroupBy([])
.Aggregate(["NAllTotal"], ["Count"])
.WithColumn("__Join2", #sql{1});
let minValueUsageEnabled = !IsNullTop(minValueUsage);
let maxNumUniqueValuesEnabled = !IsNullTop(maxNumUniqueValues);
if (minValueUsageEnabled || maxNumUniqueValuesEnabled) {
// Perform column value-based filtering if minValueUsageEnabled or maxNumUniqueValuesEnabled is defined.
let valueColumnName = "__ValueNew";
let filteredValuesColumns = [valueColumnName: columnName];
let filteredValues = df
.GroupBy([columnName]).Aggregate(["Count"], ["Count"]);
if (minValueUsageEnabled) {
filteredValues = filteredValues
.WithColumn("__Join", #sql{1})
.Join(all, ["__Join": "__Join2"], "leftouter")
.WithColumn("Usage", #sql{Column("Count") / Column("NAllTotal")});
filteredValuesColumns = Concat(filteredValuesColumns, ["Usage"]);
}
if (maxNumUniqueValuesEnabled) {
filteredValues = filteredValues
.WithRowNumberColumn("RowNumber", ["Count"], null, [false]);
filteredValuesColumns = Concat(filteredValuesColumns, ["RowNumber"]);
}

filteredValues = filteredValues
.Select(filteredValuesColumns);

// Generate select returning all the accepted values.
let allValues = filteredValues
.(minValueUsageEnabled ? Where(#sql{Column("Usage") >= #expr{minValueUsage}}) : _)
.(maxNumUniqueValuesEnabled ? Where(#sql{Column("RowNumber") <= #expr{maxNumUniqueValues}}) : _)
.Select([valueColumnName, newColumnName: valueColumnName]);

if (!IsNullTop(includeOthers)) {
// If includeOthers is defined, replace original values with the variable defined in includeOthers.
let otherValues = filteredValues
.(minValueUsageEnabled ? Where(#sql{Column("Usage") < #expr{minValueUsage}}) : _)
.(maxNumUniqueValuesEnabled ? Where(#sql{Column("RowNumber") > #expr{maxNumUniqueValues}}) : _)
.WithColumn(newColumnName, #sql{#expr{includeOthers}})
.Select([valueColumnName, newColumnName]);
allValues = allValues.Append(otherValues)
}
df.Join(allValues, [columnName: valueColumnName], "inner")
.RemoveColumns([valueColumnName]);
}
}

// The following example will return only rows containing two of the most common values for Region-column.
//let df = DataTableById(<data table id>).SqlDataFrame;
//df = ColumnWithMinUsage(df, "Region", "_Filtered", 2, null, null);
//df.Collect().ToCsv();

// The following example will return all input rows, but will replace the values of rows whose Region-column
// has a value used by less than 15% of all the rows with a new value: "_Others".
//let df = DataTableById(<data table id>).SqlDataFrame;
//df = ColumnWithMinUsage(df, "Region", "_Filtered", null, 0.15, "_Others");
//df.Collect().ToCsv();
</syntaxhighlight>

=== Export model events and cases ===
<syntaxhighlight lang="typescript" line="1">
function ExportModelEvents(m) {
let attrs = m.EventAttributes;
ToDataFrame(
m.EventLog.Events.Concat(
[Case.Name, Type.Name, ToString(TimeStamp, "yyyy-MM-dd HH:mm:ss.fff")],
{let evt = _; attrs.{let att = _; evt.Attribute(att)}}
),
Concat(
["CaseId", "EventType", "TimeStamp"],
attrs.Name
)
).ToCsv(true);
}

function ExportModelCases(m) {
let attrs = m.CaseAttributes;
ToDataFrame(
m.EventLog.Cases.Concat(
[Name],
{let cas = _; attrs.{let att = _; cas.Attribute(att)}}
),
Concat(
["CaseId"],
attrs.Name
)
).ToCsv(true);
}
</pre>

<pre>
First(Models.Where(Name=="SAP OtC Extended")).EventsDataTable.DataFrame.ToCsv(true)
</pre>

<pre>
First(Models.Where(Name=="SAP OtC Extended")).CasesDataTable.DataFrame.ToCsv(true)
</syntaxhighlight>

=== Calculate all the value usages of a single column for each event in event data table ===
This query could be used, e.g., to find out the maximum resource usage for every resource found in the event data table.<syntaxhighlight lang="typescript" line="1">
function WithUsageColumns(resourceColumn)
{
function WithTotalUsageColumnOfSingleResource(resourceColumn, resourceValue)
{
_
.WithColumn("_Prev", #sql{Lag(Column(resourceColumn), [TimeStamp, EventType], [true, true], [CaseId], 1, null)})
.WithColumn("_UsageDiff", #sql{
CaseWhen(
Column(resourceColumn) == Column("_Prev"), 0,
Column("_Prev") == #expr{resourceValue}, -1,
Column(resourceColumn) == #expr{resourceValue}, 1,
0)
})
.WithColumn(`${resourceValue}_Usage`, #sql{Sum(Column("_UsageDiff"), [TimeStamp, EventType])})
.RemoveColumns(["_Prev", "_UsageDiff"])
}

let sdf = _;
let allValues = sdf.SelectDistinct([resourceColumn]).OrderByColumns([resourceColumn], [true]).Collect().Column(resourceColumn);
allValues.{
let v = _;
sdf = sdf.WithTotalUsageColumnOfSingleResource(resourceColumn, v)
}
sdf
}

let dt = ModelById(<model id>).EventsDataTable;
dt
.SqlDataFrame
.WithUsageColumns(<resource column name>)
.OrderByColumns([dt.ColumnMappings["TimeStamp"]], [true])
.Collect().ToCsv()
</syntaxhighlight>Where:

* <model id> is the id of the model containing event data to be examined.
* <resource column name> is the name of the column in the event data table of the specified model containing the resource being used by that event.

NOTE: This expression uses functionalities that are only supported in Snowflake-based data tables.

=== Create new Snowflake model from filter ===
This script creates a new Snowflake model (and two datatables for cases and events) containing filtered event log from given filter id. The script also works if the model doesn't have a cases datatable.

<syntaxhighlight lang="typescript" line="1">
let filter = FilterById(1); // filter id
let model = filter.model;
let project = model.project;
let nameSuffix = " - " + filter.name + " - " + ToString(Now, "dd-MM-yyyy HH:mm:ss");
let eventsDatatableName = model.EventsDataTable.Name + nameSuffix;
if (eventsDatatableName.length > 440) {
eventsDatatableName = eventsDatatableName.Substring(eventsDatatableName.length - 440);
}
let eventsData = model
.EventsDataTable
.SqlDataFrame
.ApplyFilter(
filter.rules,
model.CasesDataTable?.SqlDataFrame
);
project
.CreateDatatable(eventsDatatableName, #{"Connection": CreateSnowflakeConnection()})
.Import(eventsData);
let modelConfiguration = model.Configuration;
modelConfiguration.DataSource.Events.Set("DataTableName", eventsDatatableName);
if (model.CasesDataTable != null) {
let eventsDataCaseIdColumn = "CaseId_" + ToString(Random());
let casesDatatableName = model.CasesDataTable.Name + nameSuffix;
if (casesDatatableName.length > 440) {
casesDatatableName = casesDatatableName.Substring(casesDatatableName.length - 440);
}
let casesData = model
.CasesDataTable
.SqlDataFrame
.join(
eventsData.SelectDistinct([eventsDataCaseIdColumn: modelConfiguration.DataSource.Events.Columns.CaseId]),
[modelConfiguration.DataSource.Cases.Columns.CaseId: eventsDataCaseIdColumn]
).Select(model.CasesDataTable.ColumnNames);
project
.CreateDatatable(casesDatatableName, #{"Connection": CreateSnowflakeConnection()})
.Import(casesData);
modelConfiguration.DataSource.Cases.Set("DataTableName", casesDatatableName);
}
let modelName = model.Name + nameSuffix;
if (modelName > 440) {
modelName = modelName.Substring(modelName.length - 440);
}
project
.CreateModel(#{
"Name": modelName,
"Description": model.Description,
"Configuration": modelConfiguration
});
return modelName;
</syntaxhighlight>

=== Creating a model consisting of multiple copies of cases in an existing model ===
<syntaxhighlight lang="typescript" line="1">
/**
* @name CreateTestModel
* @description
* Creates a new model (or overwrites an existing) to given target project with given number of
* repetitions of given source model.
* Each repetition will generate "<N>-"-prefix to CaseId-columns, where N equals to the repeat index.
* @param sourceModel
* PA model used for the source data and from where the connection is copied for the target model if a
* new one has to be created.
* @param numRepeats
* Number of times the data in the source model should be repeated in the generated model.
* @param targetProject
* Project in which the target model resides.
* @param targetModelName
* Specifies the name of the test model in the given target project. If a model already exists with
* given name, event and case data in this model will be replaced with the new generated event and
* case data.
* @returns
* Model object of the test model having the newly generated data.
*/
function CreateTestModel(sourceModel, numRepeats, targetProject, targetModelName)
{
let eventsColumnMappings = sourceModel.EventsDataTable.ColumnMappings;
let casesColumnMappings = sourceModel.CasesDataTable.ColumnMappings;
let connection = sourceModel.EventsDataTable.DataSourceConnection;

function CreateResultModel()
{
function GetTable(tableName)
{
let tableConfiguration = #{
"Name": tableName,
"Connection": connection
};
let resultTable = targetProject.DataTableByName(tableName);
if (resultTable == null)
{
resultTable = targetProject.CreateDataTable(tableConfiguration)
.Modify(#{"NameInDataSource": null})
.Synchronize();
}
return resultTable;
}

let eventsTableName = `${targetModelName} - events`;
let casesTableName = `${targetModelName} - cases`;
let targetModel = targetProject.ModelByName(targetModelName);
let eventsTable, casesTable = null;

if (targetModel != null)
{
eventsTable = targetModel.EventsDataTable;
casesTable = targetModel.CasesDataTable;
}
else {
eventsTable = GetTable(eventsTableName);
if (sourceModel.CasesDataTable != null) {
casesTable = GetTable(casesTableName);
}

let timestampMapping = eventsColumnMappings["TimeStamp"];
eventsColumnMappings.Remove("TimeStamp");
eventsColumnMappings.Set("Timestamp", timestampMapping);

let modelConfiguration = #{
"DataSource": #{
"Events":#{
"DataSourceType": "datatable",
"DataTableName": eventsTableName,
"Columns": eventsColumnMappings
}
}
};

if (casesColumnMappings != null) {
modelConfiguration["DataSource"].Set("Cases", #{
"DataSourceType": "datatable",
"DataTableName": casesTableName,
"Columns": casesColumnMappings
});
}

targetModel = targetProject.CreateModel(#{"Name": targetModelName, "Configuration": modelConfiguration});
}

eventsTable.Truncate();
casesTable?.Truncate();

return #{
"TargetModel": targetModel,
"Events": eventsTable,
"Cases": casesTable
};
}

function RepeatNTimes(sourceDf, caseIdColumn, numRepeats)
{
let resultDf = null;
for (let i = 1; i <= numRepeats; ++i) {
let iterationDf = sourceDf
.WithColumn(caseIdColumn, #sql{Concat(#expr{i}, "-", Column(#expr{caseIdColumn}))});
resultDf = resultDf == null ? iterationDf : resultDf.Append(iterationDf);
}
resultDf;
}

let resultModel = CreateResultModel();
let sourceEventDataDf = sourceModel.EventsDataTable.SqlDataFrame;
let resultEventDataDf = RepeatNTimes(sourceEventDataDf, eventsColumnMappings["CaseId"], numRepeats);
resultModel["Events"].Import(resultEventDataDf);

let sourceCaseDataDf = sourceModel.CasesDataTable?.SqlDataFrame;
if (sourceCaseDataDf != null) {
let resultCaseDataDf = RepeatNTimes(sourceCaseDataDf, casesColumnMappings["CaseId"], numRepeats);
resultModel["Cases"].Import(resultCaseDataDf);
}
resultModel["TargetModel"];
}

</syntaxhighlight>Example usage:<blockquote>CreateTestModel(ProjectByName("Project").ModelByName("SAP_OrderToCash - Snowflake"), 3, ProjectByName("TestData"), "TestModel");</blockquote>Creates a new model named "TestModel" (or overwrites old one) into project named "TestData" containing the data from model "SAP_OrderToCash - Snowflake" in project "Project" repeated three times.

=== Analyzing declare patterns found in event log ===

This is an example expression that shows how POSIX-style regular expressions can be used to search for cases in an event log having certain event type patterns [https://www.researchgate.net/publication/277631859_Generating_Event_Logs_Through_the_Simulation_of_Declare_Models declare patterns].

Note: Before use, replace <model id> with a valid model identifier having event data.

<syntaxhighlight lang="typescript" line="1">
let dt = ModelById(<model id>).EventsDataTable;
let sdf = dt.SqlDataFrame.Head(1000);
let caseIdColumn = dt.ColumnMappings["CaseId"], timeStampColumn = dt.ColumnMappings["TimeStamp"], eventTypeColumn = dt.ColumnMappings["EventType"];

let eventTypesDf = sdf
.SelectDistinct([eventTypeColumn])
.OrderByColumns([eventTypeColumn], [true])
.WithRowNumberColumn("Token", [eventTypeColumn])
.WithColumn("Token", #sql{Char(Column("Token") + Unicode("a") - 1)});

sdf = sdf
.Join(eventTypesDf.Select(["_EventType2": eventTypeColumn, "Token"]), [eventTypeColumn: "_EventType2"], "leftouter");

function RespondedExistencePattern(a, b)
{
// [^a]*((a.*b.*)|(b.*a.*))*[^a]*
ToSqlExpression(`Regexp(Column("Trace"), "[^${a}]*((${a}.*${b}.*)|(${b}.*${a}.*))*[^${a}]")`)
}

function ResponsePattern(a, b)
{
// [^a]*(a.*b)*[^a]*
ToSqlExpression(`Regexp(Column("Trace"), "[^${a}]*(${a}.*${b})*[^${a}]")`)
}

function AlternateResponsePattern(a, b)
{
// [^a]*(a[^a]*b[^a]*)*[^a]*
ToSqlExpression(`Regexp(Column("Trace"), "[^${a}]*(${a}[^${a}]*${b}[^${a}]*)*[^${a}]")`)
}

function ChainResponsePattern(a, b)
{
// [^a]*(ab[^a]*)*[^a]*
ToSqlExpression(`Regexp(Column("Trace"), "[^${a}]*(${a}${b}[^${a}]*)*[^${a}]")`)
}

function PrecedencePattern(a, b)
{
// [^b]*(a.*b)*[^b]*
ToSqlExpression(`Regexp(Column("Trace"), "[^${b}]*(${a}.*${b})*[^${b}]")`)
}

let tracesDf = sdf
.GroupBy([caseIdColumn])
.Aggregate(["Trace": "Token"], [#{"Function": "list", "Ordering": [timeStampColumn], "Separator": ""}])
.WithColumn("RespondedExistenceNL", #sql{#expr{RespondedExistencePattern("n", "l")}})
.WithColumn("RespondedExistenceCA", #sql{#expr{RespondedExistencePattern("c", "a")}})
.WithColumn("ResponsePatternNL", #sql{#expr{ResponsePattern("n", "l")}})
.WithColumn("ResponsePatternCA", #sql{#expr{ResponsePattern("c", "a")}})
.WithColumn("AlternateResponsePatternNL", #sql{#expr{AlternateResponsePattern("n", "l")}})
.WithColumn("AlternateResponsePatternCA", #sql{#expr{AlternateResponsePattern("c", "a")}})
.WithColumn("ChainResponsePatternNL", #sql{#expr{ChainResponsePattern("n", "l")}})
.WithColumn("ChainResponsePatternCA", #sql{#expr{ChainResponsePattern("c", "a")}})
.WithColumn("PrecedencePatternNL", #sql{#expr{PrecedencePattern("n", "l")}})
.WithColumn("PrecedencePatternCA", #sql{#expr{PrecedencePattern("c", "a")}})
;

[tracesDf.Collect().ToCsv(), eventTypesDf.Collect().ToCsv()]
</syntaxhighlight>

=== Perform a query and send results as E-mail in a HTML table ===
This example requires two scripts that are both located in the same project:

# Expression-type script to execute (can be, e.g., scheduled to run daily).
# Expression-type script containing the query JSON to use as basis for the e-mail. In this example, this script is named as "Send query as E-mail - query JSON". The contents of this script is just the JSON representation of a query that can be extracted, e.g., from any PA chart view.

Script #1 should contain the following code:<syntaxhighlight lang="typescript">
let replyToAddress = "noreply@test.com";
let recipientsArray = ["testuser@test.com"];
let queryConfiguration = ParseJson(Project.ScriptByName("Send query as E-mail - query JSON").Code)
.Set("IncludeCollect", true);
let resultDf = Query(queryConfiguration);
let mailBodyHtml = resultDf.`
<table>
<caption>Example query</caption>
<thead>
<tr>
${StringJoin("", _.Columns.`
<th>${_}</th>
`)}
</tr>
</thead>
<tbody>
${StringJoin("", _.Rows.`
<tr>
${StringJoin("", _.`
<td>${_}</td>
`)}
</tr>
`)}
</tbody>
</table>
`;

SendEmail(#{
"ReplyTo": [replyToAddress],
"To": recipientsArray,
"Subject": "Example query E-mail",
"IsBodyHtml": true,
"Body": mailBodyHtml
});

</syntaxhighlight>NOTE: QPR ProcessAnalyzer Server has to have SMTP server configured. Also, remember to update the values of replyToAddress and recipientsArray before using.

=== Converting a case-centric model to object-centric model ===
This function serves as an example on how a case-centric model could be converted into an object-centric model having just one object type: "Case".

Note: Before use, replace <model id> with a valid model identifier having event data.<syntaxhighlight lang="typescript">
function ConvertCCModelToOCModel(model, newModelName)
{
let connection = model.EventsDataTable.DataSourceConnection;
let caseIdColumn = model.EventsDataTable.ColumnMappings["CaseId"];
let eventToObjectTableName = `${newModelName} - event-to-object`;
let eventsTableName = `${newModelName} - events`;
let objectsTableName = `${newModelName} - objects`;

let eventsDf = model.EventsDataTable
.SqlDataFrame
.RenameColumns([
"OcelEventType": model.EventsDataTable.ColumnMappings["EventType"],
"OcelEventTime": model.EventsDataTable.ColumnMappings["TimeStamp"]])
.WithColumn("OcelEventId", #sql{Concat("evt-", Cast(RowNumber([Column("OcelEventTime")]), "String"))});

eventsDf
.RenameColumns([
"OcelEventToObjectSourceId": "OcelEventId",
"OcelEventToObjectTargetId": caseIdColumn])
.Select(["OcelEventToObjectSourceId", "OcelEventToObjectTargetId"])
.WithColumn("OcelEventToObjectQualifier", #sql{#expr{caseIdColumn} })
.Persist(eventToObjectTableName, #{"Connection": connection});

eventsDf
.RemoveColumns([caseIdColumn])
.Persist(eventsTableName, #{"Connection": connection});

let casesDt = model.CasesDataTable
.SqlDataFrame
.RenameColumns([
"OcelObjectId": model.CasesDataTable.ColumnMappings["CaseId"]
])
.WithColumn("OcelObjectType", #sql{"Case"})
.Persist(objectsTableName, #{"Connection": connection});

let newConfiguration = #{
"OcelDataSource": #{
"Events": eventsTableName,
"Objects": objectsTableName,
"EventToObject": eventToObjectTableName
}
};

model.Project
.CreateModel(#{
"Name": newModelName,
"Description": model.Description,
"Configuration": newConfiguration
});
}

let ccModel = ModelById(<model id>);
ConvertCCModelToOCModel(ccModel, `ocel - ${ccModel.Name}`);
</syntaxhighlight>

=== Show duplicate rows in datatable ===
This script returns all rows in a datatable appearing more than once (sorted by the most frequent occurrences first). Additionally, the number of occurrences is returned as the last column. Replace the datatable id with the correct one.

<syntaxhighlight lang="typescript">
let datatableId = 1;
let rowCountColumn = "Row count";
let columns = DatatableById(datatableId).Columns.Name;
DatatableById(datatableId)
.SqlDataFrame
.GroupBy(columns)
.Aggregate(
[rowCountColumn: columns[0]],
["Count"]
)
.Where(Column(rowCountColumn) > 1)
.OrderByColumns([rowCountColumn], [false])
.Collect()
.ToCSV();
</syntaxhighlight>

=== Create a new object-centric model based on already existing object-centric model containing only filtered objects and events ===
Script for creating a new OCPM-model based on another already existing model in PA that applies given filter to the original model and creates all the needed tables to store only the relevant rows for all the OCPM data tables.

NOTE: Replace <nowiki><source model id>, <target model name>, <target project id> as well as the value of ocelItems with the applicable values before use.</nowiki><syntaxhighlight lang="typescript">
let newModelName = "<target model name>";
let sourceModelId = <source model id>;
let targetProjectId = <target project id>;

// Replace the value of the ocelItems below with the configuration of object-centric filter to apply.
// Note: Use expression language syntax here, not JSON.
let ocelItems = [#{
"Include": true,
"ObjectAttributeValue": #{
"ObjectType": "Purchase Order",
"Attribute": "po_product",
"Values": [
"0Cows"
]
}
}];

let m = ModelById(sourceModelId);
if (!m.IsOcelModel)
throw `Model ${m.Name} is not an OCPM model`;
if (CountTop(m.CheckModelValidity()) > 0)
throw `Model ${m.Name} is not a valid OCPM model`;

let targetProject = ProjectById(targetProjectId);
if (CountTop(targetProject.Models.Where(Name == newModelName)) > 0)
throw `Model having name ${newModelName} already exists in project ${targetProject.Name}. Rename or delete the existing model before running this script.`;

let newConfiguration = #{
"OcelDataSource": #{
"EventTypes": #{},
"ObjectTypes": #{}
}
};

WriteLog("Creating common tables...");

function PersistDataFrame(dataFrame, configurationName, configurationDict, dataTableName)
{
WriteLog(`Creating filtered table: ${dataTableName}`);
configurationDict.Set(configurationName, dataTableName);
return dataFrame.Persist(dataTableName, #{"ProjectId": targetProjectId, "Append": false});
}

let objectsDf = m.CacheTableSqlDataFrame(#{
"CacheTableType": "Objects",
"Perspective": #{},
"OcelItems": ocelItems
}).SetPrimaryKey(m.OcelObjects.PrimaryKey);
let newObjectsDt = PersistDataFrame(objectsDf, "Objects", newConfiguration.OcelDataSource, `${newModelName} - objects`);

let objectToObjectDf = m.CacheTableSqlDataFrame(#{
"CacheTableType": "ObjectToObject",
"Perspective": #{},
"OcelItems": ocelItems
}).SetPrimaryKey(m.OcelObjectToObject.PrimaryKey)
.Select(["OcelObjectToObjectSourceId", "OcelObjectToObjectTargetId", "OcelObjectToObjectQualifier"]);
let newObjectToObjectDt = PersistDataFrame(objectToObjectDf, "ObjectToObject", newConfiguration.OcelDataSource, `${newModelName} - object-to-object`);

let eventToObjectDf = m.CacheTableSqlDataFrame(#{
"CacheTableType": "EventToObject",
"Perspective": #{},
"OcelItems": ocelItems
}).SetPrimaryKey(m.OcelEventToObject.PrimaryKey)
.Select(["OcelEventToObjectSourceId", "OcelEventToObjectTargetId", "OcelEventToObjectQualifier"]);
let newEventToObjectDt = PersistDataFrame(eventToObjectDf, "EventToObject", newConfiguration.OcelDataSource, `${newModelName} - event-to-object`);

let eventsDf = m.OcelEvents.SqlDataFrame
.Join(eventToObjectDf.SelectDistinct(["OcelEventToObjectSourceId"]), ["OcelEventId": "OcelEventToObjectSourceId"], "inner")
.RemoveColumns(["OcelEventToObjectSourceId"])
.SetPrimaryKey(m.OcelEvents.PrimaryKey);
let newEventsDt = PersistDataFrame(eventsDf, "Events", newConfiguration.OcelDataSource, `${newModelName} - events`);

WriteLog("Creating event type tables...");

m.Configuration.OcelDataSource.EventTypes.ToArray().{
let ar = _;
let eventTypeName = GetContext(ar);
let eventTypeDt = m.OcelEventType(eventTypeName);
let eventTypeDf = eventTypeDt.SqlDataFrame
.Join(newEventsDt.SqlDataFrame.Select(["OcelEventId"]), ["OcelEventTypeEventId": "OcelEventId"], "inner")
.RemoveColumns(["OcelEventId"])
.SetPrimaryKey(eventTypeDt.PrimaryKey);
PersistDataFrame(eventTypeDf, eventTypeName, newConfiguration.OcelDataSource.EventTypes, `${newModelName} - eventtype - ${eventTypeName}`);
};

WriteLog("Creating object type tables...");
let newUniqueObjectsDf = newObjectsDt
.SqlDataFrame
.SelectDistinct(["OcelObjectId"]);

m.Configuration.OcelDataSource.ObjectTypes.ToArray().{
let ar = _;
let objectTypeName = GetContext(ar);
let objectTypeDt = m.OcelObjectType(objectTypeName);
let objectTypeDf = objectTypeDt.SqlDataFrame
.Join(newUniqueObjectsDf, ["OcelObjectTypeObjectId": "OcelObjectId"], "inner")
.RemoveColumns(["OcelObjectId"])
.SetPrimaryKey(objectTypeDt.PrimaryKey);
PersistDataFrame(objectTypeDf, objectTypeName, newConfiguration.OcelDataSource.ObjectTypes, `${newModelName} - objecttype - ${objectTypeName}`);
};

WriteLog(`Creating model ${newModelName}...`);

let newModel = targetProject.CreateModel(#{"Name": newModelName, "Configuration": newConfiguration});

WriteLog(`Model (id=${newModel.Id}) created with configuration:\r\n${ToJson(newConfiguration)}`);
</syntaxhighlight>

Embed to Website

2025-11-11T12:58:02Z

MarHink: Added required change details on Cross-Origin-Opener-Policy header.

QPR ProcessAnalyzer can be embedded to another website, such as Microsoft Sharepoint. The embedding is based on the ''iframe'' html element. By default, QPR ProcessAnalyzer only allows to be embedded by a website in the same origin that is restricted by the Content-Security-Policy HTTL header. More information how to change the CSP setting in [[QPR_ProcessAnalyzer_Security_Hardening#HTTP_Response_Headers|Security Hardening]]. Note also the special behavior when using the [[#Using_SAML_authentication_with_embedding|SAML authentication with embedding]].

== Example embedding webpage ==
This is a simple example page containing an iframe element that embeds QPR ProcessAnalyzer.

<pre>
<!DOCTYPE html>
<html>
<body>
<iframe src="https://processanalyzer.company.com/qprpa/ui/#/dashboard?sys:dashboardIdentifier=/MyProject/MyDashboard" height="600" width="900"></iframe>
</body>
</html>
</pre>

== Change CSP frame-ancestors setting ==
By default, the Content-Security-Policy HTTL header '''frame-ancestors''' directive is set to '''self''', allowing the parent website where QPR ProcessAnalyzer is embedded to, to be located in the same origin. If the parent website is in another origin, the CSP needs to be changed as follows (allowing to embed QPR ProcessAnalyzer in the example.com website):
<pre>
frame-ancestors 'self' example.com;
</pre>

If the CSP prevents the embedded website from showing, there will be a descriptive error message in the browser console. It contains details which CSP directive didn't allow to open the page.

More information about the Content-Security-Policy HTTL header: https://developer.mozilla.org/en-US/docs/Web/HTTP/Guides/CSP.

== Using SAML authentication with embedding ==
When QPR ProcessAnalyzer is embedded into another website with the [[SAML_2.0_Federated_Authentication|SAML authentication]] enabled, the authentication process is handled in a separate browser window or tab. This is necessary because some identity providers (IdP) do not permit authentication within an embedded frame (iframe element). Consequently, when SAML authentication begins, the identity provider's page will open in a new browser window. While this window is active, the embedded (original) window displays a message informing the user that authentication is in progress in a separate window.

If the browser has popup blocking enabled, the new window cannot open. In such cases, a descriptive message prompts the user to disable popup blocking. Additionally, the user can manually open the authentication window by clicking the embedded frame, even if popups are blocked. Once the identity provider login is successful, the new window will close, and the original browser window will redirect to QPR ProcessAnalyzer. However, if the separate browser window is closed before completing the login, the original window will display the standard QPR ProcessAnalyzer login view.

In addition, for SAML authentication to work correctly when the identity provider is located on a '''different origin''' than the QPR ProcessAnalyzer server itself, the <code>Cross-Origin-Opener-Policy</code> HTTP header must be changed from its default value of <code>"same-origin"</code> to <code>"unsafe-none"</code>.
----'''Additional Information:'''

* '''Cross-Origin-Opener-Policy (COOP) Header:''' This is a response header that provides a way for a document to control whether a new document opened in a top-level context (like a new tab or window) shares the same browsing context group as its opener. Setting it to <code>unsafe-none</code> allows the cross-origin authentication flow to complete successfully in this embedded scenario. For more technical details, see https://developer.mozilla.org/en-US/docs/Web/HTTP/Reference/Headers/Cross-Origin-Opener-Policy

Create Predicted Eventlog

2025-10-21T06:59:26Z

MarHink: /* Predicting case attribute values */

This article has instructions how to install, configure and use eventlog predictions. The prediction creates a new model that contains the source model data and the predictions. It's able to predict case attributes for the generated new cases and event attributes for the predicted events. To distinguish the real (source data) and predicted events and cases, there are following attributes in the model:
* Event attribute '''Predicted''' denotes whether the event is from the source data (''false'') or whether it's predicted (''true'').
* Case attribute '''Generated''' denotes whether the case is in the source data (''false'') or whether the prediction generated it as a new case (''true'').

== Prerequisites for prediction ==
Following prerequisites need to be fulfilled to run the eventlog prediction:
* QPR ProcessAnalyzer 2024.8 or later in use
* Snowflake connection is configured
* Source models are stored to Snowflake

== Install prediction to Snowflake ==
To install the eventlog prediction to Snowflake:
# Go to Snowflake, and create a Snowflake-managed stage with name '''DECISION_INTELLIGENCE''' to the same schema configured to QPR ProcessAnalyzer (in the Snowflake connection string). Use settings in the following image: [[File:Create_Snowflake_stage.png]]
# Open the created stage and upload the '''predict.pyz''' file into the stage (ask the file from your QPR representative).
# Create the following procedure to the same schema:
<syntaxhighlight lang="sql">
CREATE OR REPLACE PROCEDURE QPRPA_SP_PREDICTION("CONFIGURATION" OBJECT)
RETURNS OBJECT
LANGUAGE PYTHON
STRICT
RUNTIME_VERSION = '3.11'
PACKAGES = ('nltk','numpy','networkx','pandas','scikit-learn','snowflake-snowpark-python','tensorflow==2.12.0','dill','psutil','prophet','holidays','python-kubernetes','docker-py','cryptography')
HANDLER = 'main'
EXECUTE AS OWNER
AS '
import sys
def main(session, parameters_in: dict) -> dict:
session.file.get(''@decision_intelligence/predict.pyz'', ''/tmp'')
sys.path.append(''/tmp/predict.pyz'')
import predict
return predict.main(session, parameters_in)
';
</syntaxhighlight>

== Create prediction script in QPR ProcessAnalyzer ==
1. Create the following example expression script (e.g., with name '''Create prediction model'''):<syntaxhighlight lang="typescript">
let sourceModel = ProjectByName("<project name>").ModelByName("<model name>");
let completeCaseEventTypeName = "<event type name found only in complete cases>";

let targetProject = Project;
let eventTypeColumnName = sourceModel.EventsDataTable.ColumnMappings["EventType"];

_system.ML.GeneratePredictionModel(#{
"Name": "My prediction model", // Name of the PA model to generate ti the target project.
"SourceModel": sourceModel, // Snowflake-based PA model used for training the prediction model.
"TargetProject": targetProject, // Target project to create the model into.
"TrainingConfiguration": #{ // Training parameters.
"num_epochs_to_train": 200
},
"GenerationConfiguration": #{ // Model generation parameters.
"cases_to_generate": 1000
},
"TrainingDataFilter": #{
"Items": [
#{
"Type": "IncludeCases",
"Items": [
#{
"Type": "EventAttributeValue",
"Attribute": eventTypeColumnName,
"StringifiedValues": [
`0${completeCaseEventTypeName}`
]
}
]
}
]
},
"IncompleteCasesFilter": #{
"Items": [
#{
"Type": "ExcludeCases",
"Items": [
#{
"Type": "EventAttributeValue",
"Attribute": eventTypeColumnName,
"StringifiedValues": [
`0${completeCaseEventTypeName}`
]
}
]
}
]
},
"RecreatePredictionModel": true, // Should a prediction model be overwritten if one already exists for this source model and target model name combination.
"TrainingCaseSampleSize": 10000 // Maximum number of cases to use from the source model (random sampled).
});
</syntaxhighlight>2. Configure prediction for the previously created script as instructed in the next chapter. At minimum, replace the tags listed below with some suitable values:
* '''<project name>''': Name of the project in which the source model is located.
* '''<model name>''': Name of the model to be used as source model. This data in this source model will be used to train the prediction model so that it can generate new cases and continuations for incomplete existing cases.
* '''<event type name found only in complete cases>''': This example script has been hard-coded to determine whether a case is complete or incomplete based on the existence of this event type.

== Configure prediction ==
Prediction script has the following settings in the GeneratePredictionModel call:
* '''Name''': Name of the QPR ProcessAnalyzer model that is created to the target project. The model will contain the source model content and the predictions.
* '''SourceModel''': Source model for which the prediction is made. Model can be selected for example based on id with ModelById function or by name with ModelByName function.
* '''TargetProject''': Target project to create the new model into.
* '''RecreatePredictionModel''': When ''true'', a new ML model is trained when the script is run. When ''false'', the prediction is run using possibly pre-existing ML model.
* '''TrainingConfiguration''': Training parameters.
** '''attributes''': Attribute configurations (for more information, see the chapter below).
** '''generate_start_time_trend_images''': If set to true, two images will be generated for each cross validated Prophet-parameter combination and also for the final selected parameters showing the results of plot and plot_components-functions.
*** The images will be generated into stage files with the following path names:
**** plot: @decision_intelligence_testing/{model_name}_st_RMSE={rmse_value or "final"}.png
****plot_components: @decision_intelligence_testing/{model_name}_st_RMSE={rmse_value or "final"}_comp.png
***The default value is false.
** '''max_num_case_clusters''': Set the maximum number of clusters to divide the case attribute values into.
*** The default value is 20.
** '''max_num_traces_in_training''': Set the maximum number of traces used in training.
*** When training, every case of length N will be split into N traces (a.k.a. prefixes) (p_1, ..., p_N), where p_x contains first x events of the all events of the full case.
**** If there are more traces available than this configured value, cases to include will be random sampled so that the maximum is exceeded by at most one case.
**** If null, all the traces will be used, no matter what (may easily lead to running out of memory).
**** The default value is 100000.
** '''num_epochs_to_train''': How many times the training set is used in training. The best performing model out of all the iterations will be selected.
*** The default value is 500.
** '''num_extra_years_to_reserve_in_created_model''': Number of additional years after the year of the last timestamp in the training data to reserve to the capacity of the created ML model, allowing the model to be able to predict timestamps in the range between the minimum timestamp year in the training data and the maximum timestamp year plus this value.
*** The default value is 20.
** '''reserve_extra_sequence_length''': How many extra events to reserve space for in the ML model compared to the number of events the longest case in the training data has.
*** The default value is 5.
** '''samples_per_epoch''': If not null, specifies (approximately) how many traces/prefixes will be used to represent one epoch of data in the training. The actual value used will be made divisible by batch_size using this formula:
***max(floor(samples_per_epoch / batch_size), 1) * batch_size
***If null, every epoch will use all the traces/prefixes in the training data.
***The default value is null
**'''validation_split''': Percentage of traces/prefixes to use to evaluate the loss and any model metrics at the end of each epoch. The model will not be trained on this data.
***If 0, separate validation data will not be used. Instead, all the training data will be used also as validation data.
***The default value is 0.
* '''GenerationConfiguration''': Event generation parameters. When null, no generation is done. For example, following parameters are supported:
** '''avoid_repeated_activities''': Array of activity names that should occur at most once in any case. The probability of selecting any of the activities specified in this configuration more than once is set to be 0.
*** Empty array means that activity generation is not restricted by this setting at all.
*** null value means that there should not be any activities that can occur more than once (shortcut for specifying all the activity names).
*** The default value is an empty array.
** '''cases_to_generate''': Maximum number cases to create. The number of created cases is further limited by the capabilities of the trained model and the ''case_generation_start_time'' and ''case_generation_end_time'' parameters.
*** The default value is such that the number of cases, by itself, is not limited.
** '''case_generation_start_time''': If defined, new cases will be generated after this timestamp (given as string in ISO datetime format).
*** If undefined, the latest start event timestamp used in the training data is used.
*** The default value is undefined.
** '''case_generation_end_time''': If defined, new events and cases will not be generated after this timestamp (given as string in ISO datetime format). E.g., "2015-01-01T00:00:00".
*** The default value is unlimited (only limit comes from the capacity of the trained model)
** '''generate_debug_event_attributes''':
*** If true, additional columns will be added containing, e.g., probabilities of the selected activity and other activities.
*** The default value is false.
** '''max_num_events''':
*** Specifies the maximum number of events to generate for any case.
*** If unspecified (=default), the value equals to ''<the maximum number of events in any case in the training data>''+''<the value of reserve_extra_sequence_length in training>''.
** '''min_prediction_probability ''':
*** The minimum probability of any prediction. If the probability of a prediction is lower than this, it will never be picked.
*** The default value is 0.01.
** '''temperature''':
*** If 0, the generated next activity will always be the one that is the most probable.
*** If 1, the generated next activity is purely based on the probabilities returned by the trained ML model.
*** This behavior is interpolated when using values between 0 and 1.
*** The default value is 0.9.
* '''TrainingDataFilter''': [[Filtering_in_QPR_ProcessAnalyzer_Queries|Filter]] to select specific cases that are used to train the prediction model. This filter is required to train the model only using the completed cases. Uncompleted cases should not be used for the training, so the model doesn't incorrectly learn that cases should end like that.
* '''IncompleteCasesFilter''': Optional [[Filtering_in_QPR_ProcessAnalyzer_Queries|filter]] to select which cases the prediction is made for. To improve performance of the prediction, it's recommended to include only the incomplete cases for which new events might appear, and skip the completed cases for which new events are not expected anymore.
* '''TrainingCaseSampleSize''': Maximum number of cases to take from the source model (cases are selected randomly). Use a lower setting to speed up the ML model training. The greater the value, the more subtle phenomena the prediction can learn from the data.

== Attribute configuration ==
Attribute configuration is used in '''TrainingConfiguration''' (see the chapter above) to configure which event- and case attributes should be used in prediction model and how they are used.

The configuration is in the top level split into two sections: "event" and "case". "Event" is used to configure event attributes, whereas "case" is used for case attributes.

The next level supports one value: "input".

The next level after that, supports the following settings:

* '''categorical_groups''': An array of categorical attribute group configuration objects used to define groups of attributes that will be bundled together in the trained model, either as separate input- or output features. Each attribute group will form its own input- or output vector used in the model training and generation.
** If null, only one group will be created with all the available categorical attributes included.
** The following settings are supported by these objects:
*** '''attributes''': An array of attribute names.
**** If null, all the input attributes are to be included in this group.
*** '''max_num_clusters''': The maximum number of clusters (input- or output vector feature values) to use to represent this group of attributes.
**** Default value: 20
**** NOTE: Clustering is used by default to convert a set of attribute values into an input- or output vector used by the prediction model.
* '''columns''': An array of attribute column configuration objects used to define columns in the input data that are to be used as event- or case attributes.
** If null, all the columns will be included as categorical attributes (except case id, event type (only for event) and timestamp (only for event) columns).
** The following settings are supported by these objects:
*** '''label''': Column name.
*** '''type''': Type of the column. Supported types are:
**** '''categorical''': Values can take on one of a limited, and usually fixed, number of possible values.
**** '''numeric''': Value is considered as a continuous numeric value.

==== Example ====
Use all event attributes as input for the prediction model. In addition, additional machine learning input vector for SAP_User-event data column supporting at most 10 unique values.

In addition, for case attributes, only "Region", "Product Group", "Account Manager" and "Customer Group" case data columns are used as categorical attributes and "Cost" as numeric attribute. Furthermore, the four categorical case attributes are grouped into three groups, each of which are used as its own input vector for the prediction model.

When generating, all event attributes will be included for generated events as columns. Generated cases will have only "Region", "Product Group", "Account Manager", "Customer Group", and "Cost" columns.<syntaxhighlight lang="json">
"attributes": #{
"event": #{
"input": #{
"categorical_groups": [
#{
"attributes": None
},
#{
"attributes": ["SAP_User"],
"max_num_clusters": 10
}
],
"columns": None
}
},
"case": #{
"input": #{
"categorical_groups": [#{
"attributes": ["Account Manager"]
}, #{
"attributes": ["Customer Group"]
}, #{
"attributes": ["Region", "Product Group"]
}],
"columns": [
#{ "label": "Region", "type": "categorical" },
#{ "label": "Product Group", "type": "categorical" },
#{ "label": "Account Manager", "type": "categorical" },
#{ "label": "Customer Group", "type": "categorical" },
#{ "label": "Cost", "type": "numeric" }
]
}
}
}
</syntaxhighlight>

== Predicting case attribute values ==
QPR ProcessAnalyzer can also be used to, e.g., predict the final values of case attributes of running cases. The following script gives an example on how to perform this.<syntaxhighlight lang="typescript">
let sourceModel = ProjectByName("<project name>").ModelByName("<model name>");
let caseAttributeToPredict = "<name of the case attribute>";
let resultModelName = "<name of the model to be created/replaced>";
let generateDebugCaseAttributes = false; // Set to true to generate columns for prediction probabilities.
let casesToPredictFilter = "<JSON filter for cases for which the prediction is to be performed>";
let casesToUseForTrainingFilter = "<JSON filter for cases to be used for ML model training>";

let targetProject = Project;

_system.ML.GenerateCaseAttributePredictionModel(#{
"Name": resultModelName, // Name of the PA model to generate ti the target project.
"SourceModel": sourceModel, // Snowflake-based PA model used for training the prediction model.
"TargetProject": targetProject, // Target project to create the model into.
"RecreatePredictionModel": false, // Should a prediction model be overwritten if one already exists for this source model and target model name combination.
"TrainingCaseSampleSize": 10000, // Maximum number of cases to use from the source model (random sampled).
"CommonConfiguration": #{ // Common parameters used by both training and generation.
"output_case_attribute_groups": [#{
"attributes": [caseAttributeToPredict] // Attribute whose value is to be predicted.
}]
},
"TrainingConfiguration": #{ // Training parameters.
"max_num_case_attribute_clusters": 80,
"num_epochs_to_train": 100
},
"GenerationConfiguration": #{ // Case attribute generation parameters.
"generate_debug_case_attributes": generateDebugCaseAttributes // Should probability and probability_all-columns be generated as well as the actual prediction created into a new column named Predicted_<attribute name>
},
"TrainingDataFilter": ParseJson(casesToUseForTrainingFilter), // Filter JSON for events to be used for training.
"IncompleteCasesFilter": ParseJson(casesToPredictFilter) // Filter JSON for events for whose case attribute value is to be predicted.
});

</syntaxhighlight>

Create Predicted Eventlog

2025-10-21T06:58:09Z

MarHink: Added example of predicting case attribute values

This article has instructions how to install, configure and use eventlog predictions. The prediction creates a new model that contains the source model data and the predictions. It's able to predict case attributes for the generated new cases and event attributes for the predicted events. To distinguish the real (source data) and predicted events and cases, there are following attributes in the model:
* Event attribute '''Predicted''' denotes whether the event is from the source data (''false'') or whether it's predicted (''true'').
* Case attribute '''Generated''' denotes whether the case is in the source data (''false'') or whether the prediction generated it as a new case (''true'').

== Prerequisites for prediction ==
Following prerequisites need to be fulfilled to run the eventlog prediction:
* QPR ProcessAnalyzer 2024.8 or later in use
* Snowflake connection is configured
* Source models are stored to Snowflake

== Install prediction to Snowflake ==
To install the eventlog prediction to Snowflake:
# Go to Snowflake, and create a Snowflake-managed stage with name '''DECISION_INTELLIGENCE''' to the same schema configured to QPR ProcessAnalyzer (in the Snowflake connection string). Use settings in the following image: [[File:Create_Snowflake_stage.png]]
# Open the created stage and upload the '''predict.pyz''' file into the stage (ask the file from your QPR representative).
# Create the following procedure to the same schema:
<syntaxhighlight lang="sql">
CREATE OR REPLACE PROCEDURE QPRPA_SP_PREDICTION("CONFIGURATION" OBJECT)
RETURNS OBJECT
LANGUAGE PYTHON
STRICT
RUNTIME_VERSION = '3.11'
PACKAGES = ('nltk','numpy','networkx','pandas','scikit-learn','snowflake-snowpark-python','tensorflow==2.12.0','dill','psutil','prophet','holidays','python-kubernetes','docker-py','cryptography')
HANDLER = 'main'
EXECUTE AS OWNER
AS '
import sys
def main(session, parameters_in: dict) -> dict:
session.file.get(''@decision_intelligence/predict.pyz'', ''/tmp'')
sys.path.append(''/tmp/predict.pyz'')
import predict
return predict.main(session, parameters_in)
';
</syntaxhighlight>

== Create prediction script in QPR ProcessAnalyzer ==
1. Create the following example expression script (e.g., with name '''Create prediction model'''):<syntaxhighlight lang="typescript">
let sourceModel = ProjectByName("<project name>").ModelByName("<model name>");
let completeCaseEventTypeName = "<event type name found only in complete cases>";

let targetProject = Project;
let eventTypeColumnName = sourceModel.EventsDataTable.ColumnMappings["EventType"];

_system.ML.GeneratePredictionModel(#{
"Name": "My prediction model", // Name of the PA model to generate ti the target project.
"SourceModel": sourceModel, // Snowflake-based PA model used for training the prediction model.
"TargetProject": targetProject, // Target project to create the model into.
"TrainingConfiguration": #{ // Training parameters.
"num_epochs_to_train": 200
},
"GenerationConfiguration": #{ // Model generation parameters.
"cases_to_generate": 1000
},
"TrainingDataFilter": #{
"Items": [
#{
"Type": "IncludeCases",
"Items": [
#{
"Type": "EventAttributeValue",
"Attribute": eventTypeColumnName,
"StringifiedValues": [
`0${completeCaseEventTypeName}`
]
}
]
}
]
},
"IncompleteCasesFilter": #{
"Items": [
#{
"Type": "ExcludeCases",
"Items": [
#{
"Type": "EventAttributeValue",
"Attribute": eventTypeColumnName,
"StringifiedValues": [
`0${completeCaseEventTypeName}`
]
}
]
}
]
},
"RecreatePredictionModel": true, // Should a prediction model be overwritten if one already exists for this source model and target model name combination.
"TrainingCaseSampleSize": 10000 // Maximum number of cases to use from the source model (random sampled).
});
</syntaxhighlight>2. Configure prediction for the previously created script as instructed in the next chapter. At minimum, replace the tags listed below with some suitable values:
* '''<project name>''': Name of the project in which the source model is located.
* '''<model name>''': Name of the model to be used as source model. This data in this source model will be used to train the prediction model so that it can generate new cases and continuations for incomplete existing cases.
* '''<event type name found only in complete cases>''': This example script has been hard-coded to determine whether a case is complete or incomplete based on the existence of this event type.

== Configure prediction ==
Prediction script has the following settings in the GeneratePredictionModel call:
* '''Name''': Name of the QPR ProcessAnalyzer model that is created to the target project. The model will contain the source model content and the predictions.
* '''SourceModel''': Source model for which the prediction is made. Model can be selected for example based on id with ModelById function or by name with ModelByName function.
* '''TargetProject''': Target project to create the new model into.
* '''RecreatePredictionModel''': When ''true'', a new ML model is trained when the script is run. When ''false'', the prediction is run using possibly pre-existing ML model.
* '''TrainingConfiguration''': Training parameters.
** '''attributes''': Attribute configurations (for more information, see the chapter below).
** '''generate_start_time_trend_images''': If set to true, two images will be generated for each cross validated Prophet-parameter combination and also for the final selected parameters showing the results of plot and plot_components-functions.
*** The images will be generated into stage files with the following path names:
**** plot: @decision_intelligence_testing/{model_name}_st_RMSE={rmse_value or "final"}.png
****plot_components: @decision_intelligence_testing/{model_name}_st_RMSE={rmse_value or "final"}_comp.png
***The default value is false.
** '''max_num_case_clusters''': Set the maximum number of clusters to divide the case attribute values into.
*** The default value is 20.
** '''max_num_traces_in_training''': Set the maximum number of traces used in training.
*** When training, every case of length N will be split into N traces (a.k.a. prefixes) (p_1, ..., p_N), where p_x contains first x events of the all events of the full case.
**** If there are more traces available than this configured value, cases to include will be random sampled so that the maximum is exceeded by at most one case.
**** If null, all the traces will be used, no matter what (may easily lead to running out of memory).
**** The default value is 100000.
** '''num_epochs_to_train''': How many times the training set is used in training. The best performing model out of all the iterations will be selected.
*** The default value is 500.
** '''num_extra_years_to_reserve_in_created_model''': Number of additional years after the year of the last timestamp in the training data to reserve to the capacity of the created ML model, allowing the model to be able to predict timestamps in the range between the minimum timestamp year in the training data and the maximum timestamp year plus this value.
*** The default value is 20.
** '''reserve_extra_sequence_length''': How many extra events to reserve space for in the ML model compared to the number of events the longest case in the training data has.
*** The default value is 5.
** '''samples_per_epoch''': If not null, specifies (approximately) how many traces/prefixes will be used to represent one epoch of data in the training. The actual value used will be made divisible by batch_size using this formula:
***max(floor(samples_per_epoch / batch_size), 1) * batch_size
***If null, every epoch will use all the traces/prefixes in the training data.
***The default value is null
**'''validation_split''': Percentage of traces/prefixes to use to evaluate the loss and any model metrics at the end of each epoch. The model will not be trained on this data.
***If 0, separate validation data will not be used. Instead, all the training data will be used also as validation data.
***The default value is 0.
* '''GenerationConfiguration''': Event generation parameters. When null, no generation is done. For example, following parameters are supported:
** '''avoid_repeated_activities''': Array of activity names that should occur at most once in any case. The probability of selecting any of the activities specified in this configuration more than once is set to be 0.
*** Empty array means that activity generation is not restricted by this setting at all.
*** null value means that there should not be any activities that can occur more than once (shortcut for specifying all the activity names).
*** The default value is an empty array.
** '''cases_to_generate''': Maximum number cases to create. The number of created cases is further limited by the capabilities of the trained model and the ''case_generation_start_time'' and ''case_generation_end_time'' parameters.
*** The default value is such that the number of cases, by itself, is not limited.
** '''case_generation_start_time''': If defined, new cases will be generated after this timestamp (given as string in ISO datetime format).
*** If undefined, the latest start event timestamp used in the training data is used.
*** The default value is undefined.
** '''case_generation_end_time''': If defined, new events and cases will not be generated after this timestamp (given as string in ISO datetime format). E.g., "2015-01-01T00:00:00".
*** The default value is unlimited (only limit comes from the capacity of the trained model)
** '''generate_debug_event_attributes''':
*** If true, additional columns will be added containing, e.g., probabilities of the selected activity and other activities.
*** The default value is false.
** '''max_num_events''':
*** Specifies the maximum number of events to generate for any case.
*** If unspecified (=default), the value equals to ''<the maximum number of events in any case in the training data>''+''<the value of reserve_extra_sequence_length in training>''.
** '''min_prediction_probability ''':
*** The minimum probability of any prediction. If the probability of a prediction is lower than this, it will never be picked.
*** The default value is 0.01.
** '''temperature''':
*** If 0, the generated next activity will always be the one that is the most probable.
*** If 1, the generated next activity is purely based on the probabilities returned by the trained ML model.
*** This behavior is interpolated when using values between 0 and 1.
*** The default value is 0.9.
* '''TrainingDataFilter''': [[Filtering_in_QPR_ProcessAnalyzer_Queries|Filter]] to select specific cases that are used to train the prediction model. This filter is required to train the model only using the completed cases. Uncompleted cases should not be used for the training, so the model doesn't incorrectly learn that cases should end like that.
* '''IncompleteCasesFilter''': Optional [[Filtering_in_QPR_ProcessAnalyzer_Queries|filter]] to select which cases the prediction is made for. To improve performance of the prediction, it's recommended to include only the incomplete cases for which new events might appear, and skip the completed cases for which new events are not expected anymore.
* '''TrainingCaseSampleSize''': Maximum number of cases to take from the source model (cases are selected randomly). Use a lower setting to speed up the ML model training. The greater the value, the more subtle phenomena the prediction can learn from the data.

== Attribute configuration ==
Attribute configuration is used in '''TrainingConfiguration''' (see the chapter above) to configure which event- and case attributes should be used in prediction model and how they are used.

The configuration is in the top level split into two sections: "event" and "case". "Event" is used to configure event attributes, whereas "case" is used for case attributes.

The next level supports one value: "input".

The next level after that, supports the following settings:

* '''categorical_groups''': An array of categorical attribute group configuration objects used to define groups of attributes that will be bundled together in the trained model, either as separate input- or output features. Each attribute group will form its own input- or output vector used in the model training and generation.
** If null, only one group will be created with all the available categorical attributes included.
** The following settings are supported by these objects:
*** '''attributes''': An array of attribute names.
**** If null, all the input attributes are to be included in this group.
*** '''max_num_clusters''': The maximum number of clusters (input- or output vector feature values) to use to represent this group of attributes.
**** Default value: 20
**** NOTE: Clustering is used by default to convert a set of attribute values into an input- or output vector used by the prediction model.
* '''columns''': An array of attribute column configuration objects used to define columns in the input data that are to be used as event- or case attributes.
** If null, all the columns will be included as categorical attributes (except case id, event type (only for event) and timestamp (only for event) columns).
** The following settings are supported by these objects:
*** '''label''': Column name.
*** '''type''': Type of the column. Supported types are:
**** '''categorical''': Values can take on one of a limited, and usually fixed, number of possible values.
**** '''numeric''': Value is considered as a continuous numeric value.

==== Example ====
Use all event attributes as input for the prediction model. In addition, additional machine learning input vector for SAP_User-event data column supporting at most 10 unique values.

In addition, for case attributes, only "Region", "Product Group", "Account Manager" and "Customer Group" case data columns are used as categorical attributes and "Cost" as numeric attribute. Furthermore, the four categorical case attributes are grouped into three groups, each of which are used as its own input vector for the prediction model.

When generating, all event attributes will be included for generated events as columns. Generated cases will have only "Region", "Product Group", "Account Manager", "Customer Group", and "Cost" columns.<syntaxhighlight lang="json">
"attributes": #{
"event": #{
"input": #{
"categorical_groups": [
#{
"attributes": None
},
#{
"attributes": ["SAP_User"],
"max_num_clusters": 10
}
],
"columns": None
}
},
"case": #{
"input": #{
"categorical_groups": [#{
"attributes": ["Account Manager"]
}, #{
"attributes": ["Customer Group"]
}, #{
"attributes": ["Region", "Product Group"]
}],
"columns": [
#{ "label": "Region", "type": "categorical" },
#{ "label": "Product Group", "type": "categorical" },
#{ "label": "Account Manager", "type": "categorical" },
#{ "label": "Customer Group", "type": "categorical" },
#{ "label": "Cost", "type": "numeric" }
]
}
}
}
</syntaxhighlight>

== Predicting case attribute values ==
QPR ProcessAnalyzer can also be used to, e.g., predict the final values of case attributes of running cases. The following script gives an example on how to perform this.<syntaxhighlight lang="typescript">
let sourceModel = ProjectByName("<project name>").ModelByName("<model name>");
let caseAttributeToPredict = "<name of the case attribute>";
let resultModelName = "<name of the model to be created/replaced>";
let generateDebugCaseAttributes = false;
let casesToPredictFilter = "<JSON filter for cases for which the prediction is to be performed>";
let casesToUseForTrainingFilter = "<JSON filter for cases to be used for ML model training>";

let targetProject = Project;

_system.ML.GenerateCaseAttributePredictionModel(#{
"Name": resultModelName, // Name of the PA model to generate ti the target project.
"SourceModel": sourceModel, // Snowflake-based PA model used for training the prediction model.
"TargetProject": targetProject, // Target project to create the model into.
"RecreatePredictionModel": false, // Should a prediction model be overwritten if one already exists for this source model and target model name combination.
"TrainingCaseSampleSize": 10000, // Maximum number of cases to use from the source model (random sampled).
"CommonConfiguration": #{ // Common parameters used by both training and generation.
"output_case_attribute_groups": [#{
"attributes": [caseAttributeToPredict] // Attribute whose value is to be predicted.
}]
},
"TrainingConfiguration": #{ // Training parameters.
"max_num_case_attribute_clusters": 80,
"num_epochs_to_train": 100
},
"GenerationConfiguration": #{ // Case attribute generation parameters.
"generate_debug_case_attributes": generateDebugCaseAttributes // Should probability and probability_all-columns be generated as well as the actual prediction created into a new column named Predicted_<attribute name>
},
"TrainingDataFilter": ParseJson(casesToUseForTrainingFilter), // Filter JSON for events to be used for training.
"IncompleteCasesFilter": ParseJson(casesToPredictFilter) // Filter JSON for events for whose case attribute value is to be predicted.
});

</syntaxhighlight>

SAML 2.0 Federated Authentication

2025-10-03T07:28:15Z

MarHink: Updated SAML related URLs to have the correct casings.

QPR ProcessAnalyzer supports authenticating users with federated authentication using the SAML 2.0 protocol. QPR ProcessAnalyzer works as a '''service provider (SP)''' and uses an external '''identity providers (IdP)''' to provide user identity (i.e. authenticating users). Commonly used identity providers are Azure AD and Microsoft Active Directory Federation Services (ADFS).

== Introduction ==
When QPR ProcessAnalyzer is configured as a SAML 2.0 service provider (SP), users can authenticate to QPR ProcessAnalyzer via the configured SAML 2.0 identity provider (IdP). When accessing QPR ProcessAnalyzer, users are automatically redirected to the identity provider for authentication. When the authentication is done, users are redirected back to QPR ProcessAnalyzer where user is then automatically logged in. When using federated authentication, users don't normally see the QPR ProcessAnalyzer login page. The login page can be accessed (e.g. when login using QPR ProcessAnalyzer user management credentials) by adding '''forceLogin=1''' parameter to the url, e.g. <nowiki>https://customer.onqpr.com/QPRPA/ui/#/login?forceLogin=1</nowiki>.

QPR ProcessAnalyzer can also automatically redirect users to the identity provider from the url '''/qprpa/Saml2''', e.g. <nowiki>https://customer.onqpr.com/qprpa/Saml2</nowiki>. Redirection to this url can be configured to IIS, when users access QPR ProcessAnalyzer with the server name only, e.g. <nowiki>https://customer.onqpr.com</nowiki>. The advantage of using this url is that the QPR ProcessAnalyzer web application is not loaded before the authentication, making the authentication flow faster. When going to the identity provider using a url starting with <nowiki>https://customer.onqpr.com/QPRPA/ui/</nowiki>, the QPR ProcessAnalyzer web application is loaded before going to the identity provider.

When a user logs in to QPR ProcessAnalyzer for the first time, user account is created to QPR ProcessAnalyzer user management. This account can only log in using the federated authentication, because the user account doesn't have a password in QPR ProcessAnalyzer. User accounts are matched between QPR ProcessAnalyzer and the identity provider using usernames.

When the ''SAMLGroupsAttribute'' setting has been configured, QPR ProcessAnalyzer user management is kept in synchronization with the identity provider information regarding which groups the users are belonging to. This way, the operative user management will be done outside QPR ProcessAnalyzer. Still, the groups to be used need to be created beforehand and permissions assigned to the groups.

Additional notes for the SAML authentication:
* QPR ProcessAnalyzer needs to use https when SAML 2.0 authentication is used.
* The identity provider needs to publish the identity provider metadata, because QPR ProcessAnalyzer reads the identity provider settings from there.
* QPR ProcessAnalyzer only supports SAML POST binding (e.g., SAML redirect binding is not supported).
*''SAML AuthnRequests'' are signed using an out-of-the-box certificate embedded to QPR ProcessAnalyzer. This certificate has validity until 1.1.2035. It's also possible to use a custom certificate (see the SAMLSigningCertificate setting). The certificate public key is available in the service provider metadata published by the [[Web API: saml2|QPR ProcessAnalyzer Web API]].
*''SAML Assertions'' must be signed (by the identity provider) to be accepted by QPR ProcessAnalyzer.
*SAML Assertions can optionally be encrypted by the identity provider. This requires the SAMLEncryptionCertificate setting to be defined in the [[PA_Configuration_database_table#SAML_2.0_Federated_Authentication_Settings|QPR ProcessAnalyzer configuration table]].
* Logout request to identity provider is not supported by QPR ProcessAnalyzer.
* If user clicks the logout button, user is redirected to the QPR ProcessAnalyzer login page. There user can click the '''Log in using SSO''' button to relogin.
* If the QPR ProcessAnalyzer session expires, user is redirected back to the identity provider for relogin.

==Configuring SAML to QPR ProcessAnalyzer==

To configure the SAML 2.0 authentication, follow these steps:
# Define settings '''SAMLMetadataUrl''', '''ServiceProviderLocation''', '''SAMLUserIdAttribute''', '''SAMLGroupsAttribute''' (optional), '''SAMLEncryptionCertificate''' (optional), and '''SAMLSigningCertificate''' (optional) in the [[PA_Configuration_database_table#SAML_2.0_Federated_Authentication_Settings|QPR ProcessAnalyzer configuration table]]. QPR ProcessAnalyzer needs to be restarted for the settings to take effect.
# Configure a redirection from the root path of the QPR ProcessAnalyzer server to '''/qprpa/Saml2''', so that users are automatically redirected to the identity provider for authentication.
# The identity provider configuration depends on which identity provide is used. See below for help how to configure [[#Using Azure AD as Identity Provider|Azure AD]] and [[#Using ADFS as Identity Provider|ADFS]] as the identity provider.

If there are any issues with the authentication, please check the [[QPR_ProcessAnalyzer_Logs|QPR ProcessAnalyzer logs]].

==Using Azure AD as Identity Provider==
Azure Active Directory (AAD) can be used as an identity provider to login to QPR ProcessAnalyzer. Following configurations are needed:
# Login to https://portal.azure.com as a cloud application admin or an application admin for your Azure AD tenant.
# Click '''Azure Active Directory''' > '''Enterprise Applications''' > '''New application'''. Select '''Non-gallery application'''.
# Define '''Name''' for the application, e.g., "QPR ProcessAnalyzer".
# Go to '''Manage''' > '''Single sign-on''' > '''SAML'''.
# Click '''Edit''' pencil on the '''Basic SAML Authentication''' and define following settings (where <hostname> is the name of the QPR ProcessAnalyzer server):
##'''Identifier (Entity ID):''' https://<hostname>/qprpa/Saml2
## '''Reply URL (Assertion Consumer Service URL):''' https://<hostname>/qprpa/Saml2/Acs
## '''Sign on URL:''' It's recommended to leave this setting empty.
# Copy the '''App Federation Metadata Url''' to the clipboard to store it to to the ''SAMLMetadataUrl'' setting in the QPR ProcessAnalyzer configuration table.
# If you want QPR Application to synchronize group membership between Azure AD and QPR ProcessAnalyzer, please add also the '''Group Claim''' from User '''Attributes & Claims'''.

More information about Azure Active Directory: https://docs.microsoft.com/en-us/azure/active-directory/

Instructions for setting up the optional SAML assertions encryption: https://docs.microsoft.com/en-us/azure/active-directory/manage-apps/howto-saml-token-encryption?WT.mc_id=migration_service_aad_-inproduct-azureportal.

==Using ADFS as Identity Provider==
ADFS (Active Directory Federation Services) can be used as an identity provider to login to QPR ProcessAnalyzer. For ADFS setup, follow the ADFS configuration guide in https://docs.microsoft.com/en-us/windows-server/identity/ad-fs/operations/create-a-relying-party-trust with the following notes:
* Step 4: Select option '''Enter data about the relying party manually''' as metadata is not available.
* Step 5: Name can be chosen freely.
* Step 7: Disable option '''Enable support for the WS-Federation Passive protocol'''. Select option '''Enable support for the SAML 2.0 WebSSO protocol''' and define url '''https://<hostname>/qprpa/Saml2/Acs''' where SERVERNAME is the QPR ProcessAnalyzer server hostname.
* Step 8: Define url '''https://<hostname>/qprpa/Saml2/Acs''' where <hostname> is the QPR ProcessAnalyzer server hostname.
* Step 11: Select option '''Configure claims issuance policy for this application'''.

Example:
<pre>
c:[Type == "http://schemas.microsoft.com/ws/2008/06/identity/claims/windowsaccountname", Issuer == "AD AUTHORITY"]
=> issue(store = "Active Directory", types = ("http://schemas.xmlsoap.org/ws/2005/05/identity/claims/upn", "http://schemas.xmlsoap.org/claims/CommonName", "http://schemas.xmlsoap.org/ws/2005/05/identity/claims/emailaddress", "http://schemas.xmlsoap.org/claims/Group"), query = ";userPrincipalName,displayName,mail,tokenGroups;{0}", param = c.Value);
</pre>

== SAML 2.0 Authentication API ==
QPR ProcessAnalyzer has [[Web_API:_saml2/acs|/Saml2/Acs]] endpoint which accepts a SAML assertion from the IdP and returns a HTTP redirection to QPR ProcessAnalyzer Web UI. The url contains a ''sys:samlHash'' parameter which is used by the Web UI to login the user using the [[Web_API:_Token|/token]] endpoint (to get a session token to use in the interactions with the Web API).

== Troubleshooting ==
'''Problem''': When trying to authenticate with SAML, QPR ProcessAnalyzer responds: ''Bad Request - Request too long''.

'''Solution''': This error comes when the SAML assertion (message from IdP to QPR ProcessAnalyzer) contains too long data. The SAML assertion is encoded to an http header (as part of a cookie), which has a certain allowed maximum length. One option is to increase the limits in the Windows http.sys web server (https://learn.microsoft.com/en-us/troubleshoot/developer/webapps/iis/iisadmin-service-inetinfo/httpsys-registry-windows). Alternatively, it may be possible reduce the amount of data included by the IdP to the SAML assertion.

==References==
* General information about federated authentication: https://en.wikipedia.org/wiki/Federated_identity
* General information about SAML 2.0: https://en.wikipedia.org/wiki/SAML_2.0
* SAML 2.0 in Azure AD: https://docs.microsoft.com/en-us/azure/active-directory/develop/single-sign-on-saml-protocol
* General information about ADFS: https://en.wikipedia.org/wiki/Active_Directory_Federation_Services
* ADFS documentation: https://msdn.microsoft.com/en-us/library/bb897402.aspx

[[Category: QPR ProcessAnalyzer]]

Create Simulated Eventlog

2025-08-22T14:22:18Z

MarHink: /* Transformation: event_resource_limits */

This article has instructions how to install, configure and use eventlog simulations. The simulation creates a new model that contains both the source model data and the new simulated data. Case attribute '''Simulated''' can be used to determine whether the case is in the source data (''false'') or whether the simulation generated it as a new simulated case (''true'').

== Prerequisites for simulation ==
As prerequisite, prediction must be installed into the used Snowflake as described in [[Create Predicted Eventlog|Install prediction in Snowflake]].

== Create simulation script in QPR ProcessAnalyzer ==
You can create your own simulation script from scratch, or use one of the scripts provided below as a starting point.

=== Creating a simulation model that deletes specific events ===
1. Create the following example expression script (e.g., with name "'''Create simulation model''' '''- delete events'''"):<syntaxhighlight lang="typescript">
let sourceModel = ProjectByName("<project name>").ModelByName("<model name>");
let flowFromEventType = "<from event type of a flow to modify>", flowToEventType = "<to event type of a flow to modify>";

let targetProject = Project;
_system.ML.ApplyTransformations(#{
"Name": "My simulation model - delete", // Name of the PA model to generate to the target project.
"SourceModel": sourceModel, // Snowflake-based PA model used for training the prediction model.
"TargetProject": targetProject, // Target project to create the model into.
"Transformations": [#{ // Transformation configurations.
"type": "modify_flow_durations",
"column": sourceModel.EventsDataTable.ColumnMappings["EventType"],
"flows": [#{
"from": flowFromEventType,
"to": flowToEventType,
"probability": 1.0,
"operation": #{
"type": "set_value",
"value": 0.0
},
"delete": true
}]
}]
});

</syntaxhighlight>2. Configure simulation for the previously created script as instructed in the next chapter. At minimum, replace the tags listed below with some suitable values:

* '''<project name>''': Name of the project in which the source model is located.
* '''<model name>''': Name of the model to be used as source model. This data in this source model will be used as source data to be modified by the simulation transformations.
* '''<from event type of a flow to modify>''': From-event type name of flows from which the from-event is to be deleted.
* '''<to event type of a flow to modify>''': To-event type name of flows from which the from-event is to be deleted.

=== Create simulation model automating all the resources belonging to the same role as specified resource ===
1. Create the following example expression script (e.g., with name "'''Create simulation model''' '''- automate'''"):<syntaxhighlight lang="typescript">
let sourceModel = ProjectByName("<project name>").ModelByName("<model name>");
let resourceColumnName = "<event data column having resource names>";
let resourceNameToAutomate = "<resource value whose role should be automated>";

let targetProject = Project;
_system.ML.ApplyTransformations(#{
"PredictionProcedureName": "qprpa_sp_prediction",
"Name": "My simulation model - automate", // Name of the PA model to generate to the target project.
"SourceModel": sourceModel, // Snowflake-based PA model used for training the prediction model.
"TargetProject": targetProject, // Target project to create the model into.
"Transformations": [#{ // Transformation configurations
"type": "resources_to_roles",
"resource_column": resourceColumnName,
"role_column": "Role",
"role_name_template": "Role %d"
}, #{
"type": "modify_flow_durations",
"input": #{
"role_name": #{
"input": "resource_to_role_map",
"value_path": [resourceNameToAutomate]
}
},
"column": "Role",
"flows": [#{
"from_input": "role_name",
"operation": #{
"type": "set_value",
"value": 1,
"probability": 0.5
}
}]
}]
});

</syntaxhighlight>2. Configure simulation for the previously created script as instructed in the next chapter. At minimum, replace the tags listed below with some suitable values:
* '''<project name>''': Name of the project in which the source model is located.
* '''<model name>''': Name of the model to be used as source model. This data in this source model will be used as source data to be modified by the simulation transformations.
* '''<event data column having resource names>''': Name of the event data column that contains resource names.
* '''<resource value whose role should be automated>''': Name of the resource whose role is to be automated.

== Configure simulation ==
Simulation script has the following settings in the ApplyTransformations call:

* '''Name''': Name of the QPR ProcessAnalyzer model that is created to the target project. The model will contain the source model content and the predictions.
* '''SourceModel''': Source model for which the simulation is made. Model can be selected for example based on id with ModelById function or by name with ModelByName function.
* '''TargetProject''': Target project to create the new model into.
* '''Transformations''': Array of transformation configuration objects. Each object supports the following parameters:
** '''type''': Defines the type of the transformation to perform. See below for more details on the supported transformations. Supported values are:
*** '''enforce_resource_limits''': Used to modify given event log using given maximum resource limits.
*** '''extract_max_resource_usages''': Used to extract, for every value of a specified column, the maximum number of concurrent cases in given event log that have that value.
*** '''generate''': Used to generate a new event log using a trained ML model.
*** '''modify_flow_durations''': Used to modify durations of flows and possibly remove events having specific flows.
*** '''modify_values''': Used to modify values of a dictionary given as input (e.g., dictionary generated by extract_max_resource_usages).
*** '''resources_to_roles''': Performs "organization mining" by trying to group together column values (e.g., resources) that are used in similar fashion in given event log (e.g., resources that are often present in similar set of activities).
**'''input''': Can be used to specify that given transformation input parameters get their values from the previous transformation result.
***Value can be either direct mapping by just the name of the transformation result property, or it can be a value mapping configuration object that supports the following parameters:
****'''input''': Name of the parameter to get from the previous transformation result as the root object of the actual value to extract.
****'''value_path''': An array of property names to traverse into the root object.

=== Transformation: enforce_resource_limits ===
Using given input data, this transformation generates a new event log which does not exceed the concurrency limitations of specified column values.

Event rows are traversed in time order, and if at some point a limit would be exceeded, instead of outputting the actual event, a new copy of the actual event, with copied event properties, is created to represent the queue for the actual event.

Only after an event leaves from the column value that contains a queue, the event that had been waiting for the longest in the queue will be generated (following the FIFO-principle).

==== Supported parameters ====
* '''column''': Name of the column having the values whose concurrent usage is to be limited by specified limits
* '''limits''': Specifies an object containing key-value -pairs where keys are column values and values contain an integer specifying the maximum number of concurrent cases in the given event log that can contain given value.
* '''queue_event_activity_name''': If set, specifies the name template used for queue-events. In this template, when a queue event is created, %s is replaced with the name of the activity this queue event is queuing to.
** If not set, the activity name is not altered at all for the queue event.
* '''queue_event_column''':
** If queue_event_activity_name is set:
*** If the event represents a queue-event, the value in this column specifies the name of the queue-activity.
*** Otherwise, the value is null.
** If queue_event_activity_name is not set:
*** If the event represents a queue-event, the value in this column True.
*** Otherwise, the value is False.

==== Inputs ====
Event log to operate on.

==== Outputs ====
Event log with resource limits enforced.

==== Example ====
<syntaxhighlight lang="json">
#{
"type": "enforce_resource_limits",
"queue_event_column": "Queue",
"queue_event_activity_name": "%s - Queue",
"limits": #{
"Role 3": None
},
"input": #{
"limits": "role_limits",
"column": "role_column"
}
}
</syntaxhighlight>

=== Transformation: extract_max_resource_usages ===
Extract, for every value of a specified column, the maximum number of concurrent cases in given event log that have that value.

==== Supported parameters ====

* '''resource_column''':
** The name of the column representing the resources whose maximum concurrent case usages are to be calculated.

==== Inputs ====
Event log to operate on.

==== Outputs ====

* max_resource_usages:
** A dictionary object containing resource names as keys (unique resource_column values) and their maximum usage in the event log.

==== Example ====
<syntaxhighlight lang="json">
#{
"type": "extract_max_resource_usages",
"resource_column": "SAP_User"
}
</syntaxhighlight>

=== Transformation: generate ===
Generate a new event log using the configured model [[Create Predicted Eventlog|prediction generation parameters (GenerationConfiguration)]].

==== Supported parameters ====
Supports all the same parameters as those supported by model prediction generation configuration.

==== Inputs ====
Does not support inputs.

==== Outputs ====
Generated event log.

==== Example ====
<syntaxhighlight lang="json">
#{
"type": "generate",
"model_name": "ML model",
"cases_to_generate": 100,
"max_num_events": 20
}
</syntaxhighlight>

=== Transformation: modify_flow_durations ===
Modify durations of flows and possibly remove events having specific flows.

==== Supported parameters ====

* '''column''': The name of the column based on which the flows are created. Usually this is the column containing activities, but could also be, e.g., organization units, users, …
* '''flows''': Flows to transform. Contains an array of flow transformation configuration objects. Each object defines transformations performed on one flow type defined by starting and ending column values. Supports the following properties:
** '''delete''': Same as delete_from.
** '''delete_from''': If defined, specifies whether the "from event" of the matched flow should be removed after applying the operation.
** '''delete_to''': If defined, specifies whether the "to event" of the matched flow should be removed after applying the operation.
** '''from''': Column value starting the flow.
*** If this and from_input are both undefined, any starting value is accepted.
** '''from_input''': If defined, specifies the name of the transformation-level parameter from which the actual column value starting the flow is read from.
*** Overrides the value defined in from-parameter.
** '''operation''': Specifies the actual flow duration modification operation to perform as value modification configuration object where the value is the duration in seconds. Supports the following properties:
*** '''probability''': If defined, specifies the percentage probability of applying the operation to any matching instance of the flow.
**** Value should be a numeric value between 0 and 1.0.
**** This probability applies only to this operation.
**** The default value is 1.0.
*** '''type''': Type of the operation. The following types are supported:
**** '''add''': Sets the value to be the current value plus the number specified by the value.
**** '''multiply''': Sets the value to be the current value multiplied by the number specified by the value.
**** '''set_value''': Sets the value to be exactly the number specified by the value.
*** '''value''': Value used by the operation.
** '''probability''': If defined, specifies the percentage probability of applying the operation to any matching instance of the flow.
*** Value should be a numeric value between 0 and 1.0.
*** This probability applies, in addition to the operation specified by the operation-parameter, also to any possible other transformations, such as event deletion.
*** Default value is 1.0
** '''to''': Column value ending the flow.
*** If this and to_input are both undefined, any ending value is accepted.
** '''to_input''': If defined, specifies the name of the transformation-level parameter to which the actual column value ending the flow is read from.
*** Overrides the value defined in to-parameter.

==== Inputs ====
Event log to operate on.

==== Outputs ====
Transformed event log.

==== Example ====
<syntaxhighlight lang="json">
#{
"type": "modify_flow_durations",
"column": "Organization",
"flows": [#{
"from": "Delivery",
"operation": #{
"type": "set_value",
"value": 0.0
},
"delete": true
}]
}
</syntaxhighlight>

=== Transformation: modify_values ===
Modify values of an object given as input (e.g., object generated by extract_max_resource_usages).

Due to the required inputs, this transformation can't be the first transformation to perform.

==== Supported parameters ====

* '''values''': Array of value configuration objects. Each object supports the following properties:
** '''input''': Name of the result to modify, where the result is the output of the previous transformation.
** '''input_key_from''': If defined, specifies the the name of the property of a input object whose value contains the name of the property whose value is to be modified.
** '''input_key_value_path''': If input_key_from is defined and is represented as an object, this configuration should specify an array of property names to traverse into the object.
*** The value at the end of this path will be used as the name of the property to modify in the input.
** '''operation''': Specifies the actual value modification operation to perform as value modification configuration object. Supports the following properties:
*** '''probability''': If defined, specifies the percentage probability of applying the operation to any matching instance of the flow.
**** Value should be a numeric value between 0 and 1.0.
**** This probability applies only to this operation.
**** The default value is 1.0.
*** '''type''': Type of the operation. The following types are supported:
**** '''add''': Sets the value to be the current value plus the number specified by the value.
**** '''multiply''': Sets the value to be the current value multiplied by the number specified by the value.
**** '''set_value''': Sets the value to be exactly the number specified by the value.
*** '''value''': Value used by the operation.

==== Inputs ====
Output of the previous transformation operation.

==== Outputs ====
The same output as the previous performed transformation, except with the specified value modifications applied.

==== Example ====
<syntaxhighlight lang="json">
#{
"type": "modify_values",
"values": [#{
"input": "role_limits",
"input_key_from": "resource_to_role_map",
"input_key_value_path": ["Tina"],
"operation": #{
"type": "multiply",
"value": 0.5
}
}]
}
</syntaxhighlight>

=== Transformation: resources_to_roles ===
Performs "organization mining" by grouping together column values (e.g., resources) that are used in similar fashion in given event log. E.g., resources that are often present in similar set of activities.

==== Supported parameters ====

* '''resource_column''': The name of the column containing names of resources.
* '''resource_limits''': Contains an object dictionary containing resource names with their maximum concurrent usages.
** If set, when building role_limits output, these values will be summed for each resource into the resulting role-based usage limit.
** If not set, each resource in a role will be counted as one, when calculating the role_limits.
* '''role_column''': The name of the column to be created and whose values will indicate the role in which the resource belongs to.
* '''role_name_template''': If set, specifies the name template used for role names. In this template, %d will be replaced by a numeric value starting from 1.
** The default value is "Role %d".
* '''similarity_threshold''': The minimum value of Pearson correlation coefficient calculated between two resources in order for them to be considered as having the same role.
** The default value is 0.7

==== Inputs ====
Event log to operate on.

==== Outputs ====

* Transformed event log.
* Result dictionary object containing the following properties:
** '''resource_column''': The name of the column containing names of resources.
** '''resource_to_role_map''': An object containing resource names as property names and role names as value.
** '''role_column''': The name of the generated column whose values will indicate the role in which the resource belongs to.
** '''role_limits''': An object containing role names as property names and maximum usage for that role as value.

==== Example ====
<syntaxhighlight lang="json">
#{
"type": "resources_to_roles",
"resource_column": "SAP_User",
"role_column": "Role",
"role_name_template": "Role %d",
"input": #{
"resource_limits": "max_resource_usages"
}
}
</syntaxhighlight>

Create Simulated Eventlog

2025-08-22T13:50:46Z

MarHink: /* Transformation: modify_flow_durations */

This article has instructions how to install, configure and use eventlog simulations. The simulation creates a new model that contains both the source model data and the new simulated data. Case attribute '''Simulated''' can be used to determine whether the case is in the source data (''false'') or whether the simulation generated it as a new simulated case (''true'').

== Prerequisites for simulation ==
As prerequisite, prediction must be installed into the used Snowflake as described in [[Create Predicted Eventlog|Install prediction in Snowflake]].

== Create simulation script in QPR ProcessAnalyzer ==
You can create your own simulation script from scratch, or use one of the scripts provided below as a starting point.

=== Creating a simulation model that deletes specific events ===
1. Create the following example expression script (e.g., with name "'''Create simulation model''' '''- delete events'''"):<syntaxhighlight lang="typescript">
let sourceModel = ProjectByName("<project name>").ModelByName("<model name>");
let flowFromEventType = "<from event type of a flow to modify>", flowToEventType = "<to event type of a flow to modify>";

let targetProject = Project;
_system.ML.ApplyTransformations(#{
"Name": "My simulation model - delete", // Name of the PA model to generate to the target project.
"SourceModel": sourceModel, // Snowflake-based PA model used for training the prediction model.
"TargetProject": targetProject, // Target project to create the model into.
"Transformations": [#{ // Transformation configurations.
"type": "modify_flow_durations",
"column": sourceModel.EventsDataTable.ColumnMappings["EventType"],
"flows": [#{
"from": flowFromEventType,
"to": flowToEventType,
"probability": 1.0,
"operation": #{
"type": "set_value",
"value": 0.0
},
"delete": true
}]
}]
});

</syntaxhighlight>2. Configure simulation for the previously created script as instructed in the next chapter. At minimum, replace the tags listed below with some suitable values:

* '''<project name>''': Name of the project in which the source model is located.
* '''<model name>''': Name of the model to be used as source model. This data in this source model will be used as source data to be modified by the simulation transformations.
* '''<from event type of a flow to modify>''': From-event type name of flows from which the from-event is to be deleted.
* '''<to event type of a flow to modify>''': To-event type name of flows from which the from-event is to be deleted.

=== Create simulation model automating all the resources belonging to the same role as specified resource ===
1. Create the following example expression script (e.g., with name "'''Create simulation model''' '''- automate'''"):<syntaxhighlight lang="typescript">
let sourceModel = ProjectByName("<project name>").ModelByName("<model name>");
let resourceColumnName = "<event data column having resource names>";
let resourceNameToAutomate = "<resource value whose role should be automated>";

let targetProject = Project;
_system.ML.ApplyTransformations(#{
"PredictionProcedureName": "qprpa_sp_prediction",
"Name": "My simulation model - automate", // Name of the PA model to generate to the target project.
"SourceModel": sourceModel, // Snowflake-based PA model used for training the prediction model.
"TargetProject": targetProject, // Target project to create the model into.
"Transformations": [#{ // Transformation configurations
"type": "resources_to_roles",
"resource_column": resourceColumnName,
"role_column": "Role",
"role_name_template": "Role %d"
}, #{
"type": "modify_flow_durations",
"input": #{
"role_name": #{
"input": "resource_to_role_map",
"value_path": [resourceNameToAutomate]
}
},
"column": "Role",
"flows": [#{
"from_input": "role_name",
"operation": #{
"type": "set_value",
"value": 1,
"probability": 0.5
}
}]
}]
});

</syntaxhighlight>2. Configure simulation for the previously created script as instructed in the next chapter. At minimum, replace the tags listed below with some suitable values:
* '''<project name>''': Name of the project in which the source model is located.
* '''<model name>''': Name of the model to be used as source model. This data in this source model will be used as source data to be modified by the simulation transformations.
* '''<event data column having resource names>''': Name of the event data column that contains resource names.
* '''<resource value whose role should be automated>''': Name of the resource whose role is to be automated.

== Configure simulation ==
Simulation script has the following settings in the ApplyTransformations call:

* '''Name''': Name of the QPR ProcessAnalyzer model that is created to the target project. The model will contain the source model content and the predictions.
* '''SourceModel''': Source model for which the simulation is made. Model can be selected for example based on id with ModelById function or by name with ModelByName function.
* '''TargetProject''': Target project to create the new model into.
* '''Transformations''': Array of transformation configuration objects. Each object supports the following parameters:
** '''type''': Defines the type of the transformation to perform. See below for more details on the supported transformations. Supported values are:
*** '''enforce_resource_limits''': Used to modify given event log using given maximum resource limits.
*** '''extract_max_resource_usages''': Used to extract, for every value of a specified column, the maximum number of concurrent cases in given event log that have that value.
*** '''generate''': Used to generate a new event log using a trained ML model.
*** '''modify_flow_durations''': Used to modify durations of flows and possibly remove events having specific flows.
*** '''modify_values''': Used to modify values of a dictionary given as input (e.g., dictionary generated by extract_max_resource_usages).
*** '''resources_to_roles''': Performs "organization mining" by trying to group together column values (e.g., resources) that are used in similar fashion in given event log (e.g., resources that are often present in similar set of activities).
**'''input''': Can be used to specify that given transformation input parameters get their values from the previous transformation result.
***Value can be either direct mapping by just the name of the transformation result property, or it can be a value mapping configuration object that supports the following parameters:
****'''input''': Name of the parameter to get from the previous transformation result as the root object of the actual value to extract.
****'''value_path''': An array of property names to traverse into the root object.

=== Transformation: event_resource_limits ===
Using given input data, this transformation generates a new event log which does not exceed the concurrency limitations of specified column values.

Event rows are traversed in time order, and if at some point a limit would be exceeded, instead of outputting the actual event, a new copy of the actual event, with copied event properties, is created to represent the queue for the actual event.

Only after an event leaves from the column value that contains a queue, the event that had been waiting for the longest in the queue will be generated (following the FIFO-principle).

==== Supported parameters ====
* '''column''': Name of the column having the values whose concurrent usage is to be limited by specified limits
* '''limits''': Specifies an object containing key-value -pairs where keys are column values and values contain an integer specifying the maximum number of concurrent cases in the given event log that can contain given value.
* '''queue_event_activity_name''': If set, specifies the name template used for queue-events. In this template, when a queue event is created, %s is replaced with the name of the activity this queue event is queuing to.
** If not set, the activity name is not altered at all for the queue event.
* '''queue_event_column''':
** If queue_event_activity_name is set:
*** If the event represents a queue-event, the value in this column specifies the name of the queue-activity.
*** Otherwise, the value is null.
** If queue_event_activity_name is not set:
*** If the event represents a queue-event, the value in this column True.
*** Otherwise, the value is False.

==== Inputs ====
Event log to operate on.

==== Outputs ====
Event log with resource limits enforced.

==== Example ====
<syntaxhighlight lang="json">
#{
"type": "enforce_resource_limits",
"queue_event_column": "Queue",
"queue_event_activity_name": "%s - Queue",
"limits": #{
"Role 3": None
},
"input": #{
"limits": "role_limits",
"column": "role_column"
}
}
</syntaxhighlight>

=== Transformation: extract_max_resource_usages ===
Extract, for every value of a specified column, the maximum number of concurrent cases in given event log that have that value.

==== Supported parameters ====

* '''resource_column''':
** The name of the column representing the resources whose maximum concurrent case usages are to be calculated.

==== Inputs ====
Event log to operate on.

==== Outputs ====

* max_resource_usages:
** A dictionary object containing resource names as keys (unique resource_column values) and their maximum usage in the event log.

==== Example ====
<syntaxhighlight lang="json">
#{
"type": "extract_max_resource_usages",
"resource_column": "SAP_User"
}
</syntaxhighlight>

=== Transformation: generate ===
Generate a new event log using the configured model [[Create Predicted Eventlog|prediction generation parameters (GenerationConfiguration)]].

==== Supported parameters ====
Supports all the same parameters as those supported by model prediction generation configuration.

==== Inputs ====
Does not support inputs.

==== Outputs ====
Generated event log.

==== Example ====
<syntaxhighlight lang="json">
#{
"type": "generate",
"model_name": "ML model",
"cases_to_generate": 100,
"max_num_events": 20
}
</syntaxhighlight>

=== Transformation: modify_flow_durations ===
Modify durations of flows and possibly remove events having specific flows.

==== Supported parameters ====

* '''column''': The name of the column based on which the flows are created. Usually this is the column containing activities, but could also be, e.g., organization units, users, …
* '''flows''': Flows to transform. Contains an array of flow transformation configuration objects. Each object defines transformations performed on one flow type defined by starting and ending column values. Supports the following properties:
** '''delete''': Same as delete_from.
** '''delete_from''': If defined, specifies whether the "from event" of the matched flow should be removed after applying the operation.
** '''delete_to''': If defined, specifies whether the "to event" of the matched flow should be removed after applying the operation.
** '''from''': Column value starting the flow.
*** If this and from_input are both undefined, any starting value is accepted.
** '''from_input''': If defined, specifies the name of the transformation-level parameter from which the actual column value starting the flow is read from.
*** Overrides the value defined in from-parameter.
** '''operation''': Specifies the actual flow duration modification operation to perform as value modification configuration object where the value is the duration in seconds. Supports the following properties:
*** '''probability''': If defined, specifies the percentage probability of applying the operation to any matching instance of the flow.
**** Value should be a numeric value between 0 and 1.0.
**** This probability applies only to this operation.
**** The default value is 1.0.
*** '''type''': Type of the operation. The following types are supported:
**** '''add''': Sets the value to be the current value plus the number specified by the value.
**** '''multiply''': Sets the value to be the current value multiplied by the number specified by the value.
**** '''set_value''': Sets the value to be exactly the number specified by the value.
*** '''value''': Value used by the operation.
** '''probability''': If defined, specifies the percentage probability of applying the operation to any matching instance of the flow.
*** Value should be a numeric value between 0 and 1.0.
*** This probability applies, in addition to the operation specified by the operation-parameter, also to any possible other transformations, such as event deletion.
*** Default value is 1.0
** '''to''': Column value ending the flow.
*** If this and to_input are both undefined, any ending value is accepted.
** '''to_input''': If defined, specifies the name of the transformation-level parameter to which the actual column value ending the flow is read from.
*** Overrides the value defined in to-parameter.

==== Inputs ====
Event log to operate on.

==== Outputs ====
Transformed event log.

==== Example ====
<syntaxhighlight lang="json">
#{
"type": "modify_flow_durations",
"column": "Organization",
"flows": [#{
"from": "Delivery",
"operation": #{
"type": "set_value",
"value": 0.0
},
"delete": true
}]
}
</syntaxhighlight>

=== Transformation: modify_values ===
Modify values of an object given as input (e.g., object generated by extract_max_resource_usages).

Due to the required inputs, this transformation can't be the first transformation to perform.

==== Supported parameters ====

* '''values''': Array of value configuration objects. Each object supports the following properties:
** '''input''': Name of the result to modify, where the result is the output of the previous transformation.
** '''input_key_from''': If defined, specifies the the name of the property of a input object whose value contains the name of the property whose value is to be modified.
** '''input_key_value_path''': If input_key_from is defined and is represented as an object, this configuration should specify an array of property names to traverse into the object.
*** The value at the end of this path will be used as the name of the property to modify in the input.
** '''operation''': Specifies the actual value modification operation to perform as value modification configuration object. Supports the following properties:
*** '''probability''': If defined, specifies the percentage probability of applying the operation to any matching instance of the flow.
**** Value should be a numeric value between 0 and 1.0.
**** This probability applies only to this operation.
**** The default value is 1.0.
*** '''type''': Type of the operation. The following types are supported:
**** '''add''': Sets the value to be the current value plus the number specified by the value.
**** '''multiply''': Sets the value to be the current value multiplied by the number specified by the value.
**** '''set_value''': Sets the value to be exactly the number specified by the value.
*** '''value''': Value used by the operation.

==== Inputs ====
Output of the previous transformation operation.

==== Outputs ====
The same output as the previous performed transformation, except with the specified value modifications applied.

==== Example ====
<syntaxhighlight lang="json">
#{
"type": "modify_values",
"values": [#{
"input": "role_limits",
"input_key_from": "resource_to_role_map",
"input_key_value_path": ["Tina"],
"operation": #{
"type": "multiply",
"value": 0.5
}
}]
}
</syntaxhighlight>

=== Transformation: resources_to_roles ===
Performs "organization mining" by grouping together column values (e.g., resources) that are used in similar fashion in given event log. E.g., resources that are often present in similar set of activities.

==== Supported parameters ====

* '''resource_column''': The name of the column containing names of resources.
* '''resource_limits''': Contains an object dictionary containing resource names with their maximum concurrent usages.
** If set, when building role_limits output, these values will be summed for each resource into the resulting role-based usage limit.
** If not set, each resource in a role will be counted as one, when calculating the role_limits.
* '''role_column''': The name of the column to be created and whose values will indicate the role in which the resource belongs to.
* '''role_name_template''': If set, specifies the name template used for role names. In this template, %d will be replaced by a numeric value starting from 1.
** The default value is "Role %d".
* '''similarity_threshold''': The minimum value of Pearson correlation coefficient calculated between two resources in order for them to be considered as having the same role.
** The default value is 0.7

==== Inputs ====
Event log to operate on.

==== Outputs ====

* Transformed event log.
* Result dictionary object containing the following properties:
** '''resource_column''': The name of the column containing names of resources.
** '''resource_to_role_map''': An object containing resource names as property names and role names as value.
** '''role_column''': The name of the generated column whose values will indicate the role in which the resource belongs to.
** '''role_limits''': An object containing role names as property names and maximum usage for that role as value.

==== Example ====
<syntaxhighlight lang="json">
#{
"type": "resources_to_roles",
"resource_column": "SAP_User",
"role_column": "Role",
"role_name_template": "Role %d",
"input": #{
"resource_limits": "max_resource_usages"
}
}
</syntaxhighlight>

Expression Script Examples

2025-04-25T11:27:24Z

MarHink:

This page contains script examples written in the QPR ProcessAnalyzer expression language. See how expression scripts can be created in the [[Managing_Scripts#Creating_Script|Workspace]]. For documentation for the syntax, functions and entities can be found from the main page in the [[QPR_ProcessAnalyzer_Wiki#For_Developers|KPI Expression Language]] section.

== Calling Expression Script from Expression ==
Expression scripts can be called from an expression using the [[QPR_ProcessAnalyzer_Objects_in_Expression_Language#Script|Run]] function with the following syntax:
<pre>
let result = ScriptById(123).Run(#{
"parameter1": "value1",
"parameter2": false,
"parameter2": 123.45
})
</pre>

The command waits until the run is completed, and the return value of the called script is returned by the Run function call.

Parameters can be passed to the called script, and the parameters are available as variables in the script. The parameters can contain any type of data.

Expression scripts can also be called from a dashboard. Expressions can be stored to scripts instead of dashboards, which is a way to separate complex expressions from dashboards and allow to reuse expressions across several dashboards.

== Calling SQL Script from Expression ==
SQL script can be called from an expression using the Run function as follows (similar to calling [[#Calling Expression Script from Expression|expression scripts]]):
<pre>
let result = ScriptById(123).Run(#{
"parameter1": "value1",
"parameter2": 321
});
let arrayOfAllReports = result.Keys;
let report1 = result.Report1;
let report2 = result.Report2;
</pre>

SQL scripts can return multiple ''reports'', which are combined to a dictionary, where the key is the name of the report ("sheet name") and value is the report data as a DataFrame. See in the above example, how the reports can be accessed by their name.

== Examples ==
=== Call web service===
Contact to a web service, fetch some data, and store it to a datatable.

<syntaxhighlight lang="typescript" line>
let datatableName = "Web Service Data";
let webServiceData = CallWebService(
#{"Address": "https://processanalyzer.onqpr.com/qprpa/api/serverinfo"}
);

let targetDatatable = Project.Datatables.Where(name==datatableName);
if (Count(targetDatatable) == 0) {
targetDatatable = Project.CreateDatatable(datatableName)
.AddColumn("Setting name", "String")
.AddColumn("Setting value", "String")
.AddColumn("Data read", "DateTime");
} else {
targetDatatable = targetDatatable[0];
}

let currentTime = Now;
let dataAsDf = ToDataFrame(
webServiceData.keys.{
let key = _;
[key, webServiceData[key], currentTime];
},
["Setting name", "Setting value", "Data read"]
);
targetDatatable.Import(dataAsDf, #{"Append":true});
WriteLog(`${CountTop(dataAsDf.Rows)} rows written to datatable`);
</syntaxhighlight>

=== Store data to datatable ===
Get all models in the system and store them to a datatable.

<syntaxhighlight lang="typescript" line>
let newDatatable = Project
.CreateDatatable("Models list " + ToString(Now, "dd.MM.yyyy HH:mm:ss"))
.AddColumn("Model name", "String")
.AddColumn("Project name", "String")
.AddColumn("Created time", "DateTime")
.AddColumn("Cases", "Integer");
let startTime = Now;
let modelsData = ToDataFrame(
Models.([Name, Project.Name, CreatedDate, NCases]),
["Model name", "Project name", "Created time", "Cases"]
);
WriteLog(`Listing models took ${(Now - startTime).TotalSeconds.Round(2)} seconds.`);
newDatatable.Import(modelsData);
WriteLog(`Datatable ${newDatatable.Id} created.`);
</syntaxhighlight>

=== Convert datatable column data ===
This script can be used to convert a single column into numerical data type. To use the script, you need to setup the following in the beginning of the script:
* Project name where the datatable is located.
* Datatable name
* Name of the column to be converted

Note that the conversion fails, if there is data that cannot be converted into numerical format. The conversion assumes that period (.) is used as the decimal point.

<syntaxhighlight lang="typescript" line>
let projectName = "New Project";
let datatableName = "qpr processanalyzer events";
let columnName = "Event order in case";

let project = (Projects.Where(Name==projectName))[0];
let datatable = (project.Datatables.Where(Name==datatableName))[0];
DatatableById(datatable.Id).DataFrame
.SetColumns([
columnName: () => {
let data = Column(columnName);
if (data == null) {
null;
} else {
ToFloat(data);
}
}
])
.Persist(datatable.Name, ["ProjectId": project.Id, "Append": false]);
</syntaxhighlight>

Instead of converting to numeric (with the ''ToFloat'' function), data can be converted into string using the ''ToString'' function.

=== Show DataFrame as HTML table ===

This script defines a function to show dataframe as a HTML table, and uses the function for a literal dataframe.

<syntaxhighlight lang="typescript" line>
function dataframeToHtmlTable(df) {
return
`<table>
<tr>
${StringJoin("\r\n\t\t", + df.columns.`<th>${_}</th>`)}
</tr>
${StringJoin("", df.Rows.(
"\r\n\t<tr>" + StringJoin("", _.`\r\n\t\t<td>${ToString(_)}</td>`) + "\r\n\t</tr>"
))}
</table>`
}

let data = ToDataFrame(
[
["one", "two", "three"],
["four", "five", "six"],
["seven", "eight", "nine"]
],
["Column 1", "Column 2", "Column 3"]
);

return dataframeToHtmlTable(data);
</syntaxhighlight>

=== Copy local datatables to Snowflake ===

<syntaxhighlight lang="typescript" line>
// Copies all datatables in a project to another project including datatable contents.
// Usage instructions:
// 1. Create expression script in the project from where you want to copy the datatables.
// 2. Create a new project named as "<name of the project to be moved> - Snowflake". New datatables will be created here. E.g., when moving project named "SAP_OrderToCash", the target project should be named as "SAP_OrderToCash - Snowflake".
// 3. Run the script.
// NOTE: Columns of type "Any" will be created as "String"-columns in Snowflake, thus it is recommended that actual data types are set for the tables prior to the move.

let sourceProject = Project;
let sourceProjectName = Project.Name;
let targetProjectName = `${sourceProjectName} - Snowflake`;
let targetProject = First(Projects.Where(Name == targetProjectName));
if (IsNull(targetProject)) {
WriteLog(`Unable to find target project named "${targetProjectName}". Aborting operation.`);
return;
}
let dts = sourceProject.DataTables;
WriteLog(`Copying all ${CountTop(dts)} data tables found in project "${sourceProject.Name}" (id: ${sourceProject.Id}) to Snowflake in project "${targetProject.Name}" (id: ${targetProject.Id})`);
dts.{
let sourceDt = _;
WriteLog(`Starting to copy data table "${Name}" (id: ${Id}) having ${NRows} rows and ${NColumns} columns.`);
let targetDt;
targetDt = targetProject.DatatableByName(sourceDt.Name);
if (targetDt == null) {
targetDt = targetProject.CreateDataTable(sourceDt.Name, #{"Connection": CreateSnowflakeConnection(#{"ProjectId": targetProject.Id})});
targetDt.Import(sourceDt.SqlDataFrame);
WriteLog(`Finished copying data table "${Name}" (id: ${Id}) to table "${targetDt.Name}" (id: ${targetDt.Id})`);
} else {
WriteLog(`Datatable already exist "${Name}" (id: ${Id}) to table "${targetDt.Name}" (id: ${targetDt.Id})`);
}
}
WriteLog(`Finished copying all the data tables found in project "${sourceProject.Name}" (id: ${sourceProject.Id}) to Snowflake in project "${targetProject.Name}" (id: ${targetProject.Id})`);
</syntaxhighlight>

If you don't need to copy the data but only create the Snowflake datatables with columns, you can change the line 22 to
<syntaxhighlight>
targetDt.Import(sourceDt.SqlDataFrame.head(0));
</syntaxhighlight>

=== Copy single datatable to Snowflake ===
This script creates a copy of a single datatable to Snowflake. Replace the ''<tableId1>'' with the id of the source datatable.
<syntaxhighlight lang="typescript" line>
function CopyDataTableToSnowflake(dataTableId)
{
let sourceDt = DataTableById(dataTableId);
sourceDt.SqlDataFrame.Persist(`${sourceDt.Name} - Snowflake`, #{"Append": false, "Connection": CreateSnowflakeConnection(#{"ProjectId": sourceDt.Project.Id})});
}
CopyDataTableToSnowflake(<tableId1>);
</syntaxhighlight>

=== Create a copy of a data table that has all Any-type columns changed to String-type columns ===
<syntaxhighlight lang="typescript" line>
function ConvertAnyDataTypesToStringsToNewTable(dataTableId)
{
let dt = DataTableById(dataTableId);
let sdf = dt.SqlDataFrame;
let cts = dt.ColumnTypes;
cts.{
let ct = _;
if (ct.DataType == "Any") {
let n = ct.Name;
sdf = sdf.WithColumn(ct.Name, #sql{Cast(Column(Variable("n")), "ShortString")});
}
};
sdf.Persist(`${dt.Name} - Converted`, #{"Append": false, "ProjectId": dt.Project.Id});
}
ConvertAnyDataTypesToStringsToNewTable(<dataTableId>);
</syntaxhighlight>

=== Query number of rows in given data table having a datetime value in given year grouped by month and return resulting table as CSV ===
SqlDataFrame is used in order to prevent loading the whole datatable into memory first. Filtering is performed as first operation in order to minimize the amount of required work for the data source of the data table.<syntaxhighlight lang="typescript" line="1">
DataTableById(<data table id>)
.SqlDataFrame
.Where(#sql{2014 == Year(Column("Start Time"))})
.WithColumn("Month", #sql{Month(Column("Start Time"))})
.GroupBy(["Month"]).Aggregate(["Count"], ["Count"])
.OrderByColumns(["Month"], [true])
.Collect().ToCsv();
</syntaxhighlight>

=== Function for filtering SqlDataFrame by removing rows having, or replacing, the most infrequently occurring column values ===
<syntaxhighlight lang="typescript" line="1">
/***
* @name ColumnWithMinUsage
* @descripion
* Generic function that can be used to filter out the most infrequently occurring attribute values or replace their Values
* with given common value.
* @param df:
* DataFrame to operate on.
* @param columnName:
* Name of the column to be filtered.
* @param newColumnName:
* Name of the column that will contain the new value of the original column after filtering (if includeOthers was applied).
* @param maxNumUniqueValues:
* Maximum number of unique values to include into the comparison for each attribute column. If the amount of unique values for any attribute exceeds this value, only given number of attributes are included that have the highest usage.
* @param minValueUsage:
* Minimum total usage of a value included into the comparison. The number of cases having every returned value should be at least given percentage (a float value between 0.0 and 1.0) of all the compared cases.
* @param includeOthers:
* Should the rest of the attribute values not included due to MinValueUsage or MaxNumUniqueValues filtering be included as an aggregated "Others" value?
* If not empty/null, defines the name used for these other-values.
*/
function ColumnWithMinUsage(df, columnName, newColumnName, maxNumUniqueValues, minValueUsage, includeOthers)
{
let all = df
.GroupBy([])
.Aggregate(["NAllTotal"], ["Count"])
.WithColumn("__Join2", #sql{1});
let minValueUsageEnabled = !IsNullTop(minValueUsage);
let maxNumUniqueValuesEnabled = !IsNullTop(maxNumUniqueValues);
if (minValueUsageEnabled || maxNumUniqueValuesEnabled) {
// Perform column value-based filtering if minValueUsageEnabled or maxNumUniqueValuesEnabled is defined.
let valueColumnName = "__ValueNew";
let filteredValuesColumns = [valueColumnName: columnName];
let filteredValues = df
.GroupBy([columnName]).Aggregate(["Count"], ["Count"]);
if (minValueUsageEnabled) {
filteredValues = filteredValues
.WithColumn("__Join", #sql{1})
.Join(all, ["__Join": "__Join2"], "leftouter")
.WithColumn("Usage", #sql{Column("Count") / Column("NAllTotal")});
filteredValuesColumns = Concat(filteredValuesColumns, ["Usage"]);
}
if (maxNumUniqueValuesEnabled) {
filteredValues = filteredValues
.WithRowNumberColumn("RowNumber", ["Count"], null, [false]);
filteredValuesColumns = Concat(filteredValuesColumns, ["RowNumber"]);
}

filteredValues = filteredValues
.Select(filteredValuesColumns);

// Generate select returning all the accepted values.
let allValues = filteredValues
.(minValueUsageEnabled ? Where(#sql{Column("Usage") >= #expr{minValueUsage}}) : _)
.(maxNumUniqueValuesEnabled ? Where(#sql{Column("RowNumber") <= #expr{maxNumUniqueValues}}) : _)
.Select([valueColumnName, newColumnName: valueColumnName]);

if (!IsNullTop(includeOthers)) {
// If includeOthers is defined, replace original values with the variable defined in includeOthers.
let otherValues = filteredValues
.(minValueUsageEnabled ? Where(#sql{Column("Usage") < #expr{minValueUsage}}) : _)
.(maxNumUniqueValuesEnabled ? Where(#sql{Column("RowNumber") > #expr{maxNumUniqueValues}}) : _)
.WithColumn(newColumnName, #sql{#expr{includeOthers}})
.Select([valueColumnName, newColumnName]);
allValues = allValues.Append(otherValues)
}
df.Join(allValues, [columnName: valueColumnName], "inner")
.RemoveColumns([valueColumnName]);
}
}

// The following example will return only rows containing two of the most common values for Region-column.
//let df = DataTableById(<data table id>).SqlDataFrame;
//df = ColumnWithMinUsage(df, "Region", "_Filtered", 2, null, null);
//df.Collect().ToCsv();

// The following example will return all input rows, but will replace the values of rows whose Region-column
// has a value used by less than 15% of all the rows with a new value: "_Others".
//let df = DataTableById(<data table id>).SqlDataFrame;
//df = ColumnWithMinUsage(df, "Region", "_Filtered", null, 0.15, "_Others");
//df.Collect().ToCsv();
</syntaxhighlight>

=== Export model events and cases ===
<syntaxhighlight lang="typescript" line="1">
function ExportModelEvents(m) {
let attrs = m.EventAttributes;
ToDataFrame(
m.EventLog.Events.Concat(
[Case.Name, Type.Name, ToString(TimeStamp, "yyyy-MM-dd HH:mm:ss.fff")],
{let evt = _; attrs.{let att = _; evt.Attribute(att)}}
),
Concat(
["CaseId", "EventType", "TimeStamp"],
attrs.Name
)
).ToCsv(true);
}

function ExportModelCases(m) {
let attrs = m.CaseAttributes;
ToDataFrame(
m.EventLog.Cases.Concat(
[Name],
{let cas = _; attrs.{let att = _; cas.Attribute(att)}}
),
Concat(
["CaseId"],
attrs.Name
)
).ToCsv(true);
}
</pre>

<pre>
First(Models.Where(Name=="SAP OtC Extended")).EventsDataTable.DataFrame.ToCsv(true)
</pre>

<pre>
First(Models.Where(Name=="SAP OtC Extended")).CasesDataTable.DataFrame.ToCsv(true)
</syntaxhighlight>

=== Calculate all the value usages of a single column for each event in event data table ===
This query could be used, e.g., to find out the maximum resource usage for every resource found in the event data table.<syntaxhighlight lang="typescript" line="1">
function WithUsageColumns(resourceColumn)
{
function WithTotalUsageColumnOfSingleResource(resourceColumn, resourceValue)
{
_
.WithColumn("_Prev", #sql{Lag(Column(resourceColumn), [TimeStamp, EventType], [true, true], [CaseId], 1, null)})
.WithColumn("_UsageDiff", #sql{
CaseWhen(
Column(resourceColumn) == Column("_Prev"), 0,
Column("_Prev") == #expr{resourceValue}, -1,
Column(resourceColumn) == #expr{resourceValue}, 1,
0)
})
.WithColumn(`${resourceValue}_Usage`, #sql{Sum(Column("_UsageDiff"), [TimeStamp, EventType])})
.RemoveColumns(["_Prev", "_UsageDiff"])
}

let sdf = _;
let allValues = sdf.SelectDistinct([resourceColumn]).OrderByColumns([resourceColumn], [true]).Collect().Column(resourceColumn);
allValues.{
let v = _;
sdf = sdf.WithTotalUsageColumnOfSingleResource(resourceColumn, v)
}
sdf
}

let dt = ModelById(<model id>).EventsDataTable;
dt
.SqlDataFrame
.WithUsageColumns(<resource column name>)
.OrderByColumns([dt.ColumnMappings["TimeStamp"]], [true])
.Collect().ToCsv()
</syntaxhighlight>Where:

* <model id> is the id of the model containing event data to be examined.
* <resource column name> is the name of the column in the event data table of the specified model containing the resource being used by that event.

NOTE: This expression uses functionalities that are only supported in Snowflake-based data tables.

=== Create new Snowflake model from filter ===
This script creates a new Snowflake model (and two datatables for cases and events) containing filtered event log from given filter id. The script also works if the model doesn't have a cases datatable.

<syntaxhighlight lang="typescript" line="1">
let filter = FilterById(1); // filter id
let model = filter.model;
let project = model.project;
let nameSuffix = " - " + filter.name + " - " + ToString(Now, "dd-MM-yyyy HH:mm:ss");
let eventsDatatableName = model.EventsDataTable.Name + nameSuffix;
if (eventsDatatableName.length > 440) {
eventsDatatableName = eventsDatatableName.Substring(eventsDatatableName.length - 440);
}
let eventsData = model
.EventsDataTable
.SqlDataFrame
.ApplyFilter(
filter.rules,
model.CasesDataTable?.SqlDataFrame
);
project
.CreateDatatable(eventsDatatableName, #{"Connection": CreateSnowflakeConnection()})
.Import(eventsData);
let modelConfiguration = model.Configuration;
modelConfiguration.DataSource.Events.Set("DataTableName", eventsDatatableName);
if (model.CasesDataTable != null) {
let eventsDataCaseIdColumn = "CaseId_" + ToString(Random());
let casesDatatableName = model.CasesDataTable.Name + nameSuffix;
if (casesDatatableName.length > 440) {
casesDatatableName = casesDatatableName.Substring(casesDatatableName.length - 440);
}
let casesData = model
.CasesDataTable
.SqlDataFrame
.join(
eventsData.SelectDistinct([eventsDataCaseIdColumn: modelConfiguration.DataSource.Events.Columns.CaseId]),
[modelConfiguration.DataSource.Cases.Columns.CaseId: eventsDataCaseIdColumn]
).Select(model.CasesDataTable.ColumnNames);
project
.CreateDatatable(casesDatatableName, #{"Connection": CreateSnowflakeConnection()})
.Import(casesData);
modelConfiguration.DataSource.Cases.Set("DataTableName", casesDatatableName);
}
let modelName = model.Name + nameSuffix;
if (modelName > 440) {
modelName = modelName.Substring(modelName.length - 440);
}
project
.CreateModel(#{
"Name": modelName,
"Description": model.Description,
"Configuration": modelConfiguration
});
return modelName;
</syntaxhighlight>

=== Creating a model consisting of multiple copies of cases in an existing model ===
<syntaxhighlight lang="typescript" line="1">
/**
* @name CreateTestModel
* @description
* Creates a new model (or overwrites an existing) to given target project with given number of
* repetitions of given source model.
* Each repetition will generate "<N>-"-prefix to CaseId-columns, where N equals to the repeat index.
* @param sourceModel
* PA model used for the source data and from where the connection is copied for the target model if a
* new one has to be created.
* @param numRepeats
* Number of times the data in the source model should be repeated in the generated model.
* @param targetProject
* Project in which the target model resides.
* @param targetModelName
* Specifies the name of the test model in the given target project. If a model already exists with
* given name, event and case data in this model will be replaced with the new generated event and
* case data.
* @returns
* Model object of the test model having the newly generated data.
*/
function CreateTestModel(sourceModel, numRepeats, targetProject, targetModelName)
{
let eventsColumnMappings = sourceModel.EventsDataTable.ColumnMappings;
let casesColumnMappings = sourceModel.CasesDataTable.ColumnMappings;
let connection = sourceModel.EventsDataTable.DataSourceConnection;

function CreateResultModel()
{
function GetTable(tableName)
{
let tableConfiguration = #{
"Name": tableName,
"Connection": connection
};
let resultTable = targetProject.DataTableByName(tableName);
if (resultTable == null)
{
resultTable = targetProject.CreateDataTable(tableConfiguration)
.Modify(#{"NameInDataSource": null})
.Synchronize();
}
return resultTable;
}

let eventsTableName = `${targetModelName} - events`;
let casesTableName = `${targetModelName} - cases`;
let targetModel = targetProject.ModelByName(targetModelName);
let eventsTable, casesTable = null;

if (targetModel != null)
{
eventsTable = targetModel.EventsDataTable;
casesTable = targetModel.CasesDataTable;
}
else {
eventsTable = GetTable(eventsTableName);
if (sourceModel.CasesDataTable != null) {
casesTable = GetTable(casesTableName);
}

let timestampMapping = eventsColumnMappings["TimeStamp"];
eventsColumnMappings.Remove("TimeStamp");
eventsColumnMappings.Set("Timestamp", timestampMapping);

let modelConfiguration = #{
"DataSource": #{
"Events":#{
"DataSourceType": "datatable",
"DataTableName": eventsTableName,
"Columns": eventsColumnMappings
}
}
};

if (casesColumnMappings != null) {
modelConfiguration["DataSource"].Set("Cases", #{
"DataSourceType": "datatable",
"DataTableName": casesTableName,
"Columns": casesColumnMappings
});
}

targetModel = targetProject.CreateModel(#{"Name": targetModelName, "Configuration": modelConfiguration});
}

eventsTable.Truncate();
casesTable?.Truncate();

return #{
"TargetModel": targetModel,
"Events": eventsTable,
"Cases": casesTable
};
}

function RepeatNTimes(sourceDf, caseIdColumn, numRepeats)
{
let resultDf = null;
for (let i = 1; i <= numRepeats; ++i) {
let iterationDf = sourceDf
.WithColumn(caseIdColumn, #sql{Concat(#expr{i}, "-", Column(#expr{caseIdColumn}))});
resultDf = resultDf == null ? iterationDf : resultDf.Append(iterationDf);
}
resultDf;
}

let resultModel = CreateResultModel();
let sourceEventDataDf = sourceModel.EventsDataTable.SqlDataFrame;
let resultEventDataDf = RepeatNTimes(sourceEventDataDf, eventsColumnMappings["CaseId"], numRepeats);
resultModel["Events"].Import(resultEventDataDf);

let sourceCaseDataDf = sourceModel.CasesDataTable?.SqlDataFrame;
if (sourceCaseDataDf != null) {
let resultCaseDataDf = RepeatNTimes(sourceCaseDataDf, casesColumnMappings["CaseId"], numRepeats);
resultModel["Cases"].Import(resultCaseDataDf);
}
resultModel["TargetModel"];
}

</syntaxhighlight>Example usage:<blockquote>CreateTestModel(ProjectByName("Project").ModelByName("SAP_OrderToCash - Snowflake"), 3, ProjectByName("TestData"), "TestModel");</blockquote>Creates a new model named "TestModel" (or overwrites old one) into project named "TestData" containing the data from model "SAP_OrderToCash - Snowflake" in project "Project" repeated three times.

=== Analyzing declare patterns found in event log ===

This is an example expression that shows how POSIX-style regular expressions can be used to search for cases in an event log having certain event type patterns [https://www.researchgate.net/publication/277631859_Generating_Event_Logs_Through_the_Simulation_of_Declare_Models declare patterns].

Note: Before use, replace <model id> with a valid model identifier having event data.

<syntaxhighlight lang="typescript" line="1">
let dt = ModelById(<model id>).EventsDataTable;
let sdf = dt.SqlDataFrame.Head(1000);
let caseIdColumn = dt.ColumnMappings["CaseId"], timeStampColumn = dt.ColumnMappings["TimeStamp"], eventTypeColumn = dt.ColumnMappings["EventType"];

let eventTypesDf = sdf
.SelectDistinct([eventTypeColumn])
.OrderByColumns([eventTypeColumn], [true])
.WithRowNumberColumn("Token", [eventTypeColumn])
.WithColumn("Token", #sql{Char(Column("Token") + Unicode("a") - 1)});

sdf = sdf
.Join(eventTypesDf.Select(["_EventType2": eventTypeColumn, "Token"]), [eventTypeColumn: "_EventType2"], "leftouter");

function RespondedExistencePattern(a, b)
{
// [^a]*((a.*b.*)|(b.*a.*))*[^a]*
ToSqlExpression(`Regexp(Column("Trace"), "[^${a}]*((${a}.*${b}.*)|(${b}.*${a}.*))*[^${a}]")`)
}

function ResponsePattern(a, b)
{
// [^a]*(a.*b)*[^a]*
ToSqlExpression(`Regexp(Column("Trace"), "[^${a}]*(${a}.*${b})*[^${a}]")`)
}

function AlternateResponsePattern(a, b)
{
// [^a]*(a[^a]*b[^a]*)*[^a]*
ToSqlExpression(`Regexp(Column("Trace"), "[^${a}]*(${a}[^${a}]*${b}[^${a}]*)*[^${a}]")`)
}

function ChainResponsePattern(a, b)
{
// [^a]*(ab[^a]*)*[^a]*
ToSqlExpression(`Regexp(Column("Trace"), "[^${a}]*(${a}${b}[^${a}]*)*[^${a}]")`)
}

function PrecedencePattern(a, b)
{
// [^b]*(a.*b)*[^b]*
ToSqlExpression(`Regexp(Column("Trace"), "[^${b}]*(${a}.*${b})*[^${b}]")`)
}

let tracesDf = sdf
.GroupBy([caseIdColumn])
.Aggregate(["Trace": "Token"], [#{"Function": "list", "Ordering": [timeStampColumn], "Separator": ""}])
.WithColumn("RespondedExistenceNL", #sql{#expr{RespondedExistencePattern("n", "l")}})
.WithColumn("RespondedExistenceCA", #sql{#expr{RespondedExistencePattern("c", "a")}})
.WithColumn("ResponsePatternNL", #sql{#expr{ResponsePattern("n", "l")}})
.WithColumn("ResponsePatternCA", #sql{#expr{ResponsePattern("c", "a")}})
.WithColumn("AlternateResponsePatternNL", #sql{#expr{AlternateResponsePattern("n", "l")}})
.WithColumn("AlternateResponsePatternCA", #sql{#expr{AlternateResponsePattern("c", "a")}})
.WithColumn("ChainResponsePatternNL", #sql{#expr{ChainResponsePattern("n", "l")}})
.WithColumn("ChainResponsePatternCA", #sql{#expr{ChainResponsePattern("c", "a")}})
.WithColumn("PrecedencePatternNL", #sql{#expr{PrecedencePattern("n", "l")}})
.WithColumn("PrecedencePatternCA", #sql{#expr{PrecedencePattern("c", "a")}})
;

[tracesDf.Collect().ToCsv(), eventTypesDf.Collect().ToCsv()]
</syntaxhighlight>

=== Perform a query and send results as E-mail in a HTML table ===
This example requires two scripts that are both located in the same project:

# Expression-type script to execute (can be, e.g., scheduled to run daily).
# Expression-type script containing the query JSON to use as basis for the e-mail. In this example, this script is named as "Send query as E-mail - query JSON". The contents of this script is just the JSON representation of a query that can be extracted, e.g., from any PA chart view.

Script #1 should contain the following code:<syntaxhighlight lang="typescript">
let replyToAddress = "noreply@test.com";
let recipientsArray = ["testuser@test.com"];
let queryConfiguration = ParseJson(Project.ScriptByName("Send query as E-mail - query JSON").Code)
.Set("IncludeCollect", true);
let resultDf = Query(queryConfiguration);
let mailBodyHtml = resultDf.`
<table>
<caption>Example query</caption>
<thead>
<tr>
${StringJoin("", _.Columns.`
<th>${_}</th>
`)}
</tr>
</thead>
<tbody>
${StringJoin("", _.Rows.`
<tr>
${StringJoin("", _.`
<td>${_}</td>
`)}
</tr>
`)}
</tbody>
</table>
`;

SendEmail(#{
"ReplyTo": [replyToAddress],
"To": recipientsArray,
"Subject": "Example query E-mail",
"IsBodyHtml": true,
"Body": mailBodyHtml
});

</syntaxhighlight>NOTE: QPR ProcessAnalyzer Server has to have SMTP server configured. Also, remember to update the values of replyToAddress and recipientsArray before using.

=== Converting a case-centric model to object-centric model ===
This function serves as an example on how a case-centric model could be converted into an object-centric model having just one object type: "Case".

Note: Before use, replace <model id> with a valid model identifier having event data.<syntaxhighlight lang="typescript">
function ConvertCCModelToOCModel(model, newModelName)
{
let connection = model.EventsDataTable.DataSourceConnection;
let caseIdColumn = model.EventsDataTable.ColumnMappings["CaseId"];
let eventToObjectTableName = `${newModelName} - event-to-object`;
let eventsTableName = `${newModelName} - events`;
let objectsTableName = `${newModelName} - objects`;

let eventsDf = model.EventsDataTable
.SqlDataFrame
.RenameColumns([
"OcelEventType": model.EventsDataTable.ColumnMappings["EventType"],
"OcelEventTime": model.EventsDataTable.ColumnMappings["TimeStamp"]])
.WithColumn("OcelEventId", #sql{Concat("evt-", Cast(RowNumber([Column("OcelEventTime")]), "String"))});

eventsDf
.RenameColumns([
"OcelEventToObjectSourceId": "OcelEventId",
"OcelEventToObjectTargetId": caseIdColumn])
.Select(["OcelEventToObjectSourceId", "OcelEventToObjectTargetId"])
.WithColumn("OcelEventToObjectQualifier", #sql{#expr{caseIdColumn} })
.Persist(eventToObjectTableName, #{"Connection": connection});

eventsDf
.RemoveColumns([caseIdColumn])
.Persist(eventsTableName, #{"Connection": connection});

let casesDt = model.CasesDataTable
.SqlDataFrame
.RenameColumns([
"OcelObjectId": model.CasesDataTable.ColumnMappings["CaseId"]
])
.WithColumn("OcelObjectType", #sql{"Case"})
.Persist(objectsTableName, #{"Connection": connection});

let newConfiguration = #{
"OcelDataSource": #{
"Events": eventsTableName,
"Objects": objectsTableName,
"EventToObject": eventToObjectTableName
}
};

model.Project
.CreateModel(#{
"Name": newModelName,
"Description": model.Description,
"Configuration": newConfiguration
});
}

let ccModel = ModelById(<model id>);
ConvertCCModelToOCModel(ccModel, `ocel - ${ccModel.Name}`);
</syntaxhighlight>

Object-centric Process Mining Model

2025-03-25T15:28:23Z

MarHink: /* Object-centric model structure */

QPR ProcessAnalyzer supports object-centric process mining (OCPM) based on the OCEL 2.0 standard (https://www.ocel-standard.org). To use object-centric functionality, you need to transform data into the [[#Object-centric_model_structure|suitable format]] for the [[#Create_object-centric_model|object-centric model]]. Object-centric models can be analyzed in the object-centric flowchart and with (case-centric) charts because the object-centric model can be converted into a case-centric eventlog using [[#Object-centric_perspectives|perspectives]]. To use the OCPM functionality, Snowflake needs to be used as the calculation engine.

== Create object-centric model ==
Create a new object-centric model as follows:
# In the Workspace, open the project where to create the model.
# Select '''NEW''' in the top right menu and select '''model'''.
# Define a name for the new model.
# Set '''Model type''' as '''Object-centric'''.
# Click '''Create'''.

== Configure object-centric model datatables ==
Datatables for the object-centric model need to exist in the same project as the model. Datatables can be set for the model as follows:
# In the Workspace, select the object-centric model and click '''Properties'''.
# In the model properties dialog, open the '''Datasource''' tab.
# Add a following kind of json configuration to the textbox:
<pre>
{
"Objects": "OCPM: objects",
"Events": "OCPM: events",
"ObjectToObject": "OCPM: object-object",
"EventToObject": "OCPM: event-object",
"ObjectTypes": {
"Invoice": "OCPM object: Invoice",
"Payment": "OCPM object: Payment",
"Purchase Order": "OCPM object: Purchase Order"
},
"EventTypes": {
"Approve Purchase Requisition": "OCPM event: Approve Purchase Requisition",
"Change PO Quantity": "OCPM event: Change PO Quantity",
"Create Purchase Order": "OCPM event: Create Purchase Order",
"Insert Invoice": "OCPM event: Insert Invoice",
"Insert Payment": "OCPM event: Insert Payment"
}
}
</pre>

The json configuration needs to have following properties:
* '''Objects''': Objects datatable name.
* '''Events''': Events datatable name.
* '''ObjectToObject''': Object-to-object relation datatable name.
* '''EventToObject''': Event-to-object relation datatable name.
* '''ObjectTypes''': Key-value-pairs of object type datatable names. Note that object names need to match with object names in the objects datatable.
* '''EventTypes''': Key-value-pairs of event type datatable names. Note that event names need to match with event names in the events datatable.

It's also possible that all object attributes and all event attributes are in the same table:
<pre>
{
"Objects": "OCPM: objects",
"Events": "OCPM: events",
"ObjectToObject": "OCPM: object-object",
"EventToObject": "OCPM: event-object",
"ObjectTypes": {
"Invoice": "OCPM object attributes",
"Payment": "OCPM object attributes",
"Purchase Order": "OCPM object attributes"
},
"EventTypes": {
"Approve Purchase Requisition": "OCPM event attributes",
"Change PO Quantity": "OCPM event attributes",
"Create Purchase Order": "OCPM event attributes",
"Insert Invoice": "OCPM event attributes",
"Insert Payment": "OCPM event attributes"
}
}
</pre>

Note also that the Objects table ("OCPM: objects" in the above example) must not contain object types which are not specified in the ObjectTypes section (and same for event types in the EventTypes section).

== Import from OCEL 2.0 JSON file ==
Object-centric model can be import from an OCEL 2.0 JSON file as follows:
# In the Workspace, open the project where to import the model.
# Select '''NEW''' in top right menu and select '''Import Model'''.
# Select the OCEL 2.0 JSON file from the disk and click '''Open'''.

An object-centric model and a list of datatables is created.

Example OCEL 2.0 eventlogs: https://www.ocel-standard.org/event-logs/overview/ (download the json format supported by QPR ProcessAnalyzer)

== Filtering object-centric model ==
Object-centric models can be filtered by object attribute values which is similar to filtering by case attribute values in case-centric models. For the object attribute filtering, following settings are defined:
* Object type to be filtered
* Object attribute name
* Object attribute values
* Include or exclude logic
* Number of object relation steps

When the number of object relation steps is zero, objects of only the selected type are filtered. For example, when including items, all other object types are excluded (when object relation steps is zero). When excluding items, all other object types are included (when object relation steps is zero).

When object relation steps is one, objects directly related to the filtered object with an object-to-object relation are included or excluded.

When object relation steps is two or more, several object-to-object relations are followed to find the included or excluded objects. The objects are traversed only in the same direction which is either forward or backward. This is based on the notion that in object-centric models, all object-to-object relations have a direction, i.e., starting object and ending object of the relation. When creating an object-centric model, this relation direction needs to be carefully selected to produce desired results in the filtering.

Object attribute filter rules can be created for the entire dashboard by pressing the blue plus button in the dashboard header (requires that an object-centric model is selected for the dashboard). Object attribute filter rules can also be added for an individual chart/flowchart by opening the ''Filter'' tab in the chart/flowchart settings and pressing the '''Object-centric filter''' button.

Alternatively, a chart showing object attribute values as dimension or columns, can be selected to start creating object attribute filter rules. When making the selection, user can choose between '''Include Objects''' and '''Exclude Objects'''. It's also possible to use the case-centric filtering by selecting '''Include Cases''' or '''Cases Cases'''. Note that the case-centric attribute filtering doesn't work with the object-centric flowchart and with other charts having different perspective selected.

=== Changes for QPR ProcessAnalyzer 2025.3 ===
Starting from QPR ProcessAnalyzer 2025.3, it's possible to leave the object relation steps setting empty. This means that the object-to-object relations are followed in a way that all the object types are traversed once to find the related objects. This also means that relation direction (forward or backward) doesn't matter anymore.

== Object-centric model structure ==
Object-centric model contains datatables described in the table below. Datatables can be named freely, as the model json configuration is used to define the datatable for each type of data. The datatables need to use column names specified in the table below because those are the column names assumed by the object-centric (i.e., column names cannot be selected freely).

{| class="wikitable"
!'''Datatable role'''
!'''Contained data'''
! '''Datatable columns'''
|-
||Objects
||Objects in the model (one row per object).
||
* '''OcelObjectId''': Unique id for the object (among all objects in the model).
* '''OcelObjectType''': Object type name (such as Order, Invoice, Delivery). Note that the model json configuration need to use same object type names.
|-
||Events
||Events in the model (one row per event).
||
* '''OcelEventId''': Unique id for the event (among all events in the model).
* '''OcelEventType''': Event type name (such as Order created, Invoice sent). Note that the model json configuration need to use same event type names.
* '''OcelEventTime''': Event timestamp.
|-
||Object-object relations
||Relations between objects (one row per relation).
||
* '''OcelObjectToObjectSourceId''': Source object id in the relation.
* '''OcelObjectToObjectTargetId''': Target object id in the relation.
* '''OcelObjectToObjectQualifier''': Describes the type of the relation as free-form text (not used currently).
|-
||Event-object relations
||Relations between events and objects (one row per relation).
||
* '''OcelEventToObjectSourceId''': Event id in the relation.
* '''OcelEventToObjectTargetId''': Object id in the relation.
* '''OcelEventToObjectQualifier''': Describes the type of the relation as free-form text (not used currently).
|-
||Object attributes (several datatables)
||Object attribute values, each object type in a separate table (one row per object).
||
* '''OcelObjectTypeObjectId''': Object id. Matches to the objects datatable ''OcelObjectId'' column.
* '''OcelObjectTypeTime''': Timestamp from which the attribute values are valid.
* '''OcelObjectTypeChangedField''': Changed object attribute name (not used currently).
* '''<Object attributes>''': Columns for each of the object attribute values (column name is the object attribute name).
|-
||Event attributes (several datatables)
||Event attribute values, each event type in a separate table (one row per event).
||
* '''OcelEventTypeEventId''': Event id. Matches to the events datatable ''OcelEventId'' column.
* '''<Event attributes>''': Columns for each of the event attribute values (column name is the event attribute name).
|}

== Object-centric perspectives ==
Perspectives convert an object-centric model into the traditional case-centric eventlog, allowing to view and analyze object-centric models in analyses provided by charts. A single perspective is not able describe the object-centric model entirely, but just from a certain limited viewpoint. By using analyses with several perspectives, it's possible to get a more complete picture of the object-centric model. The perspective starts from a certain object type and traverses the object-object relations as many steps as desired.

To define a perspective, the following settings are defined in the chart settings:
* '''Base Object type''': Object of this type will be cases in the projected case-centric eventlog.
* '''Object Relation Steps''': Specifies how many object-object relations will be traversed in order to find events connected to the base objects. Value zero means that only those events are returned that are directly connected to the base objects.
* '''Show Event Types''': List of event type names which are included into the perspective eventlog. If no events are explicitly defined, all events will be included, but their event attributes are not included.

The resulting perspective eventlog will have the following columns:
* '''OcelObjectId''' (mapped to case id)
* '''OcelEventType''' (mapped to event type)
* '''OcelEventTime''' (mapped to timestamp)
* '''OcelEventId'''
* Object attributes of the base object type. Note that the object attribute values are "repeated" for all events belonging to the same object.
* Event attributes of the selected event types. Values are null for events that don't have the attribute.

The base object type attributes are available as case attributes. As the object attribute values may change over time in the OCEL 2.0 data, the last attribute value is used as the case attribute value. Note that other object type's attributes are not available as case attributes, so the object for which the attributes are used, need to be set as the base object.

== Save perspective to filter ==
It's possible to include the object-centric perspective to a stored filter. When a filter is selected, also the perspective in the filter is applied to the dashboard. This allows to quickly change perspectives for the entire dashboard. The chart specific perspective overrides the dashboard level perspective, so the dashboard level perspective is only applied for charts that don't have the chart specific perspective defined.

Perspective can be added to a filter as follows:
# Go to the ''Process Discovery'' dashboard.
# Open the ''Session variables'' dialog in the dots menu on top right.
# Paste the filter json to the ''Value'' of the ''Filter'' variable (it might be easiest to start with a filter without filter rules, and then add the filter rules using the UI).
# Click ''Done'' button for the dialog.
# Save the filter by hovering the ''Unsaved filter'' (filters dropdown list) in the header and click ''Save as new filter''.

Example: Filter json without any filter rules:
<pre>
{
"Items": [],
"Perspective": {
"ObjectType": "Container",
"RecursionDepth": 0
}
}
</pre>

Example: Filter json with a filter rule:
<pre>
{
"Items": [
{
"Type": "IncludeEvents",
"Items": [
{
"Type": "Attribute",
"Attribute": "OcelEventId",
"StringifiedValues": [ "0Event 1" ]
}
]
}
],
"Perspective": {
"ObjectType": "Container",
"RecursionDepth": 0
}
}
</pre>

== Differences to OCEL 2.0 standard ==
Object-centric models in QPR ProcessAnalyzer are mainly following the OCEL 2.0 standard, but there are the following differences:
* Changing of object attributes values over time is not supported.
* ''ocel_time'' field of each event type table is moved to events datatable (as every event has a timestemp).
* ''*_map_type'' columns are not needed as the model settings are used for the same purpose.
* Object type tables: If OcelObjectTypeChangedField is not null, all the other field values are copied from the previous entry except:
** ''OcelObjectTypeChangedField'' which has the names of the changed fields as a comma separated string.
** The actual changed field which has the new value.
** ''OcelObjectTypeTime'' which has the timestamp when the value changed.

Object-centric Process Mining Model

2025-03-25T14:17:14Z

MarHink: /* Object-centric model structure */

QPR ProcessAnalyzer supports object-centric process mining (OCPM) based on the OCEL 2.0 standard (https://www.ocel-standard.org). To use object-centric functionality, you need to transform data into the [[#Object-centric_model_structure|suitable format]] for the [[#Create_object-centric_model|object-centric model]]. Object-centric models can be analyzed in the object-centric flowchart and with (case-centric) charts because the object-centric model can be converted into a case-centric eventlog using [[#Object-centric_perspectives|perspectives]]. To use the OCPM functionality, Snowflake needs to be used as the calculation engine.

== Create object-centric model ==
Create a new object-centric model as follows:
# In the Workspace, open the project where to create the model.
# Select '''NEW''' in the top right menu and select '''model'''.
# Define a name for the new model.
# Set '''Model type''' as '''Object-centric'''.
# Click '''Create'''.

== Configure object-centric model datatables ==
Datatables for the object-centric model need to exist in the same project as the model. Datatables can be set for the model as follows:
# In the Workspace, select the object-centric model and click '''Properties'''.
# In the model properties dialog, open the '''Datasource''' tab.
# Add a following kind of json configuration to the textbox:
<pre>
{
"Objects": "OCPM: objects",
"Events": "OCPM: events",
"ObjectToObject": "OCPM: object-object",
"EventToObject": "OCPM: event-object",
"ObjectTypes": {
"Invoice": "OCPM object: Invoice",
"Payment": "OCPM object: Payment",
"Purchase Order": "OCPM object: Purchase Order"
},
"EventTypes": {
"Approve Purchase Requisition": "OCPM event: Approve Purchase Requisition",
"Change PO Quantity": "OCPM event: Change PO Quantity",
"Create Purchase Order": "OCPM event: Create Purchase Order",
"Insert Invoice": "OCPM event: Insert Invoice",
"Insert Payment": "OCPM event: Insert Payment"
}
}
</pre>

The json configuration needs to have following properties:
* '''Objects''': Objects datatable name.
* '''Events''': Events datatable name.
* '''ObjectToObject''': Object-to-object relation datatable name.
* '''EventToObject''': Event-to-object relation datatable name.
* '''ObjectTypes''': Key-value-pairs of object type datatable names. Note that object names need to match with object names in the objects datatable.
* '''EventTypes''': Key-value-pairs of event type datatable names. Note that event names need to match with event names in the events datatable.

It's also possible that all object attributes and all event attributes are in the same table:
<pre>
{
"Objects": "OCPM: objects",
"Events": "OCPM: events",
"ObjectToObject": "OCPM: object-object",
"EventToObject": "OCPM: event-object",
"ObjectTypes": {
"Invoice": "OCPM object attributes",
"Payment": "OCPM object attributes",
"Purchase Order": "OCPM object attributes"
},
"EventTypes": {
"Approve Purchase Requisition": "OCPM event attributes",
"Change PO Quantity": "OCPM event attributes",
"Create Purchase Order": "OCPM event attributes",
"Insert Invoice": "OCPM event attributes",
"Insert Payment": "OCPM event attributes"
}
}
</pre>

Note also that the Objects table ("OCPM: objects" in the above example) must not contain object types which are not specified in the ObjectTypes section (and same for event types in the EventTypes section).

== Import from OCEL 2.0 JSON file ==
Object-centric model can be import from an OCEL 2.0 JSON file as follows:
# In the Workspace, open the project where to import the model.
# Select '''NEW''' in top right menu and select '''Import Model'''.
# Select the OCEL 2.0 JSON file from the disk and click '''Open'''.

An object-centric model and a list of datatables is created.

Example OCEL 2.0 eventlogs: https://www.ocel-standard.org/event-logs/overview/ (download the json format supported by QPR ProcessAnalyzer)

== Filtering object-centric model ==
Object-centric models can be filtered by object attribute values which is similar to filtering by case attribute values in case-centric models. For the object attribute filtering, following settings are defined:
* Object type to be filtered
* Object attribute name
* Object attribute values
* Include or exclude logic
* Number of object relation steps

When the number of object relation steps is zero, objects of only the selected type are filtered. For example, when including items, all other object types are excluded (when object relation steps is zero). When excluding items, all other object types are included (when object relation steps is zero).

When object relation steps is one, objects directly related to the filtered object with an object-to-object relation are included or excluded.

When object relation steps is two or more, several object-to-object relations are followed to find the included or excluded objects. The objects are traversed only in the same direction which is either forward or backward. This is based on the notion that in object-centric models, all object-to-object relations have a direction, i.e., starting object and ending object of the relation. When creating an object-centric model, this relation direction needs to be carefully selected to produce desired results in the filtering.

Object attribute filter rules can be created for the entire dashboard by pressing the blue plus button in the dashboard header (requires that an object-centric model is selected for the dashboard). Object attribute filter rules can also be added for an individual chart/flowchart by opening the ''Filter'' tab in the chart/flowchart settings and pressing the '''Object-centric filter''' button.

Alternatively, a chart showing object attribute values as dimension or columns, can be selected to start creating object attribute filter rules. When making the selection, user can choose between '''Include Objects''' and '''Exclude Objects'''. It's also possible to use the case-centric filtering by selecting '''Include Cases''' or '''Cases Cases'''. Note that the case-centric attribute filtering doesn't work with the object-centric flowchart and with other charts having different perspective selected.

=== Changes for QPR ProcessAnalyzer 2025.3 ===
Starting from QPR ProcessAnalyzer 2025.3, it's possible to leave the object relation steps setting empty. This means that the object-to-object relations are followed in a way that all the object types are traversed once to find the related objects. This also means that relation direction (forward or backward) doesn't matter anymore.

== Object-centric model structure ==
Object-centric model contains datatables described in the table below. Datatables can be named freely, as the model json configuration is used to define the datatable for each type of data. The datatables need to use column names specified in the table below because those are the column names assumed by the object-centric (i.e., column names cannot be selected freely).

{| class="wikitable"
!'''Datatable role'''
!'''Contained data'''
! '''Datatable columns'''
|-
||Objects
||Objects in the model (one row per object).
||
* '''OcelObjectId''': Unique id for the object (among all objects in the model).
* '''OcelObjectType''': Object type name (such as Order, Invoice, Delivery). Note that the model json configuration need to use same object type names.
|-
||Events
||Events in the model (one row per event).
||
* '''OcelEventId''': Unique id for the event (among all events in the model).
* '''OcelEventType''': Event type name (such as Order created, Invoice sent). Note that the model json configuration need to use same event type names.
* '''OcelEventTime''': Event timestamp.
|-
||Object-object relations
||Relations between objects (one row per relation).
||
* '''OcelObjectToObjectSourceId''': Source object id in the relation.
* '''OcelObjectToObjectTargetId''': Target object id in the relation.
* '''OcelObjectToObjectQualifier''': Describes the type of the relation as free-form text (not used currently).
|-
||Event-object relations
||Relations between events and objects (one row per relation).
||
* '''OcelEventToObjectSourceId''': Event id in the relation.
* '''OcelEventToObjectTargetId''': Object id in the relation.
* '''OcelEventToObjectQualifier''': Describes the type of the relation as free-form text (not used currently).
|-
||Object attributes (several datatables)
||Object attribute values, each object type in a separate table (one row per object).
||
* '''OcelObjectTypeObjectId''': Object id. Matches to the objects datatable ''OcelObjectId'' column.
* '''OcelObjectTypeTime''': Timestamp which the attribute value is valid from (not used currently).
* '''OcelObjectTypeChangedField''': Changed object attribute name (not used currently).
* '''<Object attributes>''': Columns for each of the object attribute values (column name is the object attribute name).
|-
||Event attributes (several datatables)
||Event attribute values, each event type in a separate table (one row per event).
||
* '''OcelEventTypeEventId''': Event id. Matches to the events datatable ''OcelEventId'' column.
* '''<Event attributes>''': Columns for each of the event attribute values (column name is the event attribute name).
|}

== Object-centric perspectives ==
Perspectives convert an object-centric model into the traditional case-centric eventlog, allowing to view and analyze object-centric models in analyses provided by charts. A single perspective is not able describe the object-centric model entirely, but just from a certain limited viewpoint. By using analyses with several perspectives, it's possible to get a more complete picture of the object-centric model. The perspective starts from a certain object type and traverses the object-object relations as many steps as desired.

To define a perspective, the following settings are defined in the chart settings:
* '''Base Object type''': Object of this type will be cases in the projected case-centric eventlog.
* '''Object Relation Steps''': Specifies how many object-object relations will be traversed in order to find events connected to the base objects. Value zero means that only those events are returned that are directly connected to the base objects.
* '''Show Event Types''': List of event type names which are included into the perspective eventlog. If no events are explicitly defined, all events will be included, but their event attributes are not included.

The resulting perspective eventlog will have the following columns:
* '''OcelObjectId''' (mapped to case id)
* '''OcelEventType''' (mapped to event type)
* '''OcelEventTime''' (mapped to timestamp)
* '''OcelEventId'''
* Object attributes of the base object type. Note that the object attribute values are "repeated" for all events belonging to the same object.
* Event attributes of the selected event types. Values are null for events that don't have the attribute.

The base object type attributes are available as case attributes. As the object attribute values may change over time in the OCEL 2.0 data, the last attribute value is used as the case attribute value. Note that other object type's attributes are not available as case attributes, so the object for which the attributes are used, need to be set as the base object.

== Save perspective to filter ==
It's possible to include the object-centric perspective to a stored filter. When a filter is selected, also the perspective in the filter is applied to the dashboard. This allows to quickly change perspectives for the entire dashboard. The chart specific perspective overrides the dashboard level perspective, so the dashboard level perspective is only applied for charts that don't have the chart specific perspective defined.

Perspective can be added to a filter as follows:
# Go to the ''Process Discovery'' dashboard.
# Open the ''Session variables'' dialog in the dots menu on top right.
# Paste the filter json to the ''Value'' of the ''Filter'' variable (it might be easiest to start with a filter without filter rules, and then add the filter rules using the UI).
# Click ''Done'' button for the dialog.
# Save the filter by hovering the ''Unsaved filter'' (filters dropdown list) in the header and click ''Save as new filter''.

Example: Filter json without any filter rules:
<pre>
{
"Items": [],
"Perspective": {
"ObjectType": "Container",
"RecursionDepth": 0
}
}
</pre>

Example: Filter json with a filter rule:
<pre>
{
"Items": [
{
"Type": "IncludeEvents",
"Items": [
{
"Type": "Attribute",
"Attribute": "OcelEventId",
"StringifiedValues": [ "0Event 1" ]
}
]
}
],
"Perspective": {
"ObjectType": "Container",
"RecursionDepth": 0
}
}
</pre>

== Differences to OCEL 2.0 standard ==
Object-centric models in QPR ProcessAnalyzer are mainly following the OCEL 2.0 standard, but there are the following differences:
* Changing of object attributes values over time is not supported.
* ''ocel_time'' field of each event type table is moved to events datatable (as every event has a timestemp).
* ''*_map_type'' columns are not needed as the model settings are used for the same purpose.
* Object type tables: If OcelObjectTypeChangedField is not null, all the other field values are copied from the previous entry except:
** ''OcelObjectTypeChangedField'' which has the names of the changed fields as a comma separated string.
** The actual changed field which has the new value.
** ''OcelObjectTypeTime'' which has the timestamp when the value changed.

Expression Script Examples

2025-03-19T14:13:25Z

MarHink: /* Perform a query and send results as E-mail in a HTML table */

This page contains script examples written in the QPR ProcessAnalyzer expression language. See how expression scripts can be created in the [[Managing_Scripts#Creating_Script|Workspace]]. For documentation for the syntax, functions and entities can be found from the main page in the [[QPR_ProcessAnalyzer_Wiki#For_Developers|KPI Expression Language]] section.

== Calling Expression Script from Expression ==
Expression scripts can be called from an expression using the [[QPR_ProcessAnalyzer_Objects_in_Expression_Language#Script|Run]] function with the following syntax:
<pre>
let result = ScriptById(123).Run(#{
"parameter1": "value1",
"parameter2": false,
"parameter2": 123.45
})
</pre>

The command waits until the run is completed, and the return value of the called script is returned by the Run function call.

Parameters can be passed to the called script, and the parameters are available as variables in the script. The parameters can contain any type of data.

Expression scripts can also be called from a dashboard. Expressions can be stored to scripts instead of dashboards, which is a way to separate complex expressions from dashboards and allow to reuse expressions across several dashboards.

== Calling SQL Script from Expression ==
SQL script can be called from an expression using the Run function as follows (similar to calling [[#Calling Expression Script from Expression|expression scripts]]):
<pre>
let result = ScriptById(123).Run(#{
"parameter1": "value1",
"parameter2": 321
});
let arrayOfAllReports = result.Keys;
let report1 = result.Report1;
let report2 = result.Report2;
</pre>

SQL scripts can return multiple ''reports'', which are combined to a dictionary, where the key is the name of the report ("sheet name") and value is the report data as a DataFrame. See in the above example, how the reports can be accessed by their name.

== Examples ==
=== Call web service===
Contact to a web service, fetch some data, and store it to a datatable.

<syntaxhighlight lang="typescript" line>
let datatableName = "Web Service Data";
let webServiceData = CallWebService(
#{"Address": "https://processanalyzer.onqpr.com/qprpa/api/serverinfo"}
);

let targetDatatable = Project.Datatables.Where(name==datatableName);
if (Count(targetDatatable) == 0) {
targetDatatable = Project.CreateDatatable(datatableName)
.AddColumn("Setting name", "String")
.AddColumn("Setting value", "String")
.AddColumn("Data read", "DateTime");
} else {
targetDatatable = targetDatatable[0];
}

let currentTime = Now;
let dataAsDf = ToDataFrame(
webServiceData.keys.{
let key = _;
[key, webServiceData[key], currentTime];
},
["Setting name", "Setting value", "Data read"]
);
targetDatatable.Import(dataAsDf, #{"Append":true});
WriteLog(`${CountTop(dataAsDf.Rows)} rows written to datatable`);
</syntaxhighlight>

=== Store data to datatable ===
Get all models in the system and store them to a datatable.

<syntaxhighlight lang="typescript" line>
let newDatatable = Project
.CreateDatatable("Models list " + ToString(Now, "dd.MM.yyyy HH:mm:ss"))
.AddColumn("Model name", "String")
.AddColumn("Project name", "String")
.AddColumn("Created time", "DateTime")
.AddColumn("Cases", "Integer");
let startTime = Now;
let modelsData = ToDataFrame(
Models.([Name, Project.Name, CreatedDate, NCases]),
["Model name", "Project name", "Created time", "Cases"]
);
WriteLog(`Listing models took ${(Now - startTime).TotalSeconds.Round(2)} seconds.`);
newDatatable.Import(modelsData);
WriteLog(`Datatable ${newDatatable.Id} created.`);
</syntaxhighlight>

=== Convert datatable column data ===
This script can be used to convert a single column into numerical data type. To use the script, you need to setup the following in the beginning of the script:
* Project name where the datatable is located.
* Datatable name
* Name of the column to be converted

Note that the conversion fails, if there is data that cannot be converted into numerical format. The conversion assumes that period (.) is used as the decimal point.

<syntaxhighlight lang="typescript" line>
let projectName = "New Project";
let datatableName = "qpr processanalyzer events";
let columnName = "Event order in case";

let project = (Projects.Where(Name==projectName))[0];
let datatable = (project.Datatables.Where(Name==datatableName))[0];
DatatableById(datatable.Id).DataFrame
.SetColumns([
columnName: () => {
let data = Column(columnName);
if (data == null) {
null;
} else {
ToFloat(data);
}
}
])
.Persist(datatable.Name, ["ProjectId": project.Id, "Append": false]);
</syntaxhighlight>

Instead of converting to numeric (with the ''ToFloat'' function), data can be converted into string using the ''ToString'' function.

=== Show DataFrame as HTML table ===

This script defines a function to show dataframe as a HTML table, and uses the function for a literal dataframe.

<syntaxhighlight lang="typescript" line>
function dataframeToHtmlTable(df) {
return
`<table>
<tr>
${StringJoin("\r\n\t\t", + df.columns.`<th>${_}</th>`)}
</tr>
${StringJoin("", df.Rows.(
"\r\n\t<tr>" + StringJoin("", _.`\r\n\t\t<td>${ToString(_)}</td>`) + "\r\n\t</tr>"
))}
</table>`
}

let data = ToDataFrame(
[
["one", "two", "three"],
["four", "five", "six"],
["seven", "eight", "nine"]
],
["Column 1", "Column 2", "Column 3"]
);

return dataframeToHtmlTable(data);
</syntaxhighlight>

=== Copy local datatables to Snowflake ===

<syntaxhighlight lang="typescript" line>
// Copies all datatables in a project to another project including datatable contents.
// Usage instructions:
// 1. Create expression script in the project from where you want to copy the datatables.
// 2. Create a new project named as "<name of the project to be moved> - Snowflake". New datatables will be created here. E.g., when moving project named "SAP_OrderToCash", the target project should be named as "SAP_OrderToCash - Snowflake".
// 3. Run the script.
// NOTE: Columns of type "Any" will be created as "String"-columns in Snowflake, thus it is recommended that actual data types are set for the tables prior to the move.

let sourceProject = Project;
let sourceProjectName = Project.Name;
let targetProjectName = `${sourceProjectName} - Snowflake`;
let targetProject = First(Projects.Where(Name == targetProjectName));
if (IsNull(targetProject)) {
WriteLog(`Unable to find target project named "${targetProjectName}". Aborting operation.`);
return;
}
let dts = sourceProject.DataTables;
WriteLog(`Copying all ${CountTop(dts)} data tables found in project "${sourceProject.Name}" (id: ${sourceProject.Id}) to Snowflake in project "${targetProject.Name}" (id: ${targetProject.Id})`);
dts.{
let sourceDt = _;
WriteLog(`Starting to copy data table "${Name}" (id: ${Id}) having ${NRows} rows and ${NColumns} columns.`);
let targetDt;
targetDt = targetProject.DatatableByName(sourceDt.Name);
if (targetDt == null) {
targetDt = targetProject.CreateDataTable(sourceDt.Name, #{"Connection": CreateSnowflakeConnection(#{"ProjectId": targetProject.Id})});
targetDt.Import(sourceDt.SqlDataFrame);
WriteLog(`Finished copying data table "${Name}" (id: ${Id}) to table "${targetDt.Name}" (id: ${targetDt.Id})`);
} else {
WriteLog(`Datatable already exist "${Name}" (id: ${Id}) to table "${targetDt.Name}" (id: ${targetDt.Id})`);
}
}
WriteLog(`Finished copying all the data tables found in project "${sourceProject.Name}" (id: ${sourceProject.Id}) to Snowflake in project "${targetProject.Name}" (id: ${targetProject.Id})`);
</syntaxhighlight>

If you don't need to copy the data but only create the Snowflake datatables with columns, you can change the line 22 to
<syntaxhighlight>
targetDt.Import(sourceDt.SqlDataFrame.head(0));
</syntaxhighlight>

=== Copy single datatable to Snowflake ===
This script creates a copy of a single datatable to Snowflake. Replace the ''<tableId1>'' with the id of the source datatable.
<syntaxhighlight lang="typescript" line>
function CopyDataTableToSnowflake(dataTableId)
{
let sourceDt = DataTableById(dataTableId);
sourceDt.SqlDataFrame.Persist(`${sourceDt.Name} - Snowflake`, #{"Append": false, "Connection": CreateSnowflakeConnection(#{"ProjectId": sourceDt.Project.Id})});
}
CopyDataTableToSnowflake(<tableId1>);
</syntaxhighlight>

=== Create a copy of a data table that has all Any-type columns changed to String-type columns ===
<syntaxhighlight lang="typescript" line>
function ConvertAnyDataTypesToStringsToNewTable(dataTableId)
{
let dt = DataTableById(dataTableId);
let sdf = dt.SqlDataFrame;
let cts = dt.ColumnTypes;
cts.{
let ct = _;
if (ct.DataType == "Any") {
let n = ct.Name;
sdf = sdf.WithColumn(ct.Name, #sql{Cast(Column(Variable("n")), "ShortString")});
}
};
sdf.Persist(`${dt.Name} - Converted`, #{"Append": false, "ProjectId": dt.Project.Id});
}
ConvertAnyDataTypesToStringsToNewTable(<dataTableId>);
</syntaxhighlight>

=== Query number of rows in given data table having a datetime value in given year grouped by month and return resulting table as CSV ===
SqlDataFrame is used in order to prevent loading the whole datatable into memory first. Filtering is performed as first operation in order to minimize the amount of required work for the data source of the data table.<syntaxhighlight lang="typescript" line="1">
DataTableById(<data table id>)
.SqlDataFrame
.Where(#sql{2014 == Year(Column("Start Time"))})
.WithColumn("Month", #sql{Month(Column("Start Time"))})
.GroupBy(["Month"]).Aggregate(["Count"], ["Count"])
.OrderByColumns(["Month"], [true])
.Collect().ToCsv();
</syntaxhighlight>

=== Function for filtering SqlDataFrame by removing rows having, or replacing, the most infrequently occurring column values ===
<syntaxhighlight lang="typescript" line="1">
/***
* @name ColumnWithMinUsage
* @descripion
* Generic function that can be used to filter out the most infrequently occurring attribute values or replace their Values
* with given common value.
* @param df:
* DataFrame to operate on.
* @param columnName:
* Name of the column to be filtered.
* @param newColumnName:
* Name of the column that will contain the new value of the original column after filtering (if includeOthers was applied).
* @param maxNumUniqueValues:
* Maximum number of unique values to include into the comparison for each attribute column. If the amount of unique values for any attribute exceeds this value, only given number of attributes are included that have the highest usage.
* @param minValueUsage:
* Minimum total usage of a value included into the comparison. The number of cases having every returned value should be at least given percentage (a float value between 0.0 and 1.0) of all the compared cases.
* @param includeOthers:
* Should the rest of the attribute values not included due to MinValueUsage or MaxNumUniqueValues filtering be included as an aggregated "Others" value?
* If not empty/null, defines the name used for these other-values.
*/
function ColumnWithMinUsage(df, columnName, newColumnName, maxNumUniqueValues, minValueUsage, includeOthers)
{
let all = df
.GroupBy([])
.Aggregate(["NAllTotal"], ["Count"])
.WithColumn("__Join2", #sql{1});
let minValueUsageEnabled = !IsNullTop(minValueUsage);
let maxNumUniqueValuesEnabled = !IsNullTop(maxNumUniqueValues);
if (minValueUsageEnabled || maxNumUniqueValuesEnabled) {
// Perform column value-based filtering if minValueUsageEnabled or maxNumUniqueValuesEnabled is defined.
let valueColumnName = "__ValueNew";
let filteredValuesColumns = [valueColumnName: columnName];
let filteredValues = df
.GroupBy([columnName]).Aggregate(["Count"], ["Count"]);
if (minValueUsageEnabled) {
filteredValues = filteredValues
.WithColumn("__Join", #sql{1})
.Join(all, ["__Join": "__Join2"], "leftouter")
.WithColumn("Usage", #sql{Column("Count") / Column("NAllTotal")});
filteredValuesColumns = Concat(filteredValuesColumns, ["Usage"]);
}
if (maxNumUniqueValuesEnabled) {
filteredValues = filteredValues
.WithRowNumberColumn("RowNumber", ["Count"], null, [false]);
filteredValuesColumns = Concat(filteredValuesColumns, ["RowNumber"]);
}

filteredValues = filteredValues
.Select(filteredValuesColumns);

// Generate select returning all the accepted values.
let allValues = filteredValues
.(minValueUsageEnabled ? Where(#sql{Column("Usage") >= #expr{minValueUsage}}) : _)
.(maxNumUniqueValuesEnabled ? Where(#sql{Column("RowNumber") <= #expr{maxNumUniqueValues}}) : _)
.Select([valueColumnName, newColumnName: valueColumnName]);

if (!IsNullTop(includeOthers)) {
// If includeOthers is defined, replace original values with the variable defined in includeOthers.
let otherValues = filteredValues
.(minValueUsageEnabled ? Where(#sql{Column("Usage") < #expr{minValueUsage}}) : _)
.(maxNumUniqueValuesEnabled ? Where(#sql{Column("RowNumber") > #expr{maxNumUniqueValues}}) : _)
.WithColumn(newColumnName, #sql{#expr{includeOthers}})
.Select([valueColumnName, newColumnName]);
allValues = allValues.Append(otherValues)
}
df.Join(allValues, [columnName: valueColumnName], "inner")
.RemoveColumns([valueColumnName]);
}
}

// The following example will return only rows containing two of the most common values for Region-column.
//let df = DataTableById(<data table id>).SqlDataFrame;
//df = ColumnWithMinUsage(df, "Region", "_Filtered", 2, null, null);
//df.Collect().ToCsv();

// The following example will return all input rows, but will replace the values of rows whose Region-column
// has a value used by less than 15% of all the rows with a new value: "_Others".
//let df = DataTableById(<data table id>).SqlDataFrame;
//df = ColumnWithMinUsage(df, "Region", "_Filtered", null, 0.15, "_Others");
//df.Collect().ToCsv();
</syntaxhighlight>

=== Export model events and cases ===
<syntaxhighlight lang="typescript" line="1">
function ExportModelEvents(m) {
let attrs = m.EventAttributes;
ToDataFrame(
m.EventLog.Events.Concat(
[Case.Name, Type.Name, ToString(TimeStamp, "yyyy-MM-dd HH:mm:ss.fff")],
{let evt = _; attrs.{let att = _; evt.Attribute(att)}}
),
Concat(
["CaseId", "EventType", "TimeStamp"],
attrs.Name
)
).ToCsv(true);
}

function ExportModelCases(m) {
let attrs = m.CaseAttributes;
ToDataFrame(
m.EventLog.Cases.Concat(
[Name],
{let cas = _; attrs.{let att = _; cas.Attribute(att)}}
),
Concat(
["CaseId"],
attrs.Name
)
).ToCsv(true);
}
</pre>

<pre>
First(Models.Where(Name=="SAP OtC Extended")).EventsDataTable.DataFrame.ToCsv(true)
</pre>

<pre>
First(Models.Where(Name=="SAP OtC Extended")).CasesDataTable.DataFrame.ToCsv(true)
</syntaxhighlight>

=== Calculate all the value usages of a single column for each event in event data table ===
This query could be used, e.g., to find out the maximum resource usage for every resource found in the event data table.<syntaxhighlight lang="typescript" line="1">
function WithUsageColumns(resourceColumn)
{
function WithTotalUsageColumnOfSingleResource(resourceColumn, resourceValue)
{
_
.WithColumn("_Prev", #sql{Lag(Column(resourceColumn), [TimeStamp, EventType], [true, true], [CaseId], 1, null)})
.WithColumn("_UsageDiff", #sql{
CaseWhen(
Column(resourceColumn) == Column("_Prev"), 0,
Column("_Prev") == #expr{resourceValue}, -1,
Column(resourceColumn) == #expr{resourceValue}, 1,
0)
})
.WithColumn(`${resourceValue}_Usage`, #sql{Sum(Column("_UsageDiff"), [TimeStamp, EventType])})
.RemoveColumns(["_Prev", "_UsageDiff"])
}

let sdf = _;
let allValues = sdf.SelectDistinct([resourceColumn]).OrderByColumns([resourceColumn], [true]).Collect().Column(resourceColumn);
allValues.{
let v = _;
sdf = sdf.WithTotalUsageColumnOfSingleResource(resourceColumn, v)
}
sdf
}

let dt = ModelById(<model id>).EventsDataTable;
dt
.SqlDataFrame
.WithUsageColumns(<resource column name>)
.OrderByColumns([dt.ColumnMappings["TimeStamp"]], [true])
.Collect().ToCsv()
</syntaxhighlight>Where:

* <model id> is the id of the model containing event data to be examined.
* <resource column name> is the name of the column in the event data table of the specified model containing the resource being used by that event.

NOTE: This expression uses functionalities that are only supported in Snowflake-based data tables.

=== Create new Snowflake model from filter ===
This script creates a new Snowflake model (and two datatables for cases and events) containing filtered event log from given filter id. The script also works if the model doesn't have a cases datatable.

<syntaxhighlight lang="typescript" line="1">
let filter = FilterById(1); // filter id
let model = filter.model;
let project = model.project;
let nameSuffix = " - " + filter.name + " - " + ToString(Now, "dd-MM-yyyy HH:mm:ss");
let eventsDatatableName = model.EventsDataTable.Name + nameSuffix;
if (eventsDatatableName.length > 440) {
eventsDatatableName = eventsDatatableName.Substring(eventsDatatableName.length - 440);
}
let eventsData = model
.EventsDataTable
.SqlDataFrame
.ApplyFilter(
filter.rules,
model.CasesDataTable?.SqlDataFrame
);
project
.CreateDatatable(eventsDatatableName, #{"Connection": CreateSnowflakeConnection()})
.Import(eventsData);
let modelConfiguration = model.Configuration;
modelConfiguration.DataSource.Events.Set("DataTableName", eventsDatatableName);
if (model.CasesDataTable != null) {
let eventsDataCaseIdColumn = "CaseId_" + ToString(Random());
let casesDatatableName = model.CasesDataTable.Name + nameSuffix;
if (casesDatatableName.length > 440) {
casesDatatableName = casesDatatableName.Substring(casesDatatableName.length - 440);
}
let casesData = model
.CasesDataTable
.SqlDataFrame
.join(
eventsData.SelectDistinct([eventsDataCaseIdColumn: modelConfiguration.DataSource.Events.Columns.CaseId]),
[modelConfiguration.DataSource.Cases.Columns.CaseId: eventsDataCaseIdColumn]
).Select(model.CasesDataTable.ColumnNames);
project
.CreateDatatable(casesDatatableName, #{"Connection": CreateSnowflakeConnection()})
.Import(casesData);
modelConfiguration.DataSource.Cases.Set("DataTableName", casesDatatableName);
}
let modelName = model.Name + nameSuffix;
if (modelName > 440) {
modelName = modelName.Substring(modelName.length - 440);
}
project
.CreateModel(#{
"Name": modelName,
"Description": model.Description,
"Configuration": modelConfiguration
});
return modelName;
</syntaxhighlight>

=== Creating a model consisting of multiple copies of cases in an existing model ===
<syntaxhighlight lang="typescript" line="1">
/**
* @name CreateTestModel
* @description
* Creates a new model (or overwrites an existing) to given target project with given number of
* repetitions of given source model.
* Each repetition will generate "<N>-"-prefix to CaseId-columns, where N equals to the repeat index.
* @param sourceModel
* PA model used for the source data and from where the connection is copied for the target model if a
* new one has to be created.
* @param numRepeats
* Number of times the data in the source model should be repeated in the generated model.
* @param targetProject
* Project in which the target model resides.
* @param targetModelName
* Specifies the name of the test model in the given target project. If a model already exists with
* given name, event and case data in this model will be replaced with the new generated event and
* case data.
* @returns
* Model object of the test model having the newly generated data.
*/
function CreateTestModel(sourceModel, numRepeats, targetProject, targetModelName)
{
let eventsColumnMappings = sourceModel.EventsDataTable.ColumnMappings;
let casesColumnMappings = sourceModel.CasesDataTable.ColumnMappings;
let connection = sourceModel.EventsDataTable.DataSourceConnection;

function CreateResultModel()
{
function GetTable(tableName)
{
let tableConfiguration = #{
"Name": tableName,
"Connection": connection
};
let resultTable = targetProject.DataTableByName(tableName);
if (resultTable == null)
{
resultTable = targetProject.CreateDataTable(tableConfiguration)
.Modify(#{"NameInDataSource": null})
.Synchronize();
}
return resultTable;
}

let eventsTableName = `${targetModelName} - events`;
let casesTableName = `${targetModelName} - cases`;
let targetModel = targetProject.ModelByName(targetModelName);
let eventsTable, casesTable = null;

if (targetModel != null)
{
eventsTable = targetModel.EventsDataTable;
casesTable = targetModel.CasesDataTable;
}
else {
eventsTable = GetTable(eventsTableName);
if (sourceModel.CasesDataTable != null) {
casesTable = GetTable(casesTableName);
}

let timestampMapping = eventsColumnMappings["TimeStamp"];
eventsColumnMappings.Remove("TimeStamp");
eventsColumnMappings.Set("Timestamp", timestampMapping);

let modelConfiguration = #{
"DataSource": #{
"Events":#{
"DataSourceType": "datatable",
"DataTableName": eventsTableName,
"Columns": eventsColumnMappings
}
}
};

if (casesColumnMappings != null) {
modelConfiguration["DataSource"].Set("Cases", #{
"DataSourceType": "datatable",
"DataTableName": casesTableName,
"Columns": casesColumnMappings
});
}

targetModel = targetProject.CreateModel(#{"Name": targetModelName, "Configuration": modelConfiguration});
}

eventsTable.Truncate();
casesTable?.Truncate();

return #{
"TargetModel": targetModel,
"Events": eventsTable,
"Cases": casesTable
};
}

function RepeatNTimes(sourceDf, caseIdColumn, numRepeats)
{
let resultDf = null;
for (let i = 1; i <= numRepeats; ++i) {
let iterationDf = sourceDf
.WithColumn(caseIdColumn, #sql{Concat(#expr{i}, "-", Column(#expr{caseIdColumn}))});
resultDf = resultDf == null ? iterationDf : resultDf.Append(iterationDf);
}
resultDf;
}

let resultModel = CreateResultModel();
let sourceEventDataDf = sourceModel.EventsDataTable.SqlDataFrame;
let resultEventDataDf = RepeatNTimes(sourceEventDataDf, eventsColumnMappings["CaseId"], numRepeats);
resultModel["Events"].Import(resultEventDataDf);

let sourceCaseDataDf = sourceModel.CasesDataTable?.SqlDataFrame;
if (sourceCaseDataDf != null) {
let resultCaseDataDf = RepeatNTimes(sourceCaseDataDf, casesColumnMappings["CaseId"], numRepeats);
resultModel["Cases"].Import(resultCaseDataDf);
}
resultModel["TargetModel"];
}

</syntaxhighlight>Example usage:<blockquote>CreateTestModel(ProjectByName("Project").ModelByName("SAP_OrderToCash - Snowflake"), 3, ProjectByName("TestData"), "TestModel");</blockquote>Creates a new model named "TestModel" (or overwrites old one) into project named "TestData" containing the data from model "SAP_OrderToCash - Snowflake" in project "Project" repeated three times.

=== Analyzing declare patterns found in event log ===

This is an example expression that shows how POSIX-style regular expressions can be used to search for cases in an event log having certain event type patterns [https://www.researchgate.net/publication/277631859_Generating_Event_Logs_Through_the_Simulation_of_Declare_Models declare patterns].

Note: Before use, replace <model id> with a valid model identifier having event data.

<syntaxhighlight lang="typescript" line="1">
let dt = ModelById(<model id>).EventsDataTable;
let sdf = dt.SqlDataFrame.Head(1000);
let caseIdColumn = dt.ColumnMappings["CaseId"], timeStampColumn = dt.ColumnMappings["TimeStamp"], eventTypeColumn = dt.ColumnMappings["EventType"];

let eventTypesDf = sdf
.SelectDistinct([eventTypeColumn])
.OrderByColumns([eventTypeColumn], [true])
.WithRowNumberColumn("Token", [eventTypeColumn])
.WithColumn("Token", #sql{Char(Column("Token") + Unicode("a") - 1)});

sdf = sdf
.Join(eventTypesDf.Select(["_EventType2": eventTypeColumn, "Token"]), [eventTypeColumn: "_EventType2"], "leftouter");

function RespondedExistencePattern(a, b)
{
// [^a]*((a.*b.*)|(b.*a.*))*[^a]*
ToSqlExpression(`Regexp(Column("Trace"), "[^${a}]*((${a}.*${b}.*)|(${b}.*${a}.*))*[^${a}]")`)
}

function ResponsePattern(a, b)
{
// [^a]*(a.*b)*[^a]*
ToSqlExpression(`Regexp(Column("Trace"), "[^${a}]*(${a}.*${b})*[^${a}]")`)
}

function AlternateResponsePattern(a, b)
{
// [^a]*(a[^a]*b[^a]*)*[^a]*
ToSqlExpression(`Regexp(Column("Trace"), "[^${a}]*(${a}[^${a}]*${b}[^${a}]*)*[^${a}]")`)
}

function ChainResponsePattern(a, b)
{
// [^a]*(ab[^a]*)*[^a]*
ToSqlExpression(`Regexp(Column("Trace"), "[^${a}]*(${a}${b}[^${a}]*)*[^${a}]")`)
}

function PrecedencePattern(a, b)
{
// [^b]*(a.*b)*[^b]*
ToSqlExpression(`Regexp(Column("Trace"), "[^${b}]*(${a}.*${b})*[^${b}]")`)
}

let tracesDf = sdf
.GroupBy([caseIdColumn])
.Aggregate(["Trace": "Token"], [#{"Function": "list", "Ordering": [timeStampColumn], "Separator": ""}])
.WithColumn("RespondedExistenceNL", #sql{#expr{RespondedExistencePattern("n", "l")}})
.WithColumn("RespondedExistenceCA", #sql{#expr{RespondedExistencePattern("c", "a")}})
.WithColumn("ResponsePatternNL", #sql{#expr{ResponsePattern("n", "l")}})
.WithColumn("ResponsePatternCA", #sql{#expr{ResponsePattern("c", "a")}})
.WithColumn("AlternateResponsePatternNL", #sql{#expr{AlternateResponsePattern("n", "l")}})
.WithColumn("AlternateResponsePatternCA", #sql{#expr{AlternateResponsePattern("c", "a")}})
.WithColumn("ChainResponsePatternNL", #sql{#expr{ChainResponsePattern("n", "l")}})
.WithColumn("ChainResponsePatternCA", #sql{#expr{ChainResponsePattern("c", "a")}})
.WithColumn("PrecedencePatternNL", #sql{#expr{PrecedencePattern("n", "l")}})
.WithColumn("PrecedencePatternCA", #sql{#expr{PrecedencePattern("c", "a")}})
;

[tracesDf.Collect().ToCsv(), eventTypesDf.Collect().ToCsv()]
</syntaxhighlight>

=== Perform a query and send results as E-mail in a HTML table ===
This example requires two scripts that are both located in the same project:

# Expression-type script to execute (can be, e.g., scheduled to run daily).
# Expression-type script containing the query JSON to use as basis for the e-mail. In this example, this script is named as "Send query as E-mail - query JSON". The contents of this script is just the JSON representation of a query that can be extracted, e.g., from any PA chart view.

Script #1 should contain the following code:<syntaxhighlight lang="typescript">
let replyToAddress = "noreply@test.com";
let recipientsArray = ["testuser@test.com"];
let queryConfiguration = ParseJson(Project.ScriptByName("Send query as E-mail - query JSON").Code)
.Set("IncludeCollect", true);
let resultDf = Query(queryConfiguration);
let mailBodyHtml = resultDf.`
<table>
<caption>Example query</caption>
<thead>
<tr>
${StringJoin("", _.Columns.`
<th>${_}</th>
`)}
</tr>
</thead>
<tbody>
${StringJoin("", _.Rows.`
<tr>
${StringJoin("", _.`
<td>${_}</td>
`)}
</tr>
`)}
</tbody>
</table>
`;

SendEmail(#{
"ReplyTo": [replyToAddress],
"To": recipientsArray,
"Subject": "Example query E-mail",
"IsBodyHtml": true,
"Body": mailBodyHtml
});

</syntaxhighlight>NOTE: QPR ProcessAnalyzer Server has to have SMTP server configured. Also, remember to update the values of replyToAddress and recipientsArray before using.

Expression Script Examples

2025-03-18T14:30:34Z

MarHink: /* Perform a query and send results as E-mail in a HTML table */

This page contains script examples written in the QPR ProcessAnalyzer expression language. See how expression scripts can be created in the [[Managing_Scripts#Creating_Script|Workspace]]. For documentation for the syntax, functions and entities can be found from the main page in the [[QPR_ProcessAnalyzer_Wiki#For_Developers|KPI Expression Language]] section.

== Calling Expression Script from Expression ==
Expression scripts can be called from an expression using the [[QPR_ProcessAnalyzer_Objects_in_Expression_Language#Script|Run]] function with the following syntax:
<pre>
let result = ScriptById(123).Run(#{
"parameter1": "value1",
"parameter2": false,
"parameter2": 123.45
})
</pre>

The command waits until the run is completed, and the return value of the called script is returned by the Run function call.

Parameters can be passed to the called script, and the parameters are available as variables in the script. The parameters can contain any type of data.

Expression scripts can also be called from a dashboard. Expressions can be stored to scripts instead of dashboards, which is a way to separate complex expressions from dashboards and allow to reuse expressions across several dashboards.

== Calling SQL Script from Expression ==
SQL script can be called from an expression using the Run function as follows (similar to calling [[#Calling Expression Script from Expression|expression scripts]]):
<pre>
let result = ScriptById(123).Run(#{
"parameter1": "value1",
"parameter2": 321
});
let arrayOfAllReports = result.Keys;
let report1 = result.Report1;
let report2 = result.Report2;
</pre>

SQL scripts can return multiple ''reports'', which are combined to a dictionary, where the key is the name of the report ("sheet name") and value is the report data as a DataFrame. See in the above example, how the reports can be accessed by their name.

== Examples ==
=== Call web service===
Contact to a web service, fetch some data, and store it to a datatable.

<syntaxhighlight lang="typescript" line>
let datatableName = "Web Service Data";
let webServiceData = CallWebService(
#{"Address": "https://processanalyzer.onqpr.com/qprpa/api/serverinfo"}
);

let targetDatatable = Project.Datatables.Where(name==datatableName);
if (Count(targetDatatable) == 0) {
targetDatatable = Project.CreateDatatable(datatableName)
.AddColumn("Setting name", "String")
.AddColumn("Setting value", "String")
.AddColumn("Data read", "DateTime");
} else {
targetDatatable = targetDatatable[0];
}

let currentTime = Now;
let dataAsDf = ToDataFrame(
webServiceData.keys.{
let key = _;
[key, webServiceData[key], currentTime];
},
["Setting name", "Setting value", "Data read"]
);
targetDatatable.Import(dataAsDf, #{"Append":true});
WriteLog(`${CountTop(dataAsDf.Rows)} rows written to datatable`);
</syntaxhighlight>

=== Store data to datatable ===
Get all models in the system and store them to a datatable.

<syntaxhighlight lang="typescript" line>
let newDatatable = Project
.CreateDatatable("Models list " + ToString(Now, "dd.MM.yyyy HH:mm:ss"))
.AddColumn("Model name", "String")
.AddColumn("Project name", "String")
.AddColumn("Created time", "DateTime")
.AddColumn("Cases", "Integer");
let startTime = Now;
let modelsData = ToDataFrame(
Models.([Name, Project.Name, CreatedDate, NCases]),
["Model name", "Project name", "Created time", "Cases"]
);
WriteLog(`Listing models took ${(Now - startTime).TotalSeconds.Round(2)} seconds.`);
newDatatable.Import(modelsData);
WriteLog(`Datatable ${newDatatable.Id} created.`);
</syntaxhighlight>

=== Convert datatable column data ===
This script can be used to convert a single column into numerical data type. To use the script, you need to setup the following in the beginning of the script:
* Project name where the datatable is located.
* Datatable name
* Name of the column to be converted

Note that the conversion fails, if there is data that cannot be converted into numerical format. The conversion assumes that period (.) is used as the decimal point.

<syntaxhighlight lang="typescript" line>
let projectName = "New Project";
let datatableName = "qpr processanalyzer events";
let columnName = "Event order in case";

let project = (Projects.Where(Name==projectName))[0];
let datatable = (project.Datatables.Where(Name==datatableName))[0];
DatatableById(datatable.Id).DataFrame
.SetColumns([
columnName: () => {
let data = Column(columnName);
if (data == null) {
null;
} else {
ToFloat(data);
}
}
])
.Persist(datatable.Name, ["ProjectId": project.Id, "Append": false]);
</syntaxhighlight>

Instead of converting to numeric (with the ''ToFloat'' function), data can be converted into string using the ''ToString'' function.

=== Show DataFrame as HTML table ===

This script defines a function to show dataframe as a HTML table, and uses the function for a literal dataframe.

<syntaxhighlight lang="typescript" line>
function dataframeToHtmlTable(df) {
return
`<table>
<tr>
${StringJoin("\r\n\t\t", + df.columns.`<th>${_}</th>`)}
</tr>
${StringJoin("", df.Rows.(
"\r\n\t<tr>" + StringJoin("", _.`\r\n\t\t<td>${ToString(_)}</td>`) + "\r\n\t</tr>"
))}
</table>`
}

let data = ToDataFrame(
[
["one", "two", "three"],
["four", "five", "six"],
["seven", "eight", "nine"]
],
["Column 1", "Column 2", "Column 3"]
);

return dataframeToHtmlTable(data);
</syntaxhighlight>

=== Copy local datatables to Snowflake ===

<syntaxhighlight lang="typescript" line>
// Copies all datatables in a project to another project including datatable contents.
// Usage instructions:
// 1. Create expression script in the project from where you want to copy the datatables.
// 2. Create a new project named as "<name of the project to be moved> - Snowflake". New datatables will be created here. E.g., when moving project named "SAP_OrderToCash", the target project should be named as "SAP_OrderToCash - Snowflake".
// 3. Run the script.
// NOTE: Columns of type "Any" will be created as "String"-columns in Snowflake, thus it is recommended that actual data types are set for the tables prior to the move.

let sourceProject = Project;
let sourceProjectName = Project.Name;
let targetProjectName = `${sourceProjectName} - Snowflake`;
let targetProject = First(Projects.Where(Name == targetProjectName));
if (IsNull(targetProject)) {
WriteLog(`Unable to find target project named "${targetProjectName}". Aborting operation.`);
return;
}
let dts = sourceProject.DataTables;
WriteLog(`Copying all ${CountTop(dts)} data tables found in project "${sourceProject.Name}" (id: ${sourceProject.Id}) to Snowflake in project "${targetProject.Name}" (id: ${targetProject.Id})`);
dts.{
let sourceDt = _;
WriteLog(`Starting to copy data table "${Name}" (id: ${Id}) having ${NRows} rows and ${NColumns} columns.`);
let targetDt;
targetDt = targetProject.DatatableByName(sourceDt.Name);
if (targetDt == null) {
targetDt = targetProject.CreateDataTable(sourceDt.Name, #{"Connection": CreateSnowflakeConnection(#{"ProjectId": targetProject.Id})});
targetDt.Import(sourceDt.SqlDataFrame);
WriteLog(`Finished copying data table "${Name}" (id: ${Id}) to table "${targetDt.Name}" (id: ${targetDt.Id})`);
} else {
WriteLog(`Datatable already exist "${Name}" (id: ${Id}) to table "${targetDt.Name}" (id: ${targetDt.Id})`);
}
}
WriteLog(`Finished copying all the data tables found in project "${sourceProject.Name}" (id: ${sourceProject.Id}) to Snowflake in project "${targetProject.Name}" (id: ${targetProject.Id})`);
</syntaxhighlight>

If you don't need to copy the data but only create the Snowflake datatables with columns, you can change the line 22 to
<syntaxhighlight>
targetDt.Import(sourceDt.SqlDataFrame.head(0));
</syntaxhighlight>

=== Copy single datatable to Snowflake ===
This script creates a copy of a single datatable to Snowflake. Replace the ''<tableId1>'' with the id of the source datatable.
<syntaxhighlight lang="typescript" line>
function CopyDataTableToSnowflake(dataTableId)
{
let sourceDt = DataTableById(dataTableId);
sourceDt.SqlDataFrame.Persist(`${sourceDt.Name} - Snowflake`, #{"Append": false, "Connection": CreateSnowflakeConnection(#{"ProjectId": sourceDt.Project.Id})});
}
CopyDataTableToSnowflake(<tableId1>);
</syntaxhighlight>

=== Create a copy of a data table that has all Any-type columns changed to String-type columns ===
<syntaxhighlight lang="typescript" line>
function ConvertAnyDataTypesToStringsToNewTable(dataTableId)
{
let dt = DataTableById(dataTableId);
let sdf = dt.SqlDataFrame;
let cts = dt.ColumnTypes;
cts.{
let ct = _;
if (ct.DataType == "Any") {
let n = ct.Name;
sdf = sdf.WithColumn(ct.Name, #sql{Cast(Column(Variable("n")), "ShortString")});
}
};
sdf.Persist(`${dt.Name} - Converted`, #{"Append": false, "ProjectId": dt.Project.Id});
}
ConvertAnyDataTypesToStringsToNewTable(<dataTableId>);
</syntaxhighlight>

=== Query number of rows in given data table having a datetime value in given year grouped by month and return resulting table as CSV ===
SqlDataFrame is used in order to prevent loading the whole datatable into memory first. Filtering is performed as first operation in order to minimize the amount of required work for the data source of the data table.<syntaxhighlight lang="typescript" line="1">
DataTableById(<data table id>)
.SqlDataFrame
.Where(#sql{2014 == Year(Column("Start Time"))})
.WithColumn("Month", #sql{Month(Column("Start Time"))})
.GroupBy(["Month"]).Aggregate(["Count"], ["Count"])
.OrderByColumns(["Month"], [true])
.Collect().ToCsv();
</syntaxhighlight>

=== Function for filtering SqlDataFrame by removing rows having, or replacing, the most infrequently occurring column values ===
<syntaxhighlight lang="typescript" line="1">
/***
* @name ColumnWithMinUsage
* @descripion
* Generic function that can be used to filter out the most infrequently occurring attribute values or replace their Values
* with given common value.
* @param df:
* DataFrame to operate on.
* @param columnName:
* Name of the column to be filtered.
* @param newColumnName:
* Name of the column that will contain the new value of the original column after filtering (if includeOthers was applied).
* @param maxNumUniqueValues:
* Maximum number of unique values to include into the comparison for each attribute column. If the amount of unique values for any attribute exceeds this value, only given number of attributes are included that have the highest usage.
* @param minValueUsage:
* Minimum total usage of a value included into the comparison. The number of cases having every returned value should be at least given percentage (a float value between 0.0 and 1.0) of all the compared cases.
* @param includeOthers:
* Should the rest of the attribute values not included due to MinValueUsage or MaxNumUniqueValues filtering be included as an aggregated "Others" value?
* If not empty/null, defines the name used for these other-values.
*/
function ColumnWithMinUsage(df, columnName, newColumnName, maxNumUniqueValues, minValueUsage, includeOthers)
{
let all = df
.GroupBy([])
.Aggregate(["NAllTotal"], ["Count"])
.WithColumn("__Join2", #sql{1});
let minValueUsageEnabled = !IsNullTop(minValueUsage);
let maxNumUniqueValuesEnabled = !IsNullTop(maxNumUniqueValues);
if (minValueUsageEnabled || maxNumUniqueValuesEnabled) {
// Perform column value-based filtering if minValueUsageEnabled or maxNumUniqueValuesEnabled is defined.
let valueColumnName = "__ValueNew";
let filteredValuesColumns = [valueColumnName: columnName];
let filteredValues = df
.GroupBy([columnName]).Aggregate(["Count"], ["Count"]);
if (minValueUsageEnabled) {
filteredValues = filteredValues
.WithColumn("__Join", #sql{1})
.Join(all, ["__Join": "__Join2"], "leftouter")
.WithColumn("Usage", #sql{Column("Count") / Column("NAllTotal")});
filteredValuesColumns = Concat(filteredValuesColumns, ["Usage"]);
}
if (maxNumUniqueValuesEnabled) {
filteredValues = filteredValues
.WithRowNumberColumn("RowNumber", ["Count"], null, [false]);
filteredValuesColumns = Concat(filteredValuesColumns, ["RowNumber"]);
}

filteredValues = filteredValues
.Select(filteredValuesColumns);

// Generate select returning all the accepted values.
let allValues = filteredValues
.(minValueUsageEnabled ? Where(#sql{Column("Usage") >= #expr{minValueUsage}}) : _)
.(maxNumUniqueValuesEnabled ? Where(#sql{Column("RowNumber") <= #expr{maxNumUniqueValues}}) : _)
.Select([valueColumnName, newColumnName: valueColumnName]);

if (!IsNullTop(includeOthers)) {
// If includeOthers is defined, replace original values with the variable defined in includeOthers.
let otherValues = filteredValues
.(minValueUsageEnabled ? Where(#sql{Column("Usage") < #expr{minValueUsage}}) : _)
.(maxNumUniqueValuesEnabled ? Where(#sql{Column("RowNumber") > #expr{maxNumUniqueValues}}) : _)
.WithColumn(newColumnName, #sql{#expr{includeOthers}})
.Select([valueColumnName, newColumnName]);
allValues = allValues.Append(otherValues)
}
df.Join(allValues, [columnName: valueColumnName], "inner")
.RemoveColumns([valueColumnName]);
}
}

// The following example will return only rows containing two of the most common values for Region-column.
//let df = DataTableById(<data table id>).SqlDataFrame;
//df = ColumnWithMinUsage(df, "Region", "_Filtered", 2, null, null);
//df.Collect().ToCsv();

// The following example will return all input rows, but will replace the values of rows whose Region-column
// has a value used by less than 15% of all the rows with a new value: "_Others".
//let df = DataTableById(<data table id>).SqlDataFrame;
//df = ColumnWithMinUsage(df, "Region", "_Filtered", null, 0.15, "_Others");
//df.Collect().ToCsv();
</syntaxhighlight>

=== Export model events and cases ===
<syntaxhighlight lang="typescript" line="1">
function ExportModelEvents(m) {
let attrs = m.EventAttributes;
ToDataFrame(
m.EventLog.Events.Concat(
[Case.Name, Type.Name, ToString(TimeStamp, "yyyy-MM-dd HH:mm:ss.fff")],
{let evt = _; attrs.{let att = _; evt.Attribute(att)}}
),
Concat(
["CaseId", "EventType", "TimeStamp"],
attrs.Name
)
).ToCsv(true);
}

function ExportModelCases(m) {
let attrs = m.CaseAttributes;
ToDataFrame(
m.EventLog.Cases.Concat(
[Name],
{let cas = _; attrs.{let att = _; cas.Attribute(att)}}
),
Concat(
["CaseId"],
attrs.Name
)
).ToCsv(true);
}
</pre>

<pre>
First(Models.Where(Name=="SAP OtC Extended")).EventsDataTable.DataFrame.ToCsv(true)
</pre>

<pre>
First(Models.Where(Name=="SAP OtC Extended")).CasesDataTable.DataFrame.ToCsv(true)
</syntaxhighlight>

=== Calculate all the value usages of a single column for each event in event data table ===
This query could be used, e.g., to find out the maximum resource usage for every resource found in the event data table.<syntaxhighlight lang="typescript" line="1">
function WithUsageColumns(resourceColumn)
{
function WithTotalUsageColumnOfSingleResource(resourceColumn, resourceValue)
{
_
.WithColumn("_Prev", #sql{Lag(Column(resourceColumn), [TimeStamp, EventType], [true, true], [CaseId], 1, null)})
.WithColumn("_UsageDiff", #sql{
CaseWhen(
Column(resourceColumn) == Column("_Prev"), 0,
Column("_Prev") == #expr{resourceValue}, -1,
Column(resourceColumn) == #expr{resourceValue}, 1,
0)
})
.WithColumn(`${resourceValue}_Usage`, #sql{Sum(Column("_UsageDiff"), [TimeStamp, EventType])})
.RemoveColumns(["_Prev", "_UsageDiff"])
}

let sdf = _;
let allValues = sdf.SelectDistinct([resourceColumn]).OrderByColumns([resourceColumn], [true]).Collect().Column(resourceColumn);
allValues.{
let v = _;
sdf = sdf.WithTotalUsageColumnOfSingleResource(resourceColumn, v)
}
sdf
}

let dt = ModelById(<model id>).EventsDataTable;
dt
.SqlDataFrame
.WithUsageColumns(<resource column name>)
.OrderByColumns([dt.ColumnMappings["TimeStamp"]], [true])
.Collect().ToCsv()
</syntaxhighlight>Where:

* <model id> is the id of the model containing event data to be examined.
* <resource column name> is the name of the column in the event data table of the specified model containing the resource being used by that event.

NOTE: This expression uses functionalities that are only supported in Snowflake-based data tables.

=== Create new Snowflake model from filter ===
This script creates a new Snowflake model (and two datatables for cases and events) containing filtered event log from given filter id. The script also works if the model doesn't have a cases datatable.

<syntaxhighlight lang="typescript" line="1">
let filter = FilterById(1); // filter id
let model = filter.model;
let project = model.project;
let nameSuffix = " - " + filter.name + " - " + ToString(Now, "dd-MM-yyyy HH:mm:ss");
let eventsDatatableName = model.EventsDataTable.Name + nameSuffix;
if (eventsDatatableName.length > 440) {
eventsDatatableName = eventsDatatableName.Substring(eventsDatatableName.length - 440);
}
let eventsData = model
.EventsDataTable
.SqlDataFrame
.ApplyFilter(
filter.rules,
model.CasesDataTable?.SqlDataFrame
);
project
.CreateDatatable(eventsDatatableName, #{"Connection": CreateSnowflakeConnection()})
.Import(eventsData);
let modelConfiguration = model.Configuration;
modelConfiguration.DataSource.Events.Set("DataTableName", eventsDatatableName);
if (model.CasesDataTable != null) {
let eventsDataCaseIdColumn = "CaseId_" + ToString(Random());
let casesDatatableName = model.CasesDataTable.Name + nameSuffix;
if (casesDatatableName.length > 440) {
casesDatatableName = casesDatatableName.Substring(casesDatatableName.length - 440);
}
let casesData = model
.CasesDataTable
.SqlDataFrame
.join(
eventsData.SelectDistinct([eventsDataCaseIdColumn: modelConfiguration.DataSource.Events.Columns.CaseId]),
[modelConfiguration.DataSource.Cases.Columns.CaseId: eventsDataCaseIdColumn]
).Select(model.CasesDataTable.ColumnNames);
project
.CreateDatatable(casesDatatableName, #{"Connection": CreateSnowflakeConnection()})
.Import(casesData);
modelConfiguration.DataSource.Cases.Set("DataTableName", casesDatatableName);
}
let modelName = model.Name + nameSuffix;
if (modelName > 440) {
modelName = modelName.Substring(modelName.length - 440);
}
project
.CreateModel(#{
"Name": modelName,
"Description": model.Description,
"Configuration": modelConfiguration
});
return modelName;
</syntaxhighlight>

=== Creating a model consisting of multiple copies of cases in an existing model ===
<syntaxhighlight lang="typescript" line="1">
/**
* @name CreateTestModel
* @description
* Creates a new model (or overwrites an existing) to given target project with given number of
* repetitions of given source model.
* Each repetition will generate "<N>-"-prefix to CaseId-columns, where N equals to the repeat index.
* @param sourceModel
* PA model used for the source data and from where the connection is copied for the target model if a
* new one has to be created.
* @param numRepeats
* Number of times the data in the source model should be repeated in the generated model.
* @param targetProject
* Project in which the target model resides.
* @param targetModelName
* Specifies the name of the test model in the given target project. If a model already exists with
* given name, event and case data in this model will be replaced with the new generated event and
* case data.
* @returns
* Model object of the test model having the newly generated data.
*/
function CreateTestModel(sourceModel, numRepeats, targetProject, targetModelName)
{
let eventsColumnMappings = sourceModel.EventsDataTable.ColumnMappings;
let casesColumnMappings = sourceModel.CasesDataTable.ColumnMappings;
let connection = sourceModel.EventsDataTable.DataSourceConnection;

function CreateResultModel()
{
function GetTable(tableName)
{
let tableConfiguration = #{
"Name": tableName,
"Connection": connection
};
let resultTable = targetProject.DataTableByName(tableName);
if (resultTable == null)
{
resultTable = targetProject.CreateDataTable(tableConfiguration)
.Modify(#{"NameInDataSource": null})
.Synchronize();
}
return resultTable;
}

let eventsTableName = `${targetModelName} - events`;
let casesTableName = `${targetModelName} - cases`;
let targetModel = targetProject.ModelByName(targetModelName);
let eventsTable, casesTable = null;

if (targetModel != null)
{
eventsTable = targetModel.EventsDataTable;
casesTable = targetModel.CasesDataTable;
}
else {
eventsTable = GetTable(eventsTableName);
if (sourceModel.CasesDataTable != null) {
casesTable = GetTable(casesTableName);
}

let timestampMapping = eventsColumnMappings["TimeStamp"];
eventsColumnMappings.Remove("TimeStamp");
eventsColumnMappings.Set("Timestamp", timestampMapping);

let modelConfiguration = #{
"DataSource": #{
"Events":#{
"DataSourceType": "datatable",
"DataTableName": eventsTableName,
"Columns": eventsColumnMappings
}
}
};

if (casesColumnMappings != null) {
modelConfiguration["DataSource"].Set("Cases", #{
"DataSourceType": "datatable",
"DataTableName": casesTableName,
"Columns": casesColumnMappings
});
}

targetModel = targetProject.CreateModel(#{"Name": targetModelName, "Configuration": modelConfiguration});
}

eventsTable.Truncate();
casesTable?.Truncate();

return #{
"TargetModel": targetModel,
"Events": eventsTable,
"Cases": casesTable
};
}

function RepeatNTimes(sourceDf, caseIdColumn, numRepeats)
{
let resultDf = null;
for (let i = 1; i <= numRepeats; ++i) {
let iterationDf = sourceDf
.WithColumn(caseIdColumn, #sql{Concat(#expr{i}, "-", Column(#expr{caseIdColumn}))});
resultDf = resultDf == null ? iterationDf : resultDf.Append(iterationDf);
}
resultDf;
}

let resultModel = CreateResultModel();
let sourceEventDataDf = sourceModel.EventsDataTable.SqlDataFrame;
let resultEventDataDf = RepeatNTimes(sourceEventDataDf, eventsColumnMappings["CaseId"], numRepeats);
resultModel["Events"].Import(resultEventDataDf);

let sourceCaseDataDf = sourceModel.CasesDataTable?.SqlDataFrame;
if (sourceCaseDataDf != null) {
let resultCaseDataDf = RepeatNTimes(sourceCaseDataDf, casesColumnMappings["CaseId"], numRepeats);
resultModel["Cases"].Import(resultCaseDataDf);
}
resultModel["TargetModel"];
}

</syntaxhighlight>Example usage:<blockquote>CreateTestModel(ProjectByName("Project").ModelByName("SAP_OrderToCash - Snowflake"), 3, ProjectByName("TestData"), "TestModel");</blockquote>Creates a new model named "TestModel" (or overwrites old one) into project named "TestData" containing the data from model "SAP_OrderToCash - Snowflake" in project "Project" repeated three times.

=== Analyzing declare patterns found in event log ===

This is an example expression that shows how POSIX-style regular expressions can be used to search for cases in an event log having certain event type patterns [https://www.researchgate.net/publication/277631859_Generating_Event_Logs_Through_the_Simulation_of_Declare_Models declare patterns].

Note: Before use, replace <model id> with a valid model identifier having event data.

<syntaxhighlight lang="typescript" line="1">
let dt = ModelById(<model id>).EventsDataTable;
let sdf = dt.SqlDataFrame.Head(1000);
let caseIdColumn = dt.ColumnMappings["CaseId"], timeStampColumn = dt.ColumnMappings["TimeStamp"], eventTypeColumn = dt.ColumnMappings["EventType"];

let eventTypesDf = sdf
.SelectDistinct([eventTypeColumn])
.OrderByColumns([eventTypeColumn], [true])
.WithRowNumberColumn("Token", [eventTypeColumn])
.WithColumn("Token", #sql{Char(Column("Token") + Unicode("a") - 1)});

sdf = sdf
.Join(eventTypesDf.Select(["_EventType2": eventTypeColumn, "Token"]), [eventTypeColumn: "_EventType2"], "leftouter");

function RespondedExistencePattern(a, b)
{
// [^a]*((a.*b.*)|(b.*a.*))*[^a]*
ToSqlExpression(`Regexp(Column("Trace"), "[^${a}]*((${a}.*${b}.*)|(${b}.*${a}.*))*[^${a}]")`)
}

function ResponsePattern(a, b)
{
// [^a]*(a.*b)*[^a]*
ToSqlExpression(`Regexp(Column("Trace"), "[^${a}]*(${a}.*${b})*[^${a}]")`)
}

function AlternateResponsePattern(a, b)
{
// [^a]*(a[^a]*b[^a]*)*[^a]*
ToSqlExpression(`Regexp(Column("Trace"), "[^${a}]*(${a}[^${a}]*${b}[^${a}]*)*[^${a}]")`)
}

function ChainResponsePattern(a, b)
{
// [^a]*(ab[^a]*)*[^a]*
ToSqlExpression(`Regexp(Column("Trace"), "[^${a}]*(${a}${b}[^${a}]*)*[^${a}]")`)
}

function PrecedencePattern(a, b)
{
// [^b]*(a.*b)*[^b]*
ToSqlExpression(`Regexp(Column("Trace"), "[^${b}]*(${a}.*${b})*[^${b}]")`)
}

let tracesDf = sdf
.GroupBy([caseIdColumn])
.Aggregate(["Trace": "Token"], [#{"Function": "list", "Ordering": [timeStampColumn], "Separator": ""}])
.WithColumn("RespondedExistenceNL", #sql{#expr{RespondedExistencePattern("n", "l")}})
.WithColumn("RespondedExistenceCA", #sql{#expr{RespondedExistencePattern("c", "a")}})
.WithColumn("ResponsePatternNL", #sql{#expr{ResponsePattern("n", "l")}})
.WithColumn("ResponsePatternCA", #sql{#expr{ResponsePattern("c", "a")}})
.WithColumn("AlternateResponsePatternNL", #sql{#expr{AlternateResponsePattern("n", "l")}})
.WithColumn("AlternateResponsePatternCA", #sql{#expr{AlternateResponsePattern("c", "a")}})
.WithColumn("ChainResponsePatternNL", #sql{#expr{ChainResponsePattern("n", "l")}})
.WithColumn("ChainResponsePatternCA", #sql{#expr{ChainResponsePattern("c", "a")}})
.WithColumn("PrecedencePatternNL", #sql{#expr{PrecedencePattern("n", "l")}})
.WithColumn("PrecedencePatternCA", #sql{#expr{PrecedencePattern("c", "a")}})
;

[tracesDf.Collect().ToCsv(), eventTypesDf.Collect().ToCsv()]
</syntaxhighlight>

=== Perform a query and send results as E-mail in a HTML table ===
This example requires two scripts that are both located in the same project:

# Expression-type script to execute (can be, e.g., scheduled to run daily).
# Expression-type script containing the query JSON to use as basis for the e-mail. In this example, this script is named as "Send query as E-mail - query JSON". The contents of this script is just the JSON representation of a query that can be extracted, e.g., from any PA chart view.

Script #1 should contain the following code:<syntaxhighlight lang="typescript">
let replyToAddress = "noreply@test.com";
let recipientsArray = ["test@test.com"];
let queryConfiguration = Project.ScriptByName("Send query as E-mail - query JSON").Code;
let resultDf = Query(ParseJson(queryConfiguration));
let mailBodyHtml = resultDf.Collect().`
<table>
<caption>Example query</caption>
<thead>
<tr>
${StringJoin("", _.Columns.`
<th>${_}</th>
`)}
</tr>
</thead>
<tbody>
${StringJoin("", _.Rows.`
<tr>
${StringJoin("", _.`
<td>${_}</td>
`)}
</tr>
`)}
</tbody>
</table>
`;

SendEmail(#{
"ReplyTo": [replyToAddress],
"To": recipientsArray,
"Subject": "Example query E-mail",
"IsBodyHtml": true,
"Body": mailBodyHtml
});

</syntaxhighlight>NOTE: QPR ProcessAnalyzer Server has to have SMTP server configured. Also, remember to update the values of replyToAddress and recipientsArray before using.

Expression Script Examples

2025-03-18T14:29:35Z

MarHink: /* Perform a query and send results as E-mail in a HTML table */

This page contains script examples written in the QPR ProcessAnalyzer expression language. See how expression scripts can be created in the [[Managing_Scripts#Creating_Script|Workspace]]. For documentation for the syntax, functions and entities can be found from the main page in the [[QPR_ProcessAnalyzer_Wiki#For_Developers|KPI Expression Language]] section.

== Calling Expression Script from Expression ==
Expression scripts can be called from an expression using the [[QPR_ProcessAnalyzer_Objects_in_Expression_Language#Script|Run]] function with the following syntax:
<pre>
let result = ScriptById(123).Run(#{
"parameter1": "value1",
"parameter2": false,
"parameter2": 123.45
})
</pre>

The command waits until the run is completed, and the return value of the called script is returned by the Run function call.

Parameters can be passed to the called script, and the parameters are available as variables in the script. The parameters can contain any type of data.

Expression scripts can also be called from a dashboard. Expressions can be stored to scripts instead of dashboards, which is a way to separate complex expressions from dashboards and allow to reuse expressions across several dashboards.

== Calling SQL Script from Expression ==
SQL script can be called from an expression using the Run function as follows (similar to calling [[#Calling Expression Script from Expression|expression scripts]]):
<pre>
let result = ScriptById(123).Run(#{
"parameter1": "value1",
"parameter2": 321
});
let arrayOfAllReports = result.Keys;
let report1 = result.Report1;
let report2 = result.Report2;
</pre>

SQL scripts can return multiple ''reports'', which are combined to a dictionary, where the key is the name of the report ("sheet name") and value is the report data as a DataFrame. See in the above example, how the reports can be accessed by their name.

== Examples ==
=== Call web service===
Contact to a web service, fetch some data, and store it to a datatable.

<syntaxhighlight lang="typescript" line>
let datatableName = "Web Service Data";
let webServiceData = CallWebService(
#{"Address": "https://processanalyzer.onqpr.com/qprpa/api/serverinfo"}
);

let targetDatatable = Project.Datatables.Where(name==datatableName);
if (Count(targetDatatable) == 0) {
targetDatatable = Project.CreateDatatable(datatableName)
.AddColumn("Setting name", "String")
.AddColumn("Setting value", "String")
.AddColumn("Data read", "DateTime");
} else {
targetDatatable = targetDatatable[0];
}

let currentTime = Now;
let dataAsDf = ToDataFrame(
webServiceData.keys.{
let key = _;
[key, webServiceData[key], currentTime];
},
["Setting name", "Setting value", "Data read"]
);
targetDatatable.Import(dataAsDf, #{"Append":true});
WriteLog(`${CountTop(dataAsDf.Rows)} rows written to datatable`);
</syntaxhighlight>

=== Store data to datatable ===
Get all models in the system and store them to a datatable.

<syntaxhighlight lang="typescript" line>
let newDatatable = Project
.CreateDatatable("Models list " + ToString(Now, "dd.MM.yyyy HH:mm:ss"))
.AddColumn("Model name", "String")
.AddColumn("Project name", "String")
.AddColumn("Created time", "DateTime")
.AddColumn("Cases", "Integer");
let startTime = Now;
let modelsData = ToDataFrame(
Models.([Name, Project.Name, CreatedDate, NCases]),
["Model name", "Project name", "Created time", "Cases"]
);
WriteLog(`Listing models took ${(Now - startTime).TotalSeconds.Round(2)} seconds.`);
newDatatable.Import(modelsData);
WriteLog(`Datatable ${newDatatable.Id} created.`);
</syntaxhighlight>

=== Convert datatable column data ===
This script can be used to convert a single column into numerical data type. To use the script, you need to setup the following in the beginning of the script:
* Project name where the datatable is located.
* Datatable name
* Name of the column to be converted

Note that the conversion fails, if there is data that cannot be converted into numerical format. The conversion assumes that period (.) is used as the decimal point.

<syntaxhighlight lang="typescript" line>
let projectName = "New Project";
let datatableName = "qpr processanalyzer events";
let columnName = "Event order in case";

let project = (Projects.Where(Name==projectName))[0];
let datatable = (project.Datatables.Where(Name==datatableName))[0];
DatatableById(datatable.Id).DataFrame
.SetColumns([
columnName: () => {
let data = Column(columnName);
if (data == null) {
null;
} else {
ToFloat(data);
}
}
])
.Persist(datatable.Name, ["ProjectId": project.Id, "Append": false]);
</syntaxhighlight>

Instead of converting to numeric (with the ''ToFloat'' function), data can be converted into string using the ''ToString'' function.

=== Show DataFrame as HTML table ===

This script defines a function to show dataframe as a HTML table, and uses the function for a literal dataframe.

<syntaxhighlight lang="typescript" line>
function dataframeToHtmlTable(df) {
return
`<table>
<tr>
${StringJoin("\r\n\t\t", + df.columns.`<th>${_}</th>`)}
</tr>
${StringJoin("", df.Rows.(
"\r\n\t<tr>" + StringJoin("", _.`\r\n\t\t<td>${ToString(_)}</td>`) + "\r\n\t</tr>"
))}
</table>`
}

let data = ToDataFrame(
[
["one", "two", "three"],
["four", "five", "six"],
["seven", "eight", "nine"]
],
["Column 1", "Column 2", "Column 3"]
);

return dataframeToHtmlTable(data);
</syntaxhighlight>

=== Copy local datatables to Snowflake ===

<syntaxhighlight lang="typescript" line>
// Copies all datatables in a project to another project including datatable contents.
// Usage instructions:
// 1. Create expression script in the project from where you want to copy the datatables.
// 2. Create a new project named as "<name of the project to be moved> - Snowflake". New datatables will be created here. E.g., when moving project named "SAP_OrderToCash", the target project should be named as "SAP_OrderToCash - Snowflake".
// 3. Run the script.
// NOTE: Columns of type "Any" will be created as "String"-columns in Snowflake, thus it is recommended that actual data types are set for the tables prior to the move.

let sourceProject = Project;
let sourceProjectName = Project.Name;
let targetProjectName = `${sourceProjectName} - Snowflake`;
let targetProject = First(Projects.Where(Name == targetProjectName));
if (IsNull(targetProject)) {
WriteLog(`Unable to find target project named "${targetProjectName}". Aborting operation.`);
return;
}
let dts = sourceProject.DataTables;
WriteLog(`Copying all ${CountTop(dts)} data tables found in project "${sourceProject.Name}" (id: ${sourceProject.Id}) to Snowflake in project "${targetProject.Name}" (id: ${targetProject.Id})`);
dts.{
let sourceDt = _;
WriteLog(`Starting to copy data table "${Name}" (id: ${Id}) having ${NRows} rows and ${NColumns} columns.`);
let targetDt;
targetDt = targetProject.DatatableByName(sourceDt.Name);
if (targetDt == null) {
targetDt = targetProject.CreateDataTable(sourceDt.Name, #{"Connection": CreateSnowflakeConnection(#{"ProjectId": targetProject.Id})});
targetDt.Import(sourceDt.SqlDataFrame);
WriteLog(`Finished copying data table "${Name}" (id: ${Id}) to table "${targetDt.Name}" (id: ${targetDt.Id})`);
} else {
WriteLog(`Datatable already exist "${Name}" (id: ${Id}) to table "${targetDt.Name}" (id: ${targetDt.Id})`);
}
}
WriteLog(`Finished copying all the data tables found in project "${sourceProject.Name}" (id: ${sourceProject.Id}) to Snowflake in project "${targetProject.Name}" (id: ${targetProject.Id})`);
</syntaxhighlight>

If you don't need to copy the data but only create the Snowflake datatables with columns, you can change the line 22 to
<syntaxhighlight>
targetDt.Import(sourceDt.SqlDataFrame.head(0));
</syntaxhighlight>

=== Copy single datatable to Snowflake ===
This script creates a copy of a single datatable to Snowflake. Replace the ''<tableId1>'' with the id of the source datatable.
<syntaxhighlight lang="typescript" line>
function CopyDataTableToSnowflake(dataTableId)
{
let sourceDt = DataTableById(dataTableId);
sourceDt.SqlDataFrame.Persist(`${sourceDt.Name} - Snowflake`, #{"Append": false, "Connection": CreateSnowflakeConnection(#{"ProjectId": sourceDt.Project.Id})});
}
CopyDataTableToSnowflake(<tableId1>);
</syntaxhighlight>

=== Create a copy of a data table that has all Any-type columns changed to String-type columns ===
<syntaxhighlight lang="typescript" line>
function ConvertAnyDataTypesToStringsToNewTable(dataTableId)
{
let dt = DataTableById(dataTableId);
let sdf = dt.SqlDataFrame;
let cts = dt.ColumnTypes;
cts.{
let ct = _;
if (ct.DataType == "Any") {
let n = ct.Name;
sdf = sdf.WithColumn(ct.Name, #sql{Cast(Column(Variable("n")), "ShortString")});
}
};
sdf.Persist(`${dt.Name} - Converted`, #{"Append": false, "ProjectId": dt.Project.Id});
}
ConvertAnyDataTypesToStringsToNewTable(<dataTableId>);
</syntaxhighlight>

=== Query number of rows in given data table having a datetime value in given year grouped by month and return resulting table as CSV ===
SqlDataFrame is used in order to prevent loading the whole datatable into memory first. Filtering is performed as first operation in order to minimize the amount of required work for the data source of the data table.<syntaxhighlight lang="typescript" line="1">
DataTableById(<data table id>)
.SqlDataFrame
.Where(#sql{2014 == Year(Column("Start Time"))})
.WithColumn("Month", #sql{Month(Column("Start Time"))})
.GroupBy(["Month"]).Aggregate(["Count"], ["Count"])
.OrderByColumns(["Month"], [true])
.Collect().ToCsv();
</syntaxhighlight>

=== Function for filtering SqlDataFrame by removing rows having, or replacing, the most infrequently occurring column values ===
<syntaxhighlight lang="typescript" line="1">
/***
* @name ColumnWithMinUsage
* @descripion
* Generic function that can be used to filter out the most infrequently occurring attribute values or replace their Values
* with given common value.
* @param df:
* DataFrame to operate on.
* @param columnName:
* Name of the column to be filtered.
* @param newColumnName:
* Name of the column that will contain the new value of the original column after filtering (if includeOthers was applied).
* @param maxNumUniqueValues:
* Maximum number of unique values to include into the comparison for each attribute column. If the amount of unique values for any attribute exceeds this value, only given number of attributes are included that have the highest usage.
* @param minValueUsage:
* Minimum total usage of a value included into the comparison. The number of cases having every returned value should be at least given percentage (a float value between 0.0 and 1.0) of all the compared cases.
* @param includeOthers:
* Should the rest of the attribute values not included due to MinValueUsage or MaxNumUniqueValues filtering be included as an aggregated "Others" value?
* If not empty/null, defines the name used for these other-values.
*/
function ColumnWithMinUsage(df, columnName, newColumnName, maxNumUniqueValues, minValueUsage, includeOthers)
{
let all = df
.GroupBy([])
.Aggregate(["NAllTotal"], ["Count"])
.WithColumn("__Join2", #sql{1});
let minValueUsageEnabled = !IsNullTop(minValueUsage);
let maxNumUniqueValuesEnabled = !IsNullTop(maxNumUniqueValues);
if (minValueUsageEnabled || maxNumUniqueValuesEnabled) {
// Perform column value-based filtering if minValueUsageEnabled or maxNumUniqueValuesEnabled is defined.
let valueColumnName = "__ValueNew";
let filteredValuesColumns = [valueColumnName: columnName];
let filteredValues = df
.GroupBy([columnName]).Aggregate(["Count"], ["Count"]);
if (minValueUsageEnabled) {
filteredValues = filteredValues
.WithColumn("__Join", #sql{1})
.Join(all, ["__Join": "__Join2"], "leftouter")
.WithColumn("Usage", #sql{Column("Count") / Column("NAllTotal")});
filteredValuesColumns = Concat(filteredValuesColumns, ["Usage"]);
}
if (maxNumUniqueValuesEnabled) {
filteredValues = filteredValues
.WithRowNumberColumn("RowNumber", ["Count"], null, [false]);
filteredValuesColumns = Concat(filteredValuesColumns, ["RowNumber"]);
}

filteredValues = filteredValues
.Select(filteredValuesColumns);

// Generate select returning all the accepted values.
let allValues = filteredValues
.(minValueUsageEnabled ? Where(#sql{Column("Usage") >= #expr{minValueUsage}}) : _)
.(maxNumUniqueValuesEnabled ? Where(#sql{Column("RowNumber") <= #expr{maxNumUniqueValues}}) : _)
.Select([valueColumnName, newColumnName: valueColumnName]);

if (!IsNullTop(includeOthers)) {
// If includeOthers is defined, replace original values with the variable defined in includeOthers.
let otherValues = filteredValues
.(minValueUsageEnabled ? Where(#sql{Column("Usage") < #expr{minValueUsage}}) : _)
.(maxNumUniqueValuesEnabled ? Where(#sql{Column("RowNumber") > #expr{maxNumUniqueValues}}) : _)
.WithColumn(newColumnName, #sql{#expr{includeOthers}})
.Select([valueColumnName, newColumnName]);
allValues = allValues.Append(otherValues)
}
df.Join(allValues, [columnName: valueColumnName], "inner")
.RemoveColumns([valueColumnName]);
}
}

// The following example will return only rows containing two of the most common values for Region-column.
//let df = DataTableById(<data table id>).SqlDataFrame;
//df = ColumnWithMinUsage(df, "Region", "_Filtered", 2, null, null);
//df.Collect().ToCsv();

// The following example will return all input rows, but will replace the values of rows whose Region-column
// has a value used by less than 15% of all the rows with a new value: "_Others".
//let df = DataTableById(<data table id>).SqlDataFrame;
//df = ColumnWithMinUsage(df, "Region", "_Filtered", null, 0.15, "_Others");
//df.Collect().ToCsv();
</syntaxhighlight>

=== Export model events and cases ===
<syntaxhighlight lang="typescript" line="1">
function ExportModelEvents(m) {
let attrs = m.EventAttributes;
ToDataFrame(
m.EventLog.Events.Concat(
[Case.Name, Type.Name, ToString(TimeStamp, "yyyy-MM-dd HH:mm:ss.fff")],
{let evt = _; attrs.{let att = _; evt.Attribute(att)}}
),
Concat(
["CaseId", "EventType", "TimeStamp"],
attrs.Name
)
).ToCsv(true);
}

function ExportModelCases(m) {
let attrs = m.CaseAttributes;
ToDataFrame(
m.EventLog.Cases.Concat(
[Name],
{let cas = _; attrs.{let att = _; cas.Attribute(att)}}
),
Concat(
["CaseId"],
attrs.Name
)
).ToCsv(true);
}
</pre>

<pre>
First(Models.Where(Name=="SAP OtC Extended")).EventsDataTable.DataFrame.ToCsv(true)
</pre>

<pre>
First(Models.Where(Name=="SAP OtC Extended")).CasesDataTable.DataFrame.ToCsv(true)
</syntaxhighlight>

=== Calculate all the value usages of a single column for each event in event data table ===
This query could be used, e.g., to find out the maximum resource usage for every resource found in the event data table.<syntaxhighlight lang="typescript" line="1">
function WithUsageColumns(resourceColumn)
{
function WithTotalUsageColumnOfSingleResource(resourceColumn, resourceValue)
{
_
.WithColumn("_Prev", #sql{Lag(Column(resourceColumn), [TimeStamp, EventType], [true, true], [CaseId], 1, null)})
.WithColumn("_UsageDiff", #sql{
CaseWhen(
Column(resourceColumn) == Column("_Prev"), 0,
Column("_Prev") == #expr{resourceValue}, -1,
Column(resourceColumn) == #expr{resourceValue}, 1,
0)
})
.WithColumn(`${resourceValue}_Usage`, #sql{Sum(Column("_UsageDiff"), [TimeStamp, EventType])})
.RemoveColumns(["_Prev", "_UsageDiff"])
}

let sdf = _;
let allValues = sdf.SelectDistinct([resourceColumn]).OrderByColumns([resourceColumn], [true]).Collect().Column(resourceColumn);
allValues.{
let v = _;
sdf = sdf.WithTotalUsageColumnOfSingleResource(resourceColumn, v)
}
sdf
}

let dt = ModelById(<model id>).EventsDataTable;
dt
.SqlDataFrame
.WithUsageColumns(<resource column name>)
.OrderByColumns([dt.ColumnMappings["TimeStamp"]], [true])
.Collect().ToCsv()
</syntaxhighlight>Where:

* <model id> is the id of the model containing event data to be examined.
* <resource column name> is the name of the column in the event data table of the specified model containing the resource being used by that event.

NOTE: This expression uses functionalities that are only supported in Snowflake-based data tables.

=== Create new Snowflake model from filter ===
This script creates a new Snowflake model (and two datatables for cases and events) containing filtered event log from given filter id. The script also works if the model doesn't have a cases datatable.

<syntaxhighlight lang="typescript" line="1">
let filter = FilterById(1); // filter id
let model = filter.model;
let project = model.project;
let nameSuffix = " - " + filter.name + " - " + ToString(Now, "dd-MM-yyyy HH:mm:ss");
let eventsDatatableName = model.EventsDataTable.Name + nameSuffix;
if (eventsDatatableName.length > 440) {
eventsDatatableName = eventsDatatableName.Substring(eventsDatatableName.length - 440);
}
let eventsData = model
.EventsDataTable
.SqlDataFrame
.ApplyFilter(
filter.rules,
model.CasesDataTable?.SqlDataFrame
);
project
.CreateDatatable(eventsDatatableName, #{"Connection": CreateSnowflakeConnection()})
.Import(eventsData);
let modelConfiguration = model.Configuration;
modelConfiguration.DataSource.Events.Set("DataTableName", eventsDatatableName);
if (model.CasesDataTable != null) {
let eventsDataCaseIdColumn = "CaseId_" + ToString(Random());
let casesDatatableName = model.CasesDataTable.Name + nameSuffix;
if (casesDatatableName.length > 440) {
casesDatatableName = casesDatatableName.Substring(casesDatatableName.length - 440);
}
let casesData = model
.CasesDataTable
.SqlDataFrame
.join(
eventsData.SelectDistinct([eventsDataCaseIdColumn: modelConfiguration.DataSource.Events.Columns.CaseId]),
[modelConfiguration.DataSource.Cases.Columns.CaseId: eventsDataCaseIdColumn]
).Select(model.CasesDataTable.ColumnNames);
project
.CreateDatatable(casesDatatableName, #{"Connection": CreateSnowflakeConnection()})
.Import(casesData);
modelConfiguration.DataSource.Cases.Set("DataTableName", casesDatatableName);
}
let modelName = model.Name + nameSuffix;
if (modelName > 440) {
modelName = modelName.Substring(modelName.length - 440);
}
project
.CreateModel(#{
"Name": modelName,
"Description": model.Description,
"Configuration": modelConfiguration
});
return modelName;
</syntaxhighlight>

=== Creating a model consisting of multiple copies of cases in an existing model ===
<syntaxhighlight lang="typescript" line="1">
/**
* @name CreateTestModel
* @description
* Creates a new model (or overwrites an existing) to given target project with given number of
* repetitions of given source model.
* Each repetition will generate "<N>-"-prefix to CaseId-columns, where N equals to the repeat index.
* @param sourceModel
* PA model used for the source data and from where the connection is copied for the target model if a
* new one has to be created.
* @param numRepeats
* Number of times the data in the source model should be repeated in the generated model.
* @param targetProject
* Project in which the target model resides.
* @param targetModelName
* Specifies the name of the test model in the given target project. If a model already exists with
* given name, event and case data in this model will be replaced with the new generated event and
* case data.
* @returns
* Model object of the test model having the newly generated data.
*/
function CreateTestModel(sourceModel, numRepeats, targetProject, targetModelName)
{
let eventsColumnMappings = sourceModel.EventsDataTable.ColumnMappings;
let casesColumnMappings = sourceModel.CasesDataTable.ColumnMappings;
let connection = sourceModel.EventsDataTable.DataSourceConnection;

function CreateResultModel()
{
function GetTable(tableName)
{
let tableConfiguration = #{
"Name": tableName,
"Connection": connection
};
let resultTable = targetProject.DataTableByName(tableName);
if (resultTable == null)
{
resultTable = targetProject.CreateDataTable(tableConfiguration)
.Modify(#{"NameInDataSource": null})
.Synchronize();
}
return resultTable;
}

let eventsTableName = `${targetModelName} - events`;
let casesTableName = `${targetModelName} - cases`;
let targetModel = targetProject.ModelByName(targetModelName);
let eventsTable, casesTable = null;

if (targetModel != null)
{
eventsTable = targetModel.EventsDataTable;
casesTable = targetModel.CasesDataTable;
}
else {
eventsTable = GetTable(eventsTableName);
if (sourceModel.CasesDataTable != null) {
casesTable = GetTable(casesTableName);
}

let timestampMapping = eventsColumnMappings["TimeStamp"];
eventsColumnMappings.Remove("TimeStamp");
eventsColumnMappings.Set("Timestamp", timestampMapping);

let modelConfiguration = #{
"DataSource": #{
"Events":#{
"DataSourceType": "datatable",
"DataTableName": eventsTableName,
"Columns": eventsColumnMappings
}
}
};

if (casesColumnMappings != null) {
modelConfiguration["DataSource"].Set("Cases", #{
"DataSourceType": "datatable",
"DataTableName": casesTableName,
"Columns": casesColumnMappings
});
}

targetModel = targetProject.CreateModel(#{"Name": targetModelName, "Configuration": modelConfiguration});
}

eventsTable.Truncate();
casesTable?.Truncate();

return #{
"TargetModel": targetModel,
"Events": eventsTable,
"Cases": casesTable
};
}

function RepeatNTimes(sourceDf, caseIdColumn, numRepeats)
{
let resultDf = null;
for (let i = 1; i <= numRepeats; ++i) {
let iterationDf = sourceDf
.WithColumn(caseIdColumn, #sql{Concat(#expr{i}, "-", Column(#expr{caseIdColumn}))});
resultDf = resultDf == null ? iterationDf : resultDf.Append(iterationDf);
}
resultDf;
}

let resultModel = CreateResultModel();
let sourceEventDataDf = sourceModel.EventsDataTable.SqlDataFrame;
let resultEventDataDf = RepeatNTimes(sourceEventDataDf, eventsColumnMappings["CaseId"], numRepeats);
resultModel["Events"].Import(resultEventDataDf);

let sourceCaseDataDf = sourceModel.CasesDataTable?.SqlDataFrame;
if (sourceCaseDataDf != null) {
let resultCaseDataDf = RepeatNTimes(sourceCaseDataDf, casesColumnMappings["CaseId"], numRepeats);
resultModel["Cases"].Import(resultCaseDataDf);
}
resultModel["TargetModel"];
}

</syntaxhighlight>Example usage:<blockquote>CreateTestModel(ProjectByName("Project").ModelByName("SAP_OrderToCash - Snowflake"), 3, ProjectByName("TestData"), "TestModel");</blockquote>Creates a new model named "TestModel" (or overwrites old one) into project named "TestData" containing the data from model "SAP_OrderToCash - Snowflake" in project "Project" repeated three times.

=== Analyzing declare patterns found in event log ===

This is an example expression that shows how POSIX-style regular expressions can be used to search for cases in an event log having certain event type patterns [https://www.researchgate.net/publication/277631859_Generating_Event_Logs_Through_the_Simulation_of_Declare_Models declare patterns].

Note: Before use, replace <model id> with a valid model identifier having event data.

<syntaxhighlight lang="typescript" line="1">
let dt = ModelById(<model id>).EventsDataTable;
let sdf = dt.SqlDataFrame.Head(1000);
let caseIdColumn = dt.ColumnMappings["CaseId"], timeStampColumn = dt.ColumnMappings["TimeStamp"], eventTypeColumn = dt.ColumnMappings["EventType"];

let eventTypesDf = sdf
.SelectDistinct([eventTypeColumn])
.OrderByColumns([eventTypeColumn], [true])
.WithRowNumberColumn("Token", [eventTypeColumn])
.WithColumn("Token", #sql{Char(Column("Token") + Unicode("a") - 1)});

sdf = sdf
.Join(eventTypesDf.Select(["_EventType2": eventTypeColumn, "Token"]), [eventTypeColumn: "_EventType2"], "leftouter");

function RespondedExistencePattern(a, b)
{
// [^a]*((a.*b.*)|(b.*a.*))*[^a]*
ToSqlExpression(`Regexp(Column("Trace"), "[^${a}]*((${a}.*${b}.*)|(${b}.*${a}.*))*[^${a}]")`)
}

function ResponsePattern(a, b)
{
// [^a]*(a.*b)*[^a]*
ToSqlExpression(`Regexp(Column("Trace"), "[^${a}]*(${a}.*${b})*[^${a}]")`)
}

function AlternateResponsePattern(a, b)
{
// [^a]*(a[^a]*b[^a]*)*[^a]*
ToSqlExpression(`Regexp(Column("Trace"), "[^${a}]*(${a}[^${a}]*${b}[^${a}]*)*[^${a}]")`)
}

function ChainResponsePattern(a, b)
{
// [^a]*(ab[^a]*)*[^a]*
ToSqlExpression(`Regexp(Column("Trace"), "[^${a}]*(${a}${b}[^${a}]*)*[^${a}]")`)
}

function PrecedencePattern(a, b)
{
// [^b]*(a.*b)*[^b]*
ToSqlExpression(`Regexp(Column("Trace"), "[^${b}]*(${a}.*${b})*[^${b}]")`)
}

let tracesDf = sdf
.GroupBy([caseIdColumn])
.Aggregate(["Trace": "Token"], [#{"Function": "list", "Ordering": [timeStampColumn], "Separator": ""}])
.WithColumn("RespondedExistenceNL", #sql{#expr{RespondedExistencePattern("n", "l")}})
.WithColumn("RespondedExistenceCA", #sql{#expr{RespondedExistencePattern("c", "a")}})
.WithColumn("ResponsePatternNL", #sql{#expr{ResponsePattern("n", "l")}})
.WithColumn("ResponsePatternCA", #sql{#expr{ResponsePattern("c", "a")}})
.WithColumn("AlternateResponsePatternNL", #sql{#expr{AlternateResponsePattern("n", "l")}})
.WithColumn("AlternateResponsePatternCA", #sql{#expr{AlternateResponsePattern("c", "a")}})
.WithColumn("ChainResponsePatternNL", #sql{#expr{ChainResponsePattern("n", "l")}})
.WithColumn("ChainResponsePatternCA", #sql{#expr{ChainResponsePattern("c", "a")}})
.WithColumn("PrecedencePatternNL", #sql{#expr{PrecedencePattern("n", "l")}})
.WithColumn("PrecedencePatternCA", #sql{#expr{PrecedencePattern("c", "a")}})
;

[tracesDf.Collect().ToCsv(), eventTypesDf.Collect().ToCsv()]
</syntaxhighlight>

=== Perform a query and send results as E-mail in a HTML table ===
This example requires two scripts that are both located in the same project:

# Expression-type script to execute (can be, e.g., scheduled to run daily).
# Expression-type script containing the query JSON to use as basis for the e-mail. In this example, this script is named as "Send query as E-mail - query JSON". The contents of this script is just the JSON representation of a query that can be extracted, e.g., from any PA chart view.

Script #1 should contain the following code:<syntaxhighlight lang="typescript">
let replyToAddress = "noreply@test.com";
let recipientsArray = ["test@test.com"];
let queryConfiguration = Project.ScriptByName("Send query as E-mail - query JSON").Code;
let resultDf = Query(ParseJson(queryConfiguration));
let mailBodyHtml = resultDf.Collect().`
<table>
<caption>Example query</caption>
<thead>
<tr>
${StringJoin("", _.Columns.`
<th>${_}</th>
`)}
</tr>
</thead>
<tbody>
${StringJoin("", _.Rows.`
<tr>
${StringJoin("", _.`
<td>${_}</td>
`)}
</tr>
`)}
</tbody>
</table>
`;

SendEmail(#{
"ReplyTo": [replyToAddress],
"To": recipientsArray,
"Subject": "Example query E-mail",
"IsBodyHtml": true,
"Body": mailBodyHtml
});

</syntaxhighlight>NOTE: PA server has to have SMTP server configured. Also, remember to update the values of replyToAddress and recipientsArray before using.

Expression Script Examples

2025-03-18T14:26:41Z

MarHink: /* Perform a query and send results as E-mail in a HTML table */

This page contains script examples written in the QPR ProcessAnalyzer expression language. See how expression scripts can be created in the [[Managing_Scripts#Creating_Script|Workspace]]. For documentation for the syntax, functions and entities can be found from the main page in the [[QPR_ProcessAnalyzer_Wiki#For_Developers|KPI Expression Language]] section.

== Calling Expression Script from Expression ==
Expression scripts can be called from an expression using the [[QPR_ProcessAnalyzer_Objects_in_Expression_Language#Script|Run]] function with the following syntax:
<pre>
let result = ScriptById(123).Run(#{
"parameter1": "value1",
"parameter2": false,
"parameter2": 123.45
})
</pre>

The command waits until the run is completed, and the return value of the called script is returned by the Run function call.

Parameters can be passed to the called script, and the parameters are available as variables in the script. The parameters can contain any type of data.

Expression scripts can also be called from a dashboard. Expressions can be stored to scripts instead of dashboards, which is a way to separate complex expressions from dashboards and allow to reuse expressions across several dashboards.

== Calling SQL Script from Expression ==
SQL script can be called from an expression using the Run function as follows (similar to calling [[#Calling Expression Script from Expression|expression scripts]]):
<pre>
let result = ScriptById(123).Run(#{
"parameter1": "value1",
"parameter2": 321
});
let arrayOfAllReports = result.Keys;
let report1 = result.Report1;
let report2 = result.Report2;
</pre>

SQL scripts can return multiple ''reports'', which are combined to a dictionary, where the key is the name of the report ("sheet name") and value is the report data as a DataFrame. See in the above example, how the reports can be accessed by their name.

== Examples ==
=== Call web service===
Contact to a web service, fetch some data, and store it to a datatable.

<syntaxhighlight lang="typescript" line>
let datatableName = "Web Service Data";
let webServiceData = CallWebService(
#{"Address": "https://processanalyzer.onqpr.com/qprpa/api/serverinfo"}
);

let targetDatatable = Project.Datatables.Where(name==datatableName);
if (Count(targetDatatable) == 0) {
targetDatatable = Project.CreateDatatable(datatableName)
.AddColumn("Setting name", "String")
.AddColumn("Setting value", "String")
.AddColumn("Data read", "DateTime");
} else {
targetDatatable = targetDatatable[0];
}

let currentTime = Now;
let dataAsDf = ToDataFrame(
webServiceData.keys.{
let key = _;
[key, webServiceData[key], currentTime];
},
["Setting name", "Setting value", "Data read"]
);
targetDatatable.Import(dataAsDf, #{"Append":true});
WriteLog(`${CountTop(dataAsDf.Rows)} rows written to datatable`);
</syntaxhighlight>

=== Store data to datatable ===
Get all models in the system and store them to a datatable.

<syntaxhighlight lang="typescript" line>
let newDatatable = Project
.CreateDatatable("Models list " + ToString(Now, "dd.MM.yyyy HH:mm:ss"))
.AddColumn("Model name", "String")
.AddColumn("Project name", "String")
.AddColumn("Created time", "DateTime")
.AddColumn("Cases", "Integer");
let startTime = Now;
let modelsData = ToDataFrame(
Models.([Name, Project.Name, CreatedDate, NCases]),
["Model name", "Project name", "Created time", "Cases"]
);
WriteLog(`Listing models took ${(Now - startTime).TotalSeconds.Round(2)} seconds.`);
newDatatable.Import(modelsData);
WriteLog(`Datatable ${newDatatable.Id} created.`);
</syntaxhighlight>

=== Convert datatable column data ===
This script can be used to convert a single column into numerical data type. To use the script, you need to setup the following in the beginning of the script:
* Project name where the datatable is located.
* Datatable name
* Name of the column to be converted

Note that the conversion fails, if there is data that cannot be converted into numerical format. The conversion assumes that period (.) is used as the decimal point.

<syntaxhighlight lang="typescript" line>
let projectName = "New Project";
let datatableName = "qpr processanalyzer events";
let columnName = "Event order in case";

let project = (Projects.Where(Name==projectName))[0];
let datatable = (project.Datatables.Where(Name==datatableName))[0];
DatatableById(datatable.Id).DataFrame
.SetColumns([
columnName: () => {
let data = Column(columnName);
if (data == null) {
null;
} else {
ToFloat(data);
}
}
])
.Persist(datatable.Name, ["ProjectId": project.Id, "Append": false]);
</syntaxhighlight>

Instead of converting to numeric (with the ''ToFloat'' function), data can be converted into string using the ''ToString'' function.

=== Show DataFrame as HTML table ===

This script defines a function to show dataframe as a HTML table, and uses the function for a literal dataframe.

<syntaxhighlight lang="typescript" line>
function dataframeToHtmlTable(df) {
return
`<table>
<tr>
${StringJoin("\r\n\t\t", + df.columns.`<th>${_}</th>`)}
</tr>
${StringJoin("", df.Rows.(
"\r\n\t<tr>" + StringJoin("", _.`\r\n\t\t<td>${ToString(_)}</td>`) + "\r\n\t</tr>"
))}
</table>`
}

let data = ToDataFrame(
[
["one", "two", "three"],
["four", "five", "six"],
["seven", "eight", "nine"]
],
["Column 1", "Column 2", "Column 3"]
);

return dataframeToHtmlTable(data);
</syntaxhighlight>

=== Copy local datatables to Snowflake ===

<syntaxhighlight lang="typescript" line>
// Copies all datatables in a project to another project including datatable contents.
// Usage instructions:
// 1. Create expression script in the project from where you want to copy the datatables.
// 2. Create a new project named as "<name of the project to be moved> - Snowflake". New datatables will be created here. E.g., when moving project named "SAP_OrderToCash", the target project should be named as "SAP_OrderToCash - Snowflake".
// 3. Run the script.
// NOTE: Columns of type "Any" will be created as "String"-columns in Snowflake, thus it is recommended that actual data types are set for the tables prior to the move.

let sourceProject = Project;
let sourceProjectName = Project.Name;
let targetProjectName = `${sourceProjectName} - Snowflake`;
let targetProject = First(Projects.Where(Name == targetProjectName));
if (IsNull(targetProject)) {
WriteLog(`Unable to find target project named "${targetProjectName}". Aborting operation.`);
return;
}
let dts = sourceProject.DataTables;
WriteLog(`Copying all ${CountTop(dts)} data tables found in project "${sourceProject.Name}" (id: ${sourceProject.Id}) to Snowflake in project "${targetProject.Name}" (id: ${targetProject.Id})`);
dts.{
let sourceDt = _;
WriteLog(`Starting to copy data table "${Name}" (id: ${Id}) having ${NRows} rows and ${NColumns} columns.`);
let targetDt;
targetDt = targetProject.DatatableByName(sourceDt.Name);
if (targetDt == null) {
targetDt = targetProject.CreateDataTable(sourceDt.Name, #{"Connection": CreateSnowflakeConnection(#{"ProjectId": targetProject.Id})});
targetDt.Import(sourceDt.SqlDataFrame);
WriteLog(`Finished copying data table "${Name}" (id: ${Id}) to table "${targetDt.Name}" (id: ${targetDt.Id})`);
} else {
WriteLog(`Datatable already exist "${Name}" (id: ${Id}) to table "${targetDt.Name}" (id: ${targetDt.Id})`);
}
}
WriteLog(`Finished copying all the data tables found in project "${sourceProject.Name}" (id: ${sourceProject.Id}) to Snowflake in project "${targetProject.Name}" (id: ${targetProject.Id})`);
</syntaxhighlight>

If you don't need to copy the data but only create the Snowflake datatables with columns, you can change the line 22 to
<syntaxhighlight>
targetDt.Import(sourceDt.SqlDataFrame.head(0));
</syntaxhighlight>

=== Copy single datatable to Snowflake ===
This script creates a copy of a single datatable to Snowflake. Replace the ''<tableId1>'' with the id of the source datatable.
<syntaxhighlight lang="typescript" line>
function CopyDataTableToSnowflake(dataTableId)
{
let sourceDt = DataTableById(dataTableId);
sourceDt.SqlDataFrame.Persist(`${sourceDt.Name} - Snowflake`, #{"Append": false, "Connection": CreateSnowflakeConnection(#{"ProjectId": sourceDt.Project.Id})});
}
CopyDataTableToSnowflake(<tableId1>);
</syntaxhighlight>

=== Create a copy of a data table that has all Any-type columns changed to String-type columns ===
<syntaxhighlight lang="typescript" line>
function ConvertAnyDataTypesToStringsToNewTable(dataTableId)
{
let dt = DataTableById(dataTableId);
let sdf = dt.SqlDataFrame;
let cts = dt.ColumnTypes;
cts.{
let ct = _;
if (ct.DataType == "Any") {
let n = ct.Name;
sdf = sdf.WithColumn(ct.Name, #sql{Cast(Column(Variable("n")), "ShortString")});
}
};
sdf.Persist(`${dt.Name} - Converted`, #{"Append": false, "ProjectId": dt.Project.Id});
}
ConvertAnyDataTypesToStringsToNewTable(<dataTableId>);
</syntaxhighlight>

=== Query number of rows in given data table having a datetime value in given year grouped by month and return resulting table as CSV ===
SqlDataFrame is used in order to prevent loading the whole datatable into memory first. Filtering is performed as first operation in order to minimize the amount of required work for the data source of the data table.<syntaxhighlight lang="typescript" line="1">
DataTableById(<data table id>)
.SqlDataFrame
.Where(#sql{2014 == Year(Column("Start Time"))})
.WithColumn("Month", #sql{Month(Column("Start Time"))})
.GroupBy(["Month"]).Aggregate(["Count"], ["Count"])
.OrderByColumns(["Month"], [true])
.Collect().ToCsv();
</syntaxhighlight>

=== Function for filtering SqlDataFrame by removing rows having, or replacing, the most infrequently occurring column values ===
<syntaxhighlight lang="typescript" line="1">
/***
* @name ColumnWithMinUsage
* @descripion
* Generic function that can be used to filter out the most infrequently occurring attribute values or replace their Values
* with given common value.
* @param df:
* DataFrame to operate on.
* @param columnName:
* Name of the column to be filtered.
* @param newColumnName:
* Name of the column that will contain the new value of the original column after filtering (if includeOthers was applied).
* @param maxNumUniqueValues:
* Maximum number of unique values to include into the comparison for each attribute column. If the amount of unique values for any attribute exceeds this value, only given number of attributes are included that have the highest usage.
* @param minValueUsage:
* Minimum total usage of a value included into the comparison. The number of cases having every returned value should be at least given percentage (a float value between 0.0 and 1.0) of all the compared cases.
* @param includeOthers:
* Should the rest of the attribute values not included due to MinValueUsage or MaxNumUniqueValues filtering be included as an aggregated "Others" value?
* If not empty/null, defines the name used for these other-values.
*/
function ColumnWithMinUsage(df, columnName, newColumnName, maxNumUniqueValues, minValueUsage, includeOthers)
{
let all = df
.GroupBy([])
.Aggregate(["NAllTotal"], ["Count"])
.WithColumn("__Join2", #sql{1});
let minValueUsageEnabled = !IsNullTop(minValueUsage);
let maxNumUniqueValuesEnabled = !IsNullTop(maxNumUniqueValues);
if (minValueUsageEnabled || maxNumUniqueValuesEnabled) {
// Perform column value-based filtering if minValueUsageEnabled or maxNumUniqueValuesEnabled is defined.
let valueColumnName = "__ValueNew";
let filteredValuesColumns = [valueColumnName: columnName];
let filteredValues = df
.GroupBy([columnName]).Aggregate(["Count"], ["Count"]);
if (minValueUsageEnabled) {
filteredValues = filteredValues
.WithColumn("__Join", #sql{1})
.Join(all, ["__Join": "__Join2"], "leftouter")
.WithColumn("Usage", #sql{Column("Count") / Column("NAllTotal")});
filteredValuesColumns = Concat(filteredValuesColumns, ["Usage"]);
}
if (maxNumUniqueValuesEnabled) {
filteredValues = filteredValues
.WithRowNumberColumn("RowNumber", ["Count"], null, [false]);
filteredValuesColumns = Concat(filteredValuesColumns, ["RowNumber"]);
}

filteredValues = filteredValues
.Select(filteredValuesColumns);

// Generate select returning all the accepted values.
let allValues = filteredValues
.(minValueUsageEnabled ? Where(#sql{Column("Usage") >= #expr{minValueUsage}}) : _)
.(maxNumUniqueValuesEnabled ? Where(#sql{Column("RowNumber") <= #expr{maxNumUniqueValues}}) : _)
.Select([valueColumnName, newColumnName: valueColumnName]);

if (!IsNullTop(includeOthers)) {
// If includeOthers is defined, replace original values with the variable defined in includeOthers.
let otherValues = filteredValues
.(minValueUsageEnabled ? Where(#sql{Column("Usage") < #expr{minValueUsage}}) : _)
.(maxNumUniqueValuesEnabled ? Where(#sql{Column("RowNumber") > #expr{maxNumUniqueValues}}) : _)
.WithColumn(newColumnName, #sql{#expr{includeOthers}})
.Select([valueColumnName, newColumnName]);
allValues = allValues.Append(otherValues)
}
df.Join(allValues, [columnName: valueColumnName], "inner")
.RemoveColumns([valueColumnName]);
}
}

// The following example will return only rows containing two of the most common values for Region-column.
//let df = DataTableById(<data table id>).SqlDataFrame;
//df = ColumnWithMinUsage(df, "Region", "_Filtered", 2, null, null);
//df.Collect().ToCsv();

// The following example will return all input rows, but will replace the values of rows whose Region-column
// has a value used by less than 15% of all the rows with a new value: "_Others".
//let df = DataTableById(<data table id>).SqlDataFrame;
//df = ColumnWithMinUsage(df, "Region", "_Filtered", null, 0.15, "_Others");
//df.Collect().ToCsv();
</syntaxhighlight>

=== Export model events and cases ===
<syntaxhighlight lang="typescript" line="1">
function ExportModelEvents(m) {
let attrs = m.EventAttributes;
ToDataFrame(
m.EventLog.Events.Concat(
[Case.Name, Type.Name, ToString(TimeStamp, "yyyy-MM-dd HH:mm:ss.fff")],
{let evt = _; attrs.{let att = _; evt.Attribute(att)}}
),
Concat(
["CaseId", "EventType", "TimeStamp"],
attrs.Name
)
).ToCsv(true);
}

function ExportModelCases(m) {
let attrs = m.CaseAttributes;
ToDataFrame(
m.EventLog.Cases.Concat(
[Name],
{let cas = _; attrs.{let att = _; cas.Attribute(att)}}
),
Concat(
["CaseId"],
attrs.Name
)
).ToCsv(true);
}
</pre>

<pre>
First(Models.Where(Name=="SAP OtC Extended")).EventsDataTable.DataFrame.ToCsv(true)
</pre>

<pre>
First(Models.Where(Name=="SAP OtC Extended")).CasesDataTable.DataFrame.ToCsv(true)
</syntaxhighlight>

=== Calculate all the value usages of a single column for each event in event data table ===
This query could be used, e.g., to find out the maximum resource usage for every resource found in the event data table.<syntaxhighlight lang="typescript" line="1">
function WithUsageColumns(resourceColumn)
{
function WithTotalUsageColumnOfSingleResource(resourceColumn, resourceValue)
{
_
.WithColumn("_Prev", #sql{Lag(Column(resourceColumn), [TimeStamp, EventType], [true, true], [CaseId], 1, null)})
.WithColumn("_UsageDiff", #sql{
CaseWhen(
Column(resourceColumn) == Column("_Prev"), 0,
Column("_Prev") == #expr{resourceValue}, -1,
Column(resourceColumn) == #expr{resourceValue}, 1,
0)
})
.WithColumn(`${resourceValue}_Usage`, #sql{Sum(Column("_UsageDiff"), [TimeStamp, EventType])})
.RemoveColumns(["_Prev", "_UsageDiff"])
}

let sdf = _;
let allValues = sdf.SelectDistinct([resourceColumn]).OrderByColumns([resourceColumn], [true]).Collect().Column(resourceColumn);
allValues.{
let v = _;
sdf = sdf.WithTotalUsageColumnOfSingleResource(resourceColumn, v)
}
sdf
}

let dt = ModelById(<model id>).EventsDataTable;
dt
.SqlDataFrame
.WithUsageColumns(<resource column name>)
.OrderByColumns([dt.ColumnMappings["TimeStamp"]], [true])
.Collect().ToCsv()
</syntaxhighlight>Where:

* <model id> is the id of the model containing event data to be examined.
* <resource column name> is the name of the column in the event data table of the specified model containing the resource being used by that event.

NOTE: This expression uses functionalities that are only supported in Snowflake-based data tables.

=== Create new Snowflake model from filter ===
This script creates a new Snowflake model (and two datatables for cases and events) containing filtered event log from given filter id. The script also works if the model doesn't have a cases datatable.

<syntaxhighlight lang="typescript" line="1">
let filter = FilterById(1); // filter id
let model = filter.model;
let project = model.project;
let nameSuffix = " - " + filter.name + " - " + ToString(Now, "dd-MM-yyyy HH:mm:ss");
let eventsDatatableName = model.EventsDataTable.Name + nameSuffix;
if (eventsDatatableName.length > 440) {
eventsDatatableName = eventsDatatableName.Substring(eventsDatatableName.length - 440);
}
let eventsData = model
.EventsDataTable
.SqlDataFrame
.ApplyFilter(
filter.rules,
model.CasesDataTable?.SqlDataFrame
);
project
.CreateDatatable(eventsDatatableName, #{"Connection": CreateSnowflakeConnection()})
.Import(eventsData);
let modelConfiguration = model.Configuration;
modelConfiguration.DataSource.Events.Set("DataTableName", eventsDatatableName);
if (model.CasesDataTable != null) {
let eventsDataCaseIdColumn = "CaseId_" + ToString(Random());
let casesDatatableName = model.CasesDataTable.Name + nameSuffix;
if (casesDatatableName.length > 440) {
casesDatatableName = casesDatatableName.Substring(casesDatatableName.length - 440);
}
let casesData = model
.CasesDataTable
.SqlDataFrame
.join(
eventsData.SelectDistinct([eventsDataCaseIdColumn: modelConfiguration.DataSource.Events.Columns.CaseId]),
[modelConfiguration.DataSource.Cases.Columns.CaseId: eventsDataCaseIdColumn]
).Select(model.CasesDataTable.ColumnNames);
project
.CreateDatatable(casesDatatableName, #{"Connection": CreateSnowflakeConnection()})
.Import(casesData);
modelConfiguration.DataSource.Cases.Set("DataTableName", casesDatatableName);
}
let modelName = model.Name + nameSuffix;
if (modelName > 440) {
modelName = modelName.Substring(modelName.length - 440);
}
project
.CreateModel(#{
"Name": modelName,
"Description": model.Description,
"Configuration": modelConfiguration
});
return modelName;
</syntaxhighlight>

=== Creating a model consisting of multiple copies of cases in an existing model ===
<syntaxhighlight lang="typescript" line="1">
/**
* @name CreateTestModel
* @description
* Creates a new model (or overwrites an existing) to given target project with given number of
* repetitions of given source model.
* Each repetition will generate "<N>-"-prefix to CaseId-columns, where N equals to the repeat index.
* @param sourceModel
* PA model used for the source data and from where the connection is copied for the target model if a
* new one has to be created.
* @param numRepeats
* Number of times the data in the source model should be repeated in the generated model.
* @param targetProject
* Project in which the target model resides.
* @param targetModelName
* Specifies the name of the test model in the given target project. If a model already exists with
* given name, event and case data in this model will be replaced with the new generated event and
* case data.
* @returns
* Model object of the test model having the newly generated data.
*/
function CreateTestModel(sourceModel, numRepeats, targetProject, targetModelName)
{
let eventsColumnMappings = sourceModel.EventsDataTable.ColumnMappings;
let casesColumnMappings = sourceModel.CasesDataTable.ColumnMappings;
let connection = sourceModel.EventsDataTable.DataSourceConnection;

function CreateResultModel()
{
function GetTable(tableName)
{
let tableConfiguration = #{
"Name": tableName,
"Connection": connection
};
let resultTable = targetProject.DataTableByName(tableName);
if (resultTable == null)
{
resultTable = targetProject.CreateDataTable(tableConfiguration)
.Modify(#{"NameInDataSource": null})
.Synchronize();
}
return resultTable;
}

let eventsTableName = `${targetModelName} - events`;
let casesTableName = `${targetModelName} - cases`;
let targetModel = targetProject.ModelByName(targetModelName);
let eventsTable, casesTable = null;

if (targetModel != null)
{
eventsTable = targetModel.EventsDataTable;
casesTable = targetModel.CasesDataTable;
}
else {
eventsTable = GetTable(eventsTableName);
if (sourceModel.CasesDataTable != null) {
casesTable = GetTable(casesTableName);
}

let timestampMapping = eventsColumnMappings["TimeStamp"];
eventsColumnMappings.Remove("TimeStamp");
eventsColumnMappings.Set("Timestamp", timestampMapping);

let modelConfiguration = #{
"DataSource": #{
"Events":#{
"DataSourceType": "datatable",
"DataTableName": eventsTableName,
"Columns": eventsColumnMappings
}
}
};

if (casesColumnMappings != null) {
modelConfiguration["DataSource"].Set("Cases", #{
"DataSourceType": "datatable",
"DataTableName": casesTableName,
"Columns": casesColumnMappings
});
}

targetModel = targetProject.CreateModel(#{"Name": targetModelName, "Configuration": modelConfiguration});
}

eventsTable.Truncate();
casesTable?.Truncate();

return #{
"TargetModel": targetModel,
"Events": eventsTable,
"Cases": casesTable
};
}

function RepeatNTimes(sourceDf, caseIdColumn, numRepeats)
{
let resultDf = null;
for (let i = 1; i <= numRepeats; ++i) {
let iterationDf = sourceDf
.WithColumn(caseIdColumn, #sql{Concat(#expr{i}, "-", Column(#expr{caseIdColumn}))});
resultDf = resultDf == null ? iterationDf : resultDf.Append(iterationDf);
}
resultDf;
}

let resultModel = CreateResultModel();
let sourceEventDataDf = sourceModel.EventsDataTable.SqlDataFrame;
let resultEventDataDf = RepeatNTimes(sourceEventDataDf, eventsColumnMappings["CaseId"], numRepeats);
resultModel["Events"].Import(resultEventDataDf);

let sourceCaseDataDf = sourceModel.CasesDataTable?.SqlDataFrame;
if (sourceCaseDataDf != null) {
let resultCaseDataDf = RepeatNTimes(sourceCaseDataDf, casesColumnMappings["CaseId"], numRepeats);
resultModel["Cases"].Import(resultCaseDataDf);
}
resultModel["TargetModel"];
}

</syntaxhighlight>Example usage:<blockquote>CreateTestModel(ProjectByName("Project").ModelByName("SAP_OrderToCash - Snowflake"), 3, ProjectByName("TestData"), "TestModel");</blockquote>Creates a new model named "TestModel" (or overwrites old one) into project named "TestData" containing the data from model "SAP_OrderToCash - Snowflake" in project "Project" repeated three times.

=== Analyzing declare patterns found in event log ===

This is an example expression that shows how POSIX-style regular expressions can be used to search for cases in an event log having certain event type patterns [https://www.researchgate.net/publication/277631859_Generating_Event_Logs_Through_the_Simulation_of_Declare_Models declare patterns].

Note: Before use, replace <model id> with a valid model identifier having event data.

<syntaxhighlight lang="typescript" line="1">
let dt = ModelById(<model id>).EventsDataTable;
let sdf = dt.SqlDataFrame.Head(1000);
let caseIdColumn = dt.ColumnMappings["CaseId"], timeStampColumn = dt.ColumnMappings["TimeStamp"], eventTypeColumn = dt.ColumnMappings["EventType"];

let eventTypesDf = sdf
.SelectDistinct([eventTypeColumn])
.OrderByColumns([eventTypeColumn], [true])
.WithRowNumberColumn("Token", [eventTypeColumn])
.WithColumn("Token", #sql{Char(Column("Token") + Unicode("a") - 1)});

sdf = sdf
.Join(eventTypesDf.Select(["_EventType2": eventTypeColumn, "Token"]), [eventTypeColumn: "_EventType2"], "leftouter");

function RespondedExistencePattern(a, b)
{
// [^a]*((a.*b.*)|(b.*a.*))*[^a]*
ToSqlExpression(`Regexp(Column("Trace"), "[^${a}]*((${a}.*${b}.*)|(${b}.*${a}.*))*[^${a}]")`)
}

function ResponsePattern(a, b)
{
// [^a]*(a.*b)*[^a]*
ToSqlExpression(`Regexp(Column("Trace"), "[^${a}]*(${a}.*${b})*[^${a}]")`)
}

function AlternateResponsePattern(a, b)
{
// [^a]*(a[^a]*b[^a]*)*[^a]*
ToSqlExpression(`Regexp(Column("Trace"), "[^${a}]*(${a}[^${a}]*${b}[^${a}]*)*[^${a}]")`)
}

function ChainResponsePattern(a, b)
{
// [^a]*(ab[^a]*)*[^a]*
ToSqlExpression(`Regexp(Column("Trace"), "[^${a}]*(${a}${b}[^${a}]*)*[^${a}]")`)
}

function PrecedencePattern(a, b)
{
// [^b]*(a.*b)*[^b]*
ToSqlExpression(`Regexp(Column("Trace"), "[^${b}]*(${a}.*${b})*[^${b}]")`)
}

let tracesDf = sdf
.GroupBy([caseIdColumn])
.Aggregate(["Trace": "Token"], [#{"Function": "list", "Ordering": [timeStampColumn], "Separator": ""}])
.WithColumn("RespondedExistenceNL", #sql{#expr{RespondedExistencePattern("n", "l")}})
.WithColumn("RespondedExistenceCA", #sql{#expr{RespondedExistencePattern("c", "a")}})
.WithColumn("ResponsePatternNL", #sql{#expr{ResponsePattern("n", "l")}})
.WithColumn("ResponsePatternCA", #sql{#expr{ResponsePattern("c", "a")}})
.WithColumn("AlternateResponsePatternNL", #sql{#expr{AlternateResponsePattern("n", "l")}})
.WithColumn("AlternateResponsePatternCA", #sql{#expr{AlternateResponsePattern("c", "a")}})
.WithColumn("ChainResponsePatternNL", #sql{#expr{ChainResponsePattern("n", "l")}})
.WithColumn("ChainResponsePatternCA", #sql{#expr{ChainResponsePattern("c", "a")}})
.WithColumn("PrecedencePatternNL", #sql{#expr{PrecedencePattern("n", "l")}})
.WithColumn("PrecedencePatternCA", #sql{#expr{PrecedencePattern("c", "a")}})
;

[tracesDf.Collect().ToCsv(), eventTypesDf.Collect().ToCsv()]
</syntaxhighlight>

=== Perform a query and send results as E-mail in a HTML table ===
This example requires two scripts that are both located in the same project:

# Expression-type script to execute (can be, e.g., scheduled to run daily).
# Expression-type script containing the query JSON to use as basis for the e-mail. In this example, this script is named as "Send query as E-mail - query JSON". The contents of this script is just the JSON representation of a query that can be extracted, e.g., from any PA chart view.

Script #1 should contain the following code:<syntaxhighlight lang="typescript">
let replyToAddress = "noreply@test.com";
let recipientsArray = ["test@test.com"];
let queryConfiguration = Project.ScriptByName("Send query as E-mail - query JSON").Code;
let resultDf = Query(ParseJson(queryConfiguration));
let mailBodyHtml = resultDf.Collect().`
<table>
<caption>Example query</caption>
<thead>
<tr>
${StringJoin("", _.Columns.`
<th>${_}</th>
`)}
</tr>
</thead>
<tbody>
${StringJoin("", _.Rows.`
<tr>
${StringJoin("", _.`
<td>${_}</td>
`)}
</tr>
`)}
</tbody>
</table>
`;

SendEmail(#{
"ReplyTo": [replyToAddress],
"To": recipientsArray,
"Subject": "Example query E-mail",
"IsBodyHtml": true,
"Body": mailBodyHtml
});

</syntaxhighlight>

Expression Script Examples

2025-03-18T14:17:10Z

MarHink:

This page contains script examples written in the QPR ProcessAnalyzer expression language. See how expression scripts can be created in the [[Managing_Scripts#Creating_Script|Workspace]]. For documentation for the syntax, functions and entities can be found from the main page in the [[QPR_ProcessAnalyzer_Wiki#For_Developers|KPI Expression Language]] section.

== Calling Expression Script from Expression ==
Expression scripts can be called from an expression using the [[QPR_ProcessAnalyzer_Objects_in_Expression_Language#Script|Run]] function with the following syntax:
<pre>
let result = ScriptById(123).Run(#{
"parameter1": "value1",
"parameter2": false,
"parameter2": 123.45
})
</pre>

The command waits until the run is completed, and the return value of the called script is returned by the Run function call.

Parameters can be passed to the called script, and the parameters are available as variables in the script. The parameters can contain any type of data.

Expression scripts can also be called from a dashboard. Expressions can be stored to scripts instead of dashboards, which is a way to separate complex expressions from dashboards and allow to reuse expressions across several dashboards.

== Calling SQL Script from Expression ==
SQL script can be called from an expression using the Run function as follows (similar to calling [[#Calling Expression Script from Expression|expression scripts]]):
<pre>
let result = ScriptById(123).Run(#{
"parameter1": "value1",
"parameter2": 321
});
let arrayOfAllReports = result.Keys;
let report1 = result.Report1;
let report2 = result.Report2;
</pre>

SQL scripts can return multiple ''reports'', which are combined to a dictionary, where the key is the name of the report ("sheet name") and value is the report data as a DataFrame. See in the above example, how the reports can be accessed by their name.

== Examples ==
=== Call web service===
Contact to a web service, fetch some data, and store it to a datatable.

<syntaxhighlight lang="typescript" line>
let datatableName = "Web Service Data";
let webServiceData = CallWebService(
#{"Address": "https://processanalyzer.onqpr.com/qprpa/api/serverinfo"}
);

let targetDatatable = Project.Datatables.Where(name==datatableName);
if (Count(targetDatatable) == 0) {
targetDatatable = Project.CreateDatatable(datatableName)
.AddColumn("Setting name", "String")
.AddColumn("Setting value", "String")
.AddColumn("Data read", "DateTime");
} else {
targetDatatable = targetDatatable[0];
}

let currentTime = Now;
let dataAsDf = ToDataFrame(
webServiceData.keys.{
let key = _;
[key, webServiceData[key], currentTime];
},
["Setting name", "Setting value", "Data read"]
);
targetDatatable.Import(dataAsDf, #{"Append":true});
WriteLog(`${CountTop(dataAsDf.Rows)} rows written to datatable`);
</syntaxhighlight>

=== Store data to datatable ===
Get all models in the system and store them to a datatable.

<syntaxhighlight lang="typescript" line>
let newDatatable = Project
.CreateDatatable("Models list " + ToString(Now, "dd.MM.yyyy HH:mm:ss"))
.AddColumn("Model name", "String")
.AddColumn("Project name", "String")
.AddColumn("Created time", "DateTime")
.AddColumn("Cases", "Integer");
let startTime = Now;
let modelsData = ToDataFrame(
Models.([Name, Project.Name, CreatedDate, NCases]),
["Model name", "Project name", "Created time", "Cases"]
);
WriteLog(`Listing models took ${(Now - startTime).TotalSeconds.Round(2)} seconds.`);
newDatatable.Import(modelsData);
WriteLog(`Datatable ${newDatatable.Id} created.`);
</syntaxhighlight>

=== Convert datatable column data ===
This script can be used to convert a single column into numerical data type. To use the script, you need to setup the following in the beginning of the script:
* Project name where the datatable is located.
* Datatable name
* Name of the column to be converted

Note that the conversion fails, if there is data that cannot be converted into numerical format. The conversion assumes that period (.) is used as the decimal point.

<syntaxhighlight lang="typescript" line>
let projectName = "New Project";
let datatableName = "qpr processanalyzer events";
let columnName = "Event order in case";

let project = (Projects.Where(Name==projectName))[0];
let datatable = (project.Datatables.Where(Name==datatableName))[0];
DatatableById(datatable.Id).DataFrame
.SetColumns([
columnName: () => {
let data = Column(columnName);
if (data == null) {
null;
} else {
ToFloat(data);
}
}
])
.Persist(datatable.Name, ["ProjectId": project.Id, "Append": false]);
</syntaxhighlight>

Instead of converting to numeric (with the ''ToFloat'' function), data can be converted into string using the ''ToString'' function.

=== Show DataFrame as HTML table ===

This script defines a function to show dataframe as a HTML table, and uses the function for a literal dataframe.

<syntaxhighlight lang="typescript" line>
function dataframeToHtmlTable(df) {
return
`<table>
<tr>
${StringJoin("\r\n\t\t", + df.columns.`<th>${_}</th>`)}
</tr>
${StringJoin("", df.Rows.(
"\r\n\t<tr>" + StringJoin("", _.`\r\n\t\t<td>${ToString(_)}</td>`) + "\r\n\t</tr>"
))}
</table>`
}

let data = ToDataFrame(
[
["one", "two", "three"],
["four", "five", "six"],
["seven", "eight", "nine"]
],
["Column 1", "Column 2", "Column 3"]
);

return dataframeToHtmlTable(data);
</syntaxhighlight>

=== Copy local datatables to Snowflake ===

<syntaxhighlight lang="typescript" line>
// Copies all datatables in a project to another project including datatable contents.
// Usage instructions:
// 1. Create expression script in the project from where you want to copy the datatables.
// 2. Create a new project named as "<name of the project to be moved> - Snowflake". New datatables will be created here. E.g., when moving project named "SAP_OrderToCash", the target project should be named as "SAP_OrderToCash - Snowflake".
// 3. Run the script.
// NOTE: Columns of type "Any" will be created as "String"-columns in Snowflake, thus it is recommended that actual data types are set for the tables prior to the move.

let sourceProject = Project;
let sourceProjectName = Project.Name;
let targetProjectName = `${sourceProjectName} - Snowflake`;
let targetProject = First(Projects.Where(Name == targetProjectName));
if (IsNull(targetProject)) {
WriteLog(`Unable to find target project named "${targetProjectName}". Aborting operation.`);
return;
}
let dts = sourceProject.DataTables;
WriteLog(`Copying all ${CountTop(dts)} data tables found in project "${sourceProject.Name}" (id: ${sourceProject.Id}) to Snowflake in project "${targetProject.Name}" (id: ${targetProject.Id})`);
dts.{
let sourceDt = _;
WriteLog(`Starting to copy data table "${Name}" (id: ${Id}) having ${NRows} rows and ${NColumns} columns.`);
let targetDt;
targetDt = targetProject.DatatableByName(sourceDt.Name);
if (targetDt == null) {
targetDt = targetProject.CreateDataTable(sourceDt.Name, #{"Connection": CreateSnowflakeConnection(#{"ProjectId": targetProject.Id})});
targetDt.Import(sourceDt.SqlDataFrame);
WriteLog(`Finished copying data table "${Name}" (id: ${Id}) to table "${targetDt.Name}" (id: ${targetDt.Id})`);
} else {
WriteLog(`Datatable already exist "${Name}" (id: ${Id}) to table "${targetDt.Name}" (id: ${targetDt.Id})`);
}
}
WriteLog(`Finished copying all the data tables found in project "${sourceProject.Name}" (id: ${sourceProject.Id}) to Snowflake in project "${targetProject.Name}" (id: ${targetProject.Id})`);
</syntaxhighlight>

If you don't need to copy the data but only create the Snowflake datatables with columns, you can change the line 22 to
<syntaxhighlight>
targetDt.Import(sourceDt.SqlDataFrame.head(0));
</syntaxhighlight>

=== Copy single datatable to Snowflake ===
This script creates a copy of a single datatable to Snowflake. Replace the ''<tableId1>'' with the id of the source datatable.
<syntaxhighlight lang="typescript" line>
function CopyDataTableToSnowflake(dataTableId)
{
let sourceDt = DataTableById(dataTableId);
sourceDt.SqlDataFrame.Persist(`${sourceDt.Name} - Snowflake`, #{"Append": false, "Connection": CreateSnowflakeConnection(#{"ProjectId": sourceDt.Project.Id})});
}
CopyDataTableToSnowflake(<tableId1>);
</syntaxhighlight>

=== Create a copy of a data table that has all Any-type columns changed to String-type columns ===
<syntaxhighlight lang="typescript" line>
function ConvertAnyDataTypesToStringsToNewTable(dataTableId)
{
let dt = DataTableById(dataTableId);
let sdf = dt.SqlDataFrame;
let cts = dt.ColumnTypes;
cts.{
let ct = _;
if (ct.DataType == "Any") {
let n = ct.Name;
sdf = sdf.WithColumn(ct.Name, #sql{Cast(Column(Variable("n")), "ShortString")});
}
};
sdf.Persist(`${dt.Name} - Converted`, #{"Append": false, "ProjectId": dt.Project.Id});
}
ConvertAnyDataTypesToStringsToNewTable(<dataTableId>);
</syntaxhighlight>

=== Query number of rows in given data table having a datetime value in given year grouped by month and return resulting table as CSV ===
SqlDataFrame is used in order to prevent loading the whole datatable into memory first. Filtering is performed as first operation in order to minimize the amount of required work for the data source of the data table.<syntaxhighlight lang="typescript" line="1">
DataTableById(<data table id>)
.SqlDataFrame
.Where(#sql{2014 == Year(Column("Start Time"))})
.WithColumn("Month", #sql{Month(Column("Start Time"))})
.GroupBy(["Month"]).Aggregate(["Count"], ["Count"])
.OrderByColumns(["Month"], [true])
.Collect().ToCsv();
</syntaxhighlight>

=== Function for filtering SqlDataFrame by removing rows having, or replacing, the most infrequently occurring column values ===
<syntaxhighlight lang="typescript" line="1">
/***
* @name ColumnWithMinUsage
* @descripion
* Generic function that can be used to filter out the most infrequently occurring attribute values or replace their Values
* with given common value.
* @param df:
* DataFrame to operate on.
* @param columnName:
* Name of the column to be filtered.
* @param newColumnName:
* Name of the column that will contain the new value of the original column after filtering (if includeOthers was applied).
* @param maxNumUniqueValues:
* Maximum number of unique values to include into the comparison for each attribute column. If the amount of unique values for any attribute exceeds this value, only given number of attributes are included that have the highest usage.
* @param minValueUsage:
* Minimum total usage of a value included into the comparison. The number of cases having every returned value should be at least given percentage (a float value between 0.0 and 1.0) of all the compared cases.
* @param includeOthers:
* Should the rest of the attribute values not included due to MinValueUsage or MaxNumUniqueValues filtering be included as an aggregated "Others" value?
* If not empty/null, defines the name used for these other-values.
*/
function ColumnWithMinUsage(df, columnName, newColumnName, maxNumUniqueValues, minValueUsage, includeOthers)
{
let all = df
.GroupBy([])
.Aggregate(["NAllTotal"], ["Count"])
.WithColumn("__Join2", #sql{1});
let minValueUsageEnabled = !IsNullTop(minValueUsage);
let maxNumUniqueValuesEnabled = !IsNullTop(maxNumUniqueValues);
if (minValueUsageEnabled || maxNumUniqueValuesEnabled) {
// Perform column value-based filtering if minValueUsageEnabled or maxNumUniqueValuesEnabled is defined.
let valueColumnName = "__ValueNew";
let filteredValuesColumns = [valueColumnName: columnName];
let filteredValues = df
.GroupBy([columnName]).Aggregate(["Count"], ["Count"]);
if (minValueUsageEnabled) {
filteredValues = filteredValues
.WithColumn("__Join", #sql{1})
.Join(all, ["__Join": "__Join2"], "leftouter")
.WithColumn("Usage", #sql{Column("Count") / Column("NAllTotal")});
filteredValuesColumns = Concat(filteredValuesColumns, ["Usage"]);
}
if (maxNumUniqueValuesEnabled) {
filteredValues = filteredValues
.WithRowNumberColumn("RowNumber", ["Count"], null, [false]);
filteredValuesColumns = Concat(filteredValuesColumns, ["RowNumber"]);
}

filteredValues = filteredValues
.Select(filteredValuesColumns);

// Generate select returning all the accepted values.
let allValues = filteredValues
.(minValueUsageEnabled ? Where(#sql{Column("Usage") >= #expr{minValueUsage}}) : _)
.(maxNumUniqueValuesEnabled ? Where(#sql{Column("RowNumber") <= #expr{maxNumUniqueValues}}) : _)
.Select([valueColumnName, newColumnName: valueColumnName]);

if (!IsNullTop(includeOthers)) {
// If includeOthers is defined, replace original values with the variable defined in includeOthers.
let otherValues = filteredValues
.(minValueUsageEnabled ? Where(#sql{Column("Usage") < #expr{minValueUsage}}) : _)
.(maxNumUniqueValuesEnabled ? Where(#sql{Column("RowNumber") > #expr{maxNumUniqueValues}}) : _)
.WithColumn(newColumnName, #sql{#expr{includeOthers}})
.Select([valueColumnName, newColumnName]);
allValues = allValues.Append(otherValues)
}
df.Join(allValues, [columnName: valueColumnName], "inner")
.RemoveColumns([valueColumnName]);
}
}

// The following example will return only rows containing two of the most common values for Region-column.
//let df = DataTableById(<data table id>).SqlDataFrame;
//df = ColumnWithMinUsage(df, "Region", "_Filtered", 2, null, null);
//df.Collect().ToCsv();

// The following example will return all input rows, but will replace the values of rows whose Region-column
// has a value used by less than 15% of all the rows with a new value: "_Others".
//let df = DataTableById(<data table id>).SqlDataFrame;
//df = ColumnWithMinUsage(df, "Region", "_Filtered", null, 0.15, "_Others");
//df.Collect().ToCsv();
</syntaxhighlight>

=== Export model events and cases ===
<syntaxhighlight lang="typescript" line="1">
function ExportModelEvents(m) {
let attrs = m.EventAttributes;
ToDataFrame(
m.EventLog.Events.Concat(
[Case.Name, Type.Name, ToString(TimeStamp, "yyyy-MM-dd HH:mm:ss.fff")],
{let evt = _; attrs.{let att = _; evt.Attribute(att)}}
),
Concat(
["CaseId", "EventType", "TimeStamp"],
attrs.Name
)
).ToCsv(true);
}

function ExportModelCases(m) {
let attrs = m.CaseAttributes;
ToDataFrame(
m.EventLog.Cases.Concat(
[Name],
{let cas = _; attrs.{let att = _; cas.Attribute(att)}}
),
Concat(
["CaseId"],
attrs.Name
)
).ToCsv(true);
}
</pre>

<pre>
First(Models.Where(Name=="SAP OtC Extended")).EventsDataTable.DataFrame.ToCsv(true)
</pre>

<pre>
First(Models.Where(Name=="SAP OtC Extended")).CasesDataTable.DataFrame.ToCsv(true)
</syntaxhighlight>

=== Calculate all the value usages of a single column for each event in event data table ===
This query could be used, e.g., to find out the maximum resource usage for every resource found in the event data table.<syntaxhighlight lang="typescript" line="1">
function WithUsageColumns(resourceColumn)
{
function WithTotalUsageColumnOfSingleResource(resourceColumn, resourceValue)
{
_
.WithColumn("_Prev", #sql{Lag(Column(resourceColumn), [TimeStamp, EventType], [true, true], [CaseId], 1, null)})
.WithColumn("_UsageDiff", #sql{
CaseWhen(
Column(resourceColumn) == Column("_Prev"), 0,
Column("_Prev") == #expr{resourceValue}, -1,
Column(resourceColumn) == #expr{resourceValue}, 1,
0)
})
.WithColumn(`${resourceValue}_Usage`, #sql{Sum(Column("_UsageDiff"), [TimeStamp, EventType])})
.RemoveColumns(["_Prev", "_UsageDiff"])
}

let sdf = _;
let allValues = sdf.SelectDistinct([resourceColumn]).OrderByColumns([resourceColumn], [true]).Collect().Column(resourceColumn);
allValues.{
let v = _;
sdf = sdf.WithTotalUsageColumnOfSingleResource(resourceColumn, v)
}
sdf
}

let dt = ModelById(<model id>).EventsDataTable;
dt
.SqlDataFrame
.WithUsageColumns(<resource column name>)
.OrderByColumns([dt.ColumnMappings["TimeStamp"]], [true])
.Collect().ToCsv()
</syntaxhighlight>Where:

* <model id> is the id of the model containing event data to be examined.
* <resource column name> is the name of the column in the event data table of the specified model containing the resource being used by that event.

NOTE: This expression uses functionalities that are only supported in Snowflake-based data tables.

=== Create new Snowflake model from filter ===
This script creates a new Snowflake model (and two datatables for cases and events) containing filtered event log from given filter id. The script also works if the model doesn't have a cases datatable.

<syntaxhighlight lang="typescript" line="1">
let filter = FilterById(1); // filter id
let model = filter.model;
let project = model.project;
let nameSuffix = " - " + filter.name + " - " + ToString(Now, "dd-MM-yyyy HH:mm:ss");
let eventsDatatableName = model.EventsDataTable.Name + nameSuffix;
if (eventsDatatableName.length > 440) {
eventsDatatableName = eventsDatatableName.Substring(eventsDatatableName.length - 440);
}
let eventsData = model
.EventsDataTable
.SqlDataFrame
.ApplyFilter(
filter.rules,
model.CasesDataTable?.SqlDataFrame
);
project
.CreateDatatable(eventsDatatableName, #{"Connection": CreateSnowflakeConnection()})
.Import(eventsData);
let modelConfiguration = model.Configuration;
modelConfiguration.DataSource.Events.Set("DataTableName", eventsDatatableName);
if (model.CasesDataTable != null) {
let eventsDataCaseIdColumn = "CaseId_" + ToString(Random());
let casesDatatableName = model.CasesDataTable.Name + nameSuffix;
if (casesDatatableName.length > 440) {
casesDatatableName = casesDatatableName.Substring(casesDatatableName.length - 440);
}
let casesData = model
.CasesDataTable
.SqlDataFrame
.join(
eventsData.SelectDistinct([eventsDataCaseIdColumn: modelConfiguration.DataSource.Events.Columns.CaseId]),
[modelConfiguration.DataSource.Cases.Columns.CaseId: eventsDataCaseIdColumn]
).Select(model.CasesDataTable.ColumnNames);
project
.CreateDatatable(casesDatatableName, #{"Connection": CreateSnowflakeConnection()})
.Import(casesData);
modelConfiguration.DataSource.Cases.Set("DataTableName", casesDatatableName);
}
let modelName = model.Name + nameSuffix;
if (modelName > 440) {
modelName = modelName.Substring(modelName.length - 440);
}
project
.CreateModel(#{
"Name": modelName,
"Description": model.Description,
"Configuration": modelConfiguration
});
return modelName;
</syntaxhighlight>

=== Creating a model consisting of multiple copies of cases in an existing model ===
<syntaxhighlight lang="typescript" line="1">
/**
* @name CreateTestModel
* @description
* Creates a new model (or overwrites an existing) to given target project with given number of
* repetitions of given source model.
* Each repetition will generate "<N>-"-prefix to CaseId-columns, where N equals to the repeat index.
* @param sourceModel
* PA model used for the source data and from where the connection is copied for the target model if a
* new one has to be created.
* @param numRepeats
* Number of times the data in the source model should be repeated in the generated model.
* @param targetProject
* Project in which the target model resides.
* @param targetModelName
* Specifies the name of the test model in the given target project. If a model already exists with
* given name, event and case data in this model will be replaced with the new generated event and
* case data.
* @returns
* Model object of the test model having the newly generated data.
*/
function CreateTestModel(sourceModel, numRepeats, targetProject, targetModelName)
{
let eventsColumnMappings = sourceModel.EventsDataTable.ColumnMappings;
let casesColumnMappings = sourceModel.CasesDataTable.ColumnMappings;
let connection = sourceModel.EventsDataTable.DataSourceConnection;

function CreateResultModel()
{
function GetTable(tableName)
{
let tableConfiguration = #{
"Name": tableName,
"Connection": connection
};
let resultTable = targetProject.DataTableByName(tableName);
if (resultTable == null)
{
resultTable = targetProject.CreateDataTable(tableConfiguration)
.Modify(#{"NameInDataSource": null})
.Synchronize();
}
return resultTable;
}

let eventsTableName = `${targetModelName} - events`;
let casesTableName = `${targetModelName} - cases`;
let targetModel = targetProject.ModelByName(targetModelName);
let eventsTable, casesTable = null;

if (targetModel != null)
{
eventsTable = targetModel.EventsDataTable;
casesTable = targetModel.CasesDataTable;
}
else {
eventsTable = GetTable(eventsTableName);
if (sourceModel.CasesDataTable != null) {
casesTable = GetTable(casesTableName);
}

let timestampMapping = eventsColumnMappings["TimeStamp"];
eventsColumnMappings.Remove("TimeStamp");
eventsColumnMappings.Set("Timestamp", timestampMapping);

let modelConfiguration = #{
"DataSource": #{
"Events":#{
"DataSourceType": "datatable",
"DataTableName": eventsTableName,
"Columns": eventsColumnMappings
}
}
};

if (casesColumnMappings != null) {
modelConfiguration["DataSource"].Set("Cases", #{
"DataSourceType": "datatable",
"DataTableName": casesTableName,
"Columns": casesColumnMappings
});
}

targetModel = targetProject.CreateModel(#{"Name": targetModelName, "Configuration": modelConfiguration});
}

eventsTable.Truncate();
casesTable?.Truncate();

return #{
"TargetModel": targetModel,
"Events": eventsTable,
"Cases": casesTable
};
}

function RepeatNTimes(sourceDf, caseIdColumn, numRepeats)
{
let resultDf = null;
for (let i = 1; i <= numRepeats; ++i) {
let iterationDf = sourceDf
.WithColumn(caseIdColumn, #sql{Concat(#expr{i}, "-", Column(#expr{caseIdColumn}))});
resultDf = resultDf == null ? iterationDf : resultDf.Append(iterationDf);
}
resultDf;
}

let resultModel = CreateResultModel();
let sourceEventDataDf = sourceModel.EventsDataTable.SqlDataFrame;
let resultEventDataDf = RepeatNTimes(sourceEventDataDf, eventsColumnMappings["CaseId"], numRepeats);
resultModel["Events"].Import(resultEventDataDf);

let sourceCaseDataDf = sourceModel.CasesDataTable?.SqlDataFrame;
if (sourceCaseDataDf != null) {
let resultCaseDataDf = RepeatNTimes(sourceCaseDataDf, casesColumnMappings["CaseId"], numRepeats);
resultModel["Cases"].Import(resultCaseDataDf);
}
resultModel["TargetModel"];
}

</syntaxhighlight>Example usage:<blockquote>CreateTestModel(ProjectByName("Project").ModelByName("SAP_OrderToCash - Snowflake"), 3, ProjectByName("TestData"), "TestModel");</blockquote>Creates a new model named "TestModel" (or overwrites old one) into project named "TestData" containing the data from model "SAP_OrderToCash - Snowflake" in project "Project" repeated three times.

=== Analyzing declare patterns found in event log ===

This is an example expression that shows how POSIX-style regular expressions can be used to search for cases in an event log having certain event type patterns [https://www.researchgate.net/publication/277631859_Generating_Event_Logs_Through_the_Simulation_of_Declare_Models declare patterns].

Note: Before use, replace <model id> with a valid model identifier having event data.

<syntaxhighlight lang="typescript" line="1">
let dt = ModelById(<model id>).EventsDataTable;
let sdf = dt.SqlDataFrame.Head(1000);
let caseIdColumn = dt.ColumnMappings["CaseId"], timeStampColumn = dt.ColumnMappings["TimeStamp"], eventTypeColumn = dt.ColumnMappings["EventType"];

let eventTypesDf = sdf
.SelectDistinct([eventTypeColumn])
.OrderByColumns([eventTypeColumn], [true])
.WithRowNumberColumn("Token", [eventTypeColumn])
.WithColumn("Token", #sql{Char(Column("Token") + Unicode("a") - 1)});

sdf = sdf
.Join(eventTypesDf.Select(["_EventType2": eventTypeColumn, "Token"]), [eventTypeColumn: "_EventType2"], "leftouter");

function RespondedExistencePattern(a, b)
{
// [^a]*((a.*b.*)|(b.*a.*))*[^a]*
ToSqlExpression(`Regexp(Column("Trace"), "[^${a}]*((${a}.*${b}.*)|(${b}.*${a}.*))*[^${a}]")`)
}

function ResponsePattern(a, b)
{
// [^a]*(a.*b)*[^a]*
ToSqlExpression(`Regexp(Column("Trace"), "[^${a}]*(${a}.*${b})*[^${a}]")`)
}

function AlternateResponsePattern(a, b)
{
// [^a]*(a[^a]*b[^a]*)*[^a]*
ToSqlExpression(`Regexp(Column("Trace"), "[^${a}]*(${a}[^${a}]*${b}[^${a}]*)*[^${a}]")`)
}

function ChainResponsePattern(a, b)
{
// [^a]*(ab[^a]*)*[^a]*
ToSqlExpression(`Regexp(Column("Trace"), "[^${a}]*(${a}${b}[^${a}]*)*[^${a}]")`)
}

function PrecedencePattern(a, b)
{
// [^b]*(a.*b)*[^b]*
ToSqlExpression(`Regexp(Column("Trace"), "[^${b}]*(${a}.*${b})*[^${b}]")`)
}

let tracesDf = sdf
.GroupBy([caseIdColumn])
.Aggregate(["Trace": "Token"], [#{"Function": "list", "Ordering": [timeStampColumn], "Separator": ""}])
.WithColumn("RespondedExistenceNL", #sql{#expr{RespondedExistencePattern("n", "l")}})
.WithColumn("RespondedExistenceCA", #sql{#expr{RespondedExistencePattern("c", "a")}})
.WithColumn("ResponsePatternNL", #sql{#expr{ResponsePattern("n", "l")}})
.WithColumn("ResponsePatternCA", #sql{#expr{ResponsePattern("c", "a")}})
.WithColumn("AlternateResponsePatternNL", #sql{#expr{AlternateResponsePattern("n", "l")}})
.WithColumn("AlternateResponsePatternCA", #sql{#expr{AlternateResponsePattern("c", "a")}})
.WithColumn("ChainResponsePatternNL", #sql{#expr{ChainResponsePattern("n", "l")}})
.WithColumn("ChainResponsePatternCA", #sql{#expr{ChainResponsePattern("c", "a")}})
.WithColumn("PrecedencePatternNL", #sql{#expr{PrecedencePattern("n", "l")}})
.WithColumn("PrecedencePatternCA", #sql{#expr{PrecedencePattern("c", "a")}})
;

[tracesDf.Collect().ToCsv(), eventTypesDf.Collect().ToCsv()]
</syntaxhighlight>

=== Perform a query and send results as E-mail in a HTML table ===
This example requires two scripts:

# Expression-type script to execute (can be, e.g., scheduled to run daily).
# Expression-type script containing the query JSON to use as basis for the e-mail. In this example, this script is named as "Send query as E-mail - query JSON". The contents of this script is just the JSON representation of a query that can be extracted, e.g., from any PA chart view.

Script #1 should contain the following code:<syntaxhighlight lang="typescript">
let replyToAddress = "noreply@test.com";
let recipientsArray = ["test@test.com"];
let queryConfiguration = Project.ScriptByName("Send query as E-mail - query JSON").Code;
let resultDf = Query(ParseJson(queryConfiguration));
let mailBodyHtml = resultDf.Collect().`
<table>
<caption>Example query</caption>
<thead>
<tr>
${StringJoin("", _.Columns.`
<th>${_}</th>
`)}
</tr>
</thead>
<tbody>
${StringJoin("", _.Rows.`
<tr>
${StringJoin("", _.`
<td>${_}</td>
`)}
</tr>
`)}
</tbody>
</table>
`;

SendEmail(#{
"ReplyTo": [replyToAddress],
"To": recipientsArray,
"Subject": "Example query E-mail",
"IsBodyHtml": true,
"Body": mailBodyHtml
});

</syntaxhighlight>

Web API: saml2

2025-03-17T12:30:36Z

MarHink:

'''Saml2''' method returns the SAML 2.0 service provider (SP) metadata. No authentication is required to fetch the metadata. Usually the service provider metadata url is configured to the identity provider (IdP), which can then read, e.g., the needed public encryption keys.

<pre>
Url: GET qprpa/Saml2
attachment; filename="customer.onqpr.com_qprpa_Saml2.xml"
Content-Type: application/samlmetadata+xml
</pre>

Example:
<pre>
<EntityDescriptor xmlns="urn:oasis:names:tc:SAML:2.0:metadata" xmlns:saml2="urn:oasis:names:tc:SAML:2.0:assertion" cacheDuration="PT1H" entityID="https://customer.onqpr.com/qprpa/Saml2" ID="_76ac281969e84420924d4e25d22b7c4e">
<Signature xmlns="http://www.w3.org/2000/09/xmldsig#">
<SignedInfo>
<CanonicalizationMethod Algorithm="http://www.w3.org/2001/10/xml-exc-c14n#" />
<SignatureMethod Algorithm="http://www.w3.org/2001/04/xmldsig-more#rsa-sha256" />
<Reference URI="...">
<Transforms>
<Transform Algorithm="http://www.w3.org/2000/09/xmldsig#enveloped-signature" />
<Transform Algorithm="http://www.w3.org/2001/10/xml-exc-c14n#" />
</Transforms>
<DigestMethod Algorithm="http://www.w3.org/2001/04/xmlenc#sha256" />
<DigestValue>...</DigestValue>
</Reference>
</SignedInfo>
<SignatureValue>...</SignatureValue>
<KeyInfo>
<X509Data>
<X509Certificate>...</X509Certificate>
</X509Data>
</KeyInfo>
</Signature>
<SPSSODescriptor AuthnRequestsSigned="false" WantAssertionsSigned="false" protocolSupportEnumeration="urn:oasis:names:tc:SAML:2.0:protocol">
<KeyDescriptor use="signing">
<KeyInfo xmlns="http://www.w3.org/2000/09/xmldsig#">
<X509Data>
<X509Certificate>...</X509Certificate>
</X509Data>
</KeyInfo>
</KeyDescriptor>
<SingleLogoutService Binding="urn:oasis:names:tc:SAML:2.0:bindings:HTTP-Redirect" Location="https://customer.onqpr.com/QPRPA/Saml2/Logout" />
<AssertionConsumerService Binding="urn:oasis:names:tc:SAML:2.0:bindings:HTTP-POST" Location="https://customer.onqpr.com/QPRPA/Saml2/Acs" isDefault="true" index="0" />
<AssertionConsumerService Binding="urn:oasis:names:tc:SAML:2.0:bindings:HTTP-Artifact" Location="https://customer.onqpr.com/QPRPA/Saml2/Acs" isDefault="false" index="1" />
</SPSSODescriptor>
</EntityDescriptor>
</pre>
[[Category: QPR ProcessAnalyzer]]

Expression Script Examples

2025-03-06T09:12:53Z

MarHink:

This page contains script examples written in the QPR ProcessAnalyzer expression language. See how expression scripts can be created in the [[Managing_Scripts#Creating_Script|Workspace]]. For documentation for the syntax, functions and entities can be found from the main page in the [[QPR_ProcessAnalyzer_Wiki#For_Developers|KPI Expression Language]] section.

== Calling Expression Script from Expression ==
Expression scripts can be called from an expression using the [[QPR_ProcessAnalyzer_Objects_in_Expression_Language#Script|Run]] function with the following syntax:
<pre>
let result = ScriptById(123).Run(#{
"parameter1": "value1",
"parameter2": false,
"parameter2": 123.45
})
</pre>

The command waits until the run is completed, and the return value of the called script is returned by the Run function call.

Parameters can be passed to the called script, and the parameters are available as variables in the script. The parameters can contain any type of data.

Expression scripts can also be called from a dashboard. Expressions can be stored to scripts instead of dashboards, which is a way to separate complex expressions from dashboards and allow to reuse expressions across several dashboards.

== Calling SQL Script from Expression ==
SQL script can be called from an expression using the Run function as follows (similar to calling [[#Calling Expression Script from Expression|expression scripts]]):
<pre>
let result = ScriptById(123).Run(#{
"parameter1": "value1",
"parameter2": 321
});
let arrayOfAllReports = result.Keys;
let report1 = result.Report1;
let report2 = result.Report2;
</pre>

SQL scripts can return multiple ''reports'', which are combined to a dictionary, where the key is the name of the report ("sheet name") and value is the report data as a DataFrame. See in the above example, how the reports can be accessed by their name.

== Examples ==
=== Call web service===
Contact to a web service, fetch some data, and store it to a datatable.

<syntaxhighlight lang="typescript" line>
let datatableName = "Web Service Data";
let webServiceData = CallWebService(
#{"Address": "https://processanalyzer.onqpr.com/qprpa/api/serverinfo"}
);

let targetDatatable = Project.Datatables.Where(name==datatableName);
if (Count(targetDatatable) == 0) {
targetDatatable = Project.CreateDatatable(datatableName)
.AddColumn("Setting name", "String")
.AddColumn("Setting value", "String")
.AddColumn("Data read", "DateTime");
} else {
targetDatatable = targetDatatable[0];
}

let currentTime = Now;
let dataAsDf = ToDataFrame(
webServiceData.keys.{
let key = _;
[key, webServiceData[key], currentTime];
},
["Setting name", "Setting value", "Data read"]
);
targetDatatable.Import(dataAsDf, #{"Append":true});
WriteLog(`${CountTop(dataAsDf.Rows)} rows written to datatable`);
</syntaxhighlight>

=== Store data to datatable ===
Get all models in the system and store them to a datatable.

<syntaxhighlight lang="typescript" line>
let newDatatable = Project
.CreateDatatable("Models list " + ToString(Now, "dd.MM.yyyy HH:mm:ss"))
.AddColumn("Model name", "String")
.AddColumn("Project name", "String")
.AddColumn("Created time", "DateTime")
.AddColumn("Cases", "Integer");
let startTime = Now;
let modelsData = ToDataFrame(
Models.([Name, Project.Name, CreatedDate, NCases]),
["Model name", "Project name", "Created time", "Cases"]
);
WriteLog(`Listing models took ${(Now - startTime).TotalSeconds.Round(2)} seconds.`);
newDatatable.Import(modelsData);
WriteLog(`Datatable ${newDatatable.Id} created.`);
</syntaxhighlight>

=== Convert datatable column data ===
This script can be used to convert a single column into numerical data type. To use the script, you need to setup the following in the beginning of the script:
* Project name where the datatable is located.
* Datatable name
* Name of the column to be converted

Note that the conversion fails, if there is data that cannot be converted into numerical format. The conversion assumes that period (.) is used as the decimal point.

<syntaxhighlight lang="typescript" line>
let projectName = "New Project";
let datatableName = "qpr processanalyzer events";
let columnName = "Event order in case";

let project = (Projects.Where(Name==projectName))[0];
let datatable = (project.Datatables.Where(Name==datatableName))[0];
DatatableById(datatable.Id).DataFrame
.SetColumns([
columnName: () => {
let data = Column(columnName);
if (data == null) {
null;
} else {
ToFloat(data);
}
}
])
.Persist(datatable.Name, ["ProjectId": project.Id, "Append": false]);
</syntaxhighlight>

Instead of converting to numeric (with the ''ToFloat'' function), data can be converted into string using the ''ToString'' function.

=== Show DataFrame as HTML table ===

This script defines a function to show dataframe as a HTML table, and uses the function for a literal dataframe.

<syntaxhighlight lang="typescript" line>
function dataframeToHtmlTable(df) {
return
`<table>
<tr>
${StringJoin("\r\n\t\t", + df.columns.`<th>${_}</th>`)}
</tr>
${StringJoin("", df.Rows.(
"\r\n\t<tr>" + StringJoin("", _.`\r\n\t\t<td>${ToString(_)}</td>`) + "\r\n\t</tr>"
))}
</table>`
}

let data = ToDataFrame(
[
["one", "two", "three"],
["four", "five", "six"],
["seven", "eight", "nine"]
],
["Column 1", "Column 2", "Column 3"]
);

return dataframeToHtmlTable(data);
</syntaxhighlight>

=== Copy local datatables to Snowflake ===

<syntaxhighlight lang="typescript" line>
// Copies all datatables in a project to another project including datatable contents.
// Usage instructions:
// 1. Create expression script in the project from where you want to copy the datatables.
// 2. Create a new project named as "<name of the project to be moved> - Snowflake". New datatables will be created here. E.g., when moving project named "SAP_OrderToCash", the target project should be named as "SAP_OrderToCash - Snowflake".
// 3. Run the script.
// NOTE: Columns of type "Any" will be created as "String"-columns in Snowflake, thus it is recommended that actual data types are set for the tables prior to the move.

let sourceProject = Project;
let sourceProjectName = Project.Name;
let targetProjectName = `${sourceProjectName} - Snowflake`;
let targetProject = First(Projects.Where(Name == targetProjectName));
if (IsNull(targetProject)) {
WriteLog(`Unable to find target project named "${targetProjectName}". Aborting operation.`);
return;
}
let dts = sourceProject.DataTables;
WriteLog(`Copying all ${CountTop(dts)} data tables found in project "${sourceProject.Name}" (id: ${sourceProject.Id}) to Snowflake in project "${targetProject.Name}" (id: ${targetProject.Id})`);
dts.{
let sourceDt = _;
WriteLog(`Starting to copy data table "${Name}" (id: ${Id}) having ${NRows} rows and ${NColumns} columns.`);
let targetDt;
targetDt = targetProject.DatatableByName(sourceDt.Name);
if (targetDt == null) {
targetDt = targetProject.CreateDataTable(sourceDt.Name, #{"Connection": CreateSnowflakeConnection(#{"ProjectId": targetProject.Id})});
targetDt.Import(sourceDt.SqlDataFrame);
WriteLog(`Finished copying data table "${Name}" (id: ${Id}) to table "${targetDt.Name}" (id: ${targetDt.Id})`);
} else {
WriteLog(`Datatable already exist "${Name}" (id: ${Id}) to table "${targetDt.Name}" (id: ${targetDt.Id})`);
}
}
WriteLog(`Finished copying all the data tables found in project "${sourceProject.Name}" (id: ${sourceProject.Id}) to Snowflake in project "${targetProject.Name}" (id: ${targetProject.Id})`);
</syntaxhighlight>

If you don't need to copy the data but only create the Snowflake datatables with columns, you can change the line 22 to
<syntaxhighlight>
targetDt.Import(sourceDt.SqlDataFrame.head(0));
</syntaxhighlight>

=== Copy single datatable to Snowflake ===
This script creates a copy of a single datatable to Snowflake. Replace the ''<tableId1>'' with the id of the source datatable.
<syntaxhighlight lang="typescript" line>
function CopyDataTableToSnowflake(dataTableId)
{
let sourceDt = DataTableById(dataTableId);
sourceDt.SqlDataFrame.Persist(`${sourceDt.Name} - Snowflake`, #{"Append": false, "Connection": CreateSnowflakeConnection(#{"ProjectId": sourceDt.Project.Id})});
}
CopyDataTableToSnowflake(<tableId1>);
</syntaxhighlight>

=== Create a copy of a data table that has all Any-type columns changed to String-type columns ===
<syntaxhighlight lang="typescript" line>
function ConvertAnyDataTypesToStringsToNewTable(dataTableId)
{
let dt = DataTableById(dataTableId);
let sdf = dt.SqlDataFrame;
let cts = dt.ColumnTypes;
cts.{
let ct = _;
if (ct.DataType == "Any") {
let n = ct.Name;
sdf = sdf.WithColumn(ct.Name, #sql{Cast(Column(Variable("n")), "ShortString")});
}
};
sdf.Persist(`${dt.Name} - Converted`, #{"Append": false, "ProjectId": dt.Project.Id});
}
ConvertAnyDataTypesToStringsToNewTable(<dataTableId>);
</syntaxhighlight>

=== Query number of rows in given data table having a datetime value in given year grouped by month and return resulting table as CSV ===
SqlDataFrame is used in order to prevent loading the whole datatable into memory first. Filtering is performed as first operation in order to minimize the amount of required work for the data source of the data table.<syntaxhighlight lang="typescript" line="1">
DataTableById(<data table id>)
.SqlDataFrame
.Where(#sql{2014 == Year(Column("Start Time"))})
.WithColumn("Month", #sql{Month(Column("Start Time"))})
.GroupBy(["Month"]).Aggregate(["Count"], ["Count"])
.OrderByColumns(["Month"], [true])
.Collect().ToCsv();
</syntaxhighlight>

=== Function for filtering SqlDataFrame by removing rows having, or replacing, the most infrequently occurring column values ===
<syntaxhighlight lang="typescript" line="1">
/***
* @name ColumnWithMinUsage
* @descripion
* Generic function that can be used to filter out the most infrequently occurring attribute values or replace their Values
* with given common value.
* @param df:
* DataFrame to operate on.
* @param columnName:
* Name of the column to be filtered.
* @param newColumnName:
* Name of the column that will contain the new value of the original column after filtering (if includeOthers was applied).
* @param maxNumUniqueValues:
* Maximum number of unique values to include into the comparison for each attribute column. If the amount of unique values for any attribute exceeds this value, only given number of attributes are included that have the highest usage.
* @param minValueUsage:
* Minimum total usage of a value included into the comparison. The number of cases having every returned value should be at least given percentage (a float value between 0.0 and 1.0) of all the compared cases.
* @param includeOthers:
* Should the rest of the attribute values not included due to MinValueUsage or MaxNumUniqueValues filtering be included as an aggregated "Others" value?
* If not empty/null, defines the name used for these other-values.
*/
function ColumnWithMinUsage(df, columnName, newColumnName, maxNumUniqueValues, minValueUsage, includeOthers)
{
let all = df
.GroupBy([])
.Aggregate(["NAllTotal"], ["Count"])
.WithColumn("__Join2", #sql{1});
let minValueUsageEnabled = !IsNullTop(minValueUsage);
let maxNumUniqueValuesEnabled = !IsNullTop(maxNumUniqueValues);
if (minValueUsageEnabled || maxNumUniqueValuesEnabled) {
// Perform column value-based filtering if minValueUsageEnabled or maxNumUniqueValuesEnabled is defined.
let valueColumnName = "__ValueNew";
let filteredValuesColumns = [valueColumnName: columnName];
let filteredValues = df
.GroupBy([columnName]).Aggregate(["Count"], ["Count"]);
if (minValueUsageEnabled) {
filteredValues = filteredValues
.WithColumn("__Join", #sql{1})
.Join(all, ["__Join": "__Join2"], "leftouter")
.WithColumn("Usage", #sql{Column("Count") / Column("NAllTotal")});
filteredValuesColumns = Concat(filteredValuesColumns, ["Usage"]);
}
if (maxNumUniqueValuesEnabled) {
filteredValues = filteredValues
.WithRowNumberColumn("RowNumber", ["Count"], null, [false]);
filteredValuesColumns = Concat(filteredValuesColumns, ["RowNumber"]);
}

filteredValues = filteredValues
.Select(filteredValuesColumns);

// Generate select returning all the accepted values.
let allValues = filteredValues
.(minValueUsageEnabled ? Where(#sql{Column("Usage") >= #expr{minValueUsage}}) : _)
.(maxNumUniqueValuesEnabled ? Where(#sql{Column("RowNumber") <= #expr{maxNumUniqueValues}}) : _)
.Select([valueColumnName, newColumnName: valueColumnName]);

if (!IsNullTop(includeOthers)) {
// If includeOthers is defined, replace original values with the variable defined in includeOthers.
let otherValues = filteredValues
.(minValueUsageEnabled ? Where(#sql{Column("Usage") < #expr{minValueUsage}}) : _)
.(maxNumUniqueValuesEnabled ? Where(#sql{Column("RowNumber") > #expr{maxNumUniqueValues}}) : _)
.WithColumn(newColumnName, #sql{#expr{includeOthers}})
.Select([valueColumnName, newColumnName]);
allValues = allValues.Append(otherValues)
}
df.Join(allValues, [columnName: valueColumnName], "inner")
.RemoveColumns([valueColumnName]);
}
}

// The following example will return only rows containing two of the most common values for Region-column.
//let df = DataTableById(<data table id>).SqlDataFrame;
//df = ColumnWithMinUsage(df, "Region", "_Filtered", 2, null, null);
//df.Collect().ToCsv();

// The following example will return all input rows, but will replace the values of rows whose Region-column
// has a value used by less than 15% of all the rows with a new value: "_Others".
//let df = DataTableById(<data table id>).SqlDataFrame;
//df = ColumnWithMinUsage(df, "Region", "_Filtered", null, 0.15, "_Others");
//df.Collect().ToCsv();
</syntaxhighlight>

=== Export model events and cases ===
<syntaxhighlight lang="typescript" line="1">
function ExportModelEvents(m) {
let attrs = m.EventAttributes;
ToDataFrame(
m.EventLog.Events.Concat(
[Case.Name, Type.Name, ToString(TimeStamp, "yyyy-MM-dd HH:mm:ss.fff")],
{let evt = _; attrs.{let att = _; evt.Attribute(att)}}
),
Concat(
["CaseId", "EventType", "TimeStamp"],
attrs.Name
)
).ToCsv(true);
}

function ExportModelCases(m) {
let attrs = m.CaseAttributes;
ToDataFrame(
m.EventLog.Cases.Concat(
[Name],
{let cas = _; attrs.{let att = _; cas.Attribute(att)}}
),
Concat(
["CaseId"],
attrs.Name
)
).ToCsv(true);
}
</pre>

<pre>
First(Models.Where(Name=="SAP OtC Extended")).EventsDataTable.DataFrame.ToCsv(true)
</pre>

<pre>
First(Models.Where(Name=="SAP OtC Extended")).CasesDataTable.DataFrame.ToCsv(true)
</syntaxhighlight>

=== Calculate all the value usages of a single column for each event in event data table ===
This query could be used, e.g., to find out the maximum resource usage for every resource found in the event data table.<syntaxhighlight lang="typescript" line="1">
function WithUsageColumns(resourceColumn)
{
function WithTotalUsageColumnOfSingleResource(resourceColumn, resourceValue)
{
_
.WithColumn("_Prev", #sql{Lag(Column(resourceColumn), [TimeStamp, EventType], [true, true], [CaseId], 1, null)})
.WithColumn("_UsageDiff", #sql{
CaseWhen(
Column(resourceColumn) == Column("_Prev"), 0,
Column("_Prev") == #expr{resourceValue}, -1,
Column(resourceColumn) == #expr{resourceValue}, 1,
0)
})
.WithColumn(`${resourceValue}_Usage`, #sql{Sum(Column("_UsageDiff"), [TimeStamp, EventType])})
.RemoveColumns(["_Prev", "_UsageDiff"])
}

let sdf = _;
let allValues = sdf.SelectDistinct([resourceColumn]).OrderByColumns([resourceColumn], [true]).Collect().Column(resourceColumn);
allValues.{
let v = _;
sdf = sdf.WithTotalUsageColumnOfSingleResource(resourceColumn, v)
}
sdf
}

let dt = ModelById(<model id>).EventsDataTable;
dt
.SqlDataFrame
.WithUsageColumns(<resource column name>)
.OrderByColumns([dt.ColumnMappings["TimeStamp"]], [true])
.Collect().ToCsv()
</syntaxhighlight>Where:

* <model id> is the id of the model containing event data to be examined.
* <resource column name> is the name of the column in the event data table of the specified model containing the resource being used by that event.

NOTE: This expression uses functionalities that are only supported in Snowflake-based data tables.

=== Create new Snowflake model from filter ===
This script creates a new Snowflake model (and two datatables for cases and events) containing filtered event log from given filter id. The script also works if the model doesn't have a cases datatable.

<syntaxhighlight lang="typescript" line="1">
let filter = FilterById(1); // filter id
let model = filter.model;
let project = model.project;
let nameSuffix = " - " + filter.name + " - " + ToString(Now, "dd-MM-yyyy HH:mm:ss");
let eventsDatatableName = model.EventsDataTable.Name + nameSuffix;
if (eventsDatatableName.length > 440) {
eventsDatatableName = eventsDatatableName.Substring(eventsDatatableName.length - 440);
}
let eventsData = model
.EventsDataTable
.SqlDataFrame
.ApplyFilter(
filter.rules,
model.CasesDataTable?.SqlDataFrame
);
project
.CreateDatatable(eventsDatatableName, #{"Connection": CreateSnowflakeConnection()})
.Import(eventsData);
let modelConfiguration = model.Configuration;
modelConfiguration.DataSource.Events.Set("DataTableName", eventsDatatableName);
if (model.CasesDataTable != null) {
let eventsDataCaseIdColumn = "CaseId_" + ToString(Random());
let casesDatatableName = model.CasesDataTable.Name + nameSuffix;
if (casesDatatableName.length > 440) {
casesDatatableName = casesDatatableName.Substring(casesDatatableName.length - 440);
}
let casesData = model
.CasesDataTable
.SqlDataFrame
.join(
eventsData.SelectDistinct([eventsDataCaseIdColumn: modelConfiguration.DataSource.Events.Columns.CaseId]),
[modelConfiguration.DataSource.Cases.Columns.CaseId: eventsDataCaseIdColumn]
).Select(model.CasesDataTable.ColumnNames);
project
.CreateDatatable(casesDatatableName, #{"Connection": CreateSnowflakeConnection()})
.Import(casesData);
modelConfiguration.DataSource.Cases.Set("DataTableName", casesDatatableName);
}
let modelName = model.Name + nameSuffix;
if (modelName > 440) {
modelName = modelName.Substring(modelName.length - 440);
}
project
.CreateModel(#{
"Name": modelName,
"Description": model.Description,
"Configuration": modelConfiguration
});
return modelName;
</syntaxhighlight>

=== Creating a model consisting of multiple copies of cases in an existing model ===
<syntaxhighlight lang="typescript" line="1">
/**
* @name CreateTestModel
* @description
* Creates a new model (or overwrites an existing) to given target project with given number of
* repetitions of given source model.
* Each repetition will generate "<N>-"-prefix to CaseId-columns, where N equals to the repeat index.
* @param sourceModel
* PA model used for the source data and from where the connection is copied for the target model if a
* new one has to be created.
* @param numRepeats
* Number of times the data in the source model should be repeated in the generated model.
* @param targetProject
* Project in which the target model resides.
* @param targetModelName
* Specifies the name of the test model in the given target project. If a model already exists with
* given name, event and case data in this model will be replaced with the new generated event and
* case data.
* @returns
* Model object of the test model having the newly generated data.
*/
function CreateTestModel(sourceModel, numRepeats, targetProject, targetModelName)
{
let eventsColumnMappings = sourceModel.EventsDataTable.ColumnMappings;
let casesColumnMappings = sourceModel.CasesDataTable.ColumnMappings;
let connection = sourceModel.EventsDataTable.DataSourceConnection;

function CreateResultModel()
{
function GetTable(tableName)
{
let tableConfiguration = #{
"Name": tableName,
"Connection": connection
};
let resultTable = targetProject.DataTableByName(tableName);
if (resultTable == null)
{
resultTable = targetProject.CreateDataTable(tableConfiguration)
.Modify(#{"NameInDataSource": null})
.Synchronize();
}
return resultTable;
}

let eventsTableName = `${targetModelName} - events`;
let casesTableName = `${targetModelName} - cases`;
let targetModel = targetProject.ModelByName(targetModelName);
let eventsTable, casesTable = null;

if (targetModel != null)
{
eventsTable = targetModel.EventsDataTable;
casesTable = targetModel.CasesDataTable;
}
else {
eventsTable = GetTable(eventsTableName);
if (sourceModel.CasesDataTable != null) {
casesTable = GetTable(casesTableName);
}

let timestampMapping = eventsColumnMappings["TimeStamp"];
eventsColumnMappings.Remove("TimeStamp");
eventsColumnMappings.Set("Timestamp", timestampMapping);

let modelConfiguration = #{
"DataSource": #{
"Events":#{
"DataSourceType": "datatable",
"DataTableName": eventsTableName,
"Columns": eventsColumnMappings
}
}
};

if (casesColumnMappings != null) {
modelConfiguration["DataSource"].Set("Cases", #{
"DataSourceType": "datatable",
"DataTableName": casesTableName,
"Columns": casesColumnMappings
});
}

targetModel = targetProject.CreateModel(#{"Name": targetModelName, "Configuration": modelConfiguration});
}

eventsTable.Truncate();
casesTable?.Truncate();

return #{
"TargetModel": targetModel,
"Events": eventsTable,
"Cases": casesTable
};
}

function RepeatNTimes(sourceDf, caseIdColumn, numRepeats)
{
let resultDf = null;
for (let i = 1; i <= numRepeats; ++i) {
let iterationDf = sourceDf
.WithColumn(caseIdColumn, #sql{Concat(#expr{i}, "-", Column(#expr{caseIdColumn}))});
resultDf = resultDf == null ? iterationDf : resultDf.Append(iterationDf);
}
resultDf;
}

let resultModel = CreateResultModel();
let sourceEventDataDf = sourceModel.EventsDataTable.SqlDataFrame;
let resultEventDataDf = RepeatNTimes(sourceEventDataDf, eventsColumnMappings["CaseId"], numRepeats);
resultModel["Events"].Import(resultEventDataDf);

let sourceCaseDataDf = sourceModel.CasesDataTable?.SqlDataFrame;
if (sourceCaseDataDf != null) {
let resultCaseDataDf = RepeatNTimes(sourceCaseDataDf, casesColumnMappings["CaseId"], numRepeats);
resultModel["Cases"].Import(resultCaseDataDf);
}
resultModel["TargetModel"];
}

</syntaxhighlight>Example usage:<blockquote>CreateTestModel(ProjectByName("Project").ModelByName("SAP_OrderToCash - Snowflake"), 3, ProjectByName("TestData"), "TestModel");</blockquote>Creates a new model named "TestModel" (or overwrites old one) into project named "TestData" containing the data from model "SAP_OrderToCash - Snowflake" in project "Project" repeated three times.

=== Analyzing declare patterns found in event log ===

This is an example expression that shows how POSIX-style regular expressions can be used to search for cases in an event log having certain event type patterns [https://www.researchgate.net/publication/277631859_Generating_Event_Logs_Through_the_Simulation_of_Declare_Models declare patterns].

Note: Before use, replace <model id> with a valid model identifier having event data.

<syntaxhighlight lang="typescript" line="1">
let dt = ModelById(<model id>).EventsDataTable;
let sdf = dt.SqlDataFrame.Head(1000);
let caseIdColumn = dt.ColumnMappings["CaseId"], timeStampColumn = dt.ColumnMappings["TimeStamp"], eventTypeColumn = dt.ColumnMappings["EventType"];

let eventTypesDf = sdf
.SelectDistinct([eventTypeColumn])
.OrderByColumns([eventTypeColumn], [true])
.WithRowNumberColumn("Token", [eventTypeColumn])
.WithColumn("Token", #sql{Char(Column("Token") + Unicode("a") - 1)});

sdf = sdf
.Join(eventTypesDf.Select(["_EventType2": eventTypeColumn, "Token"]), [eventTypeColumn: "_EventType2"], "leftouter");

function RespondedExistencePattern(a, b)
{
// [^a]*((a.*b.*)|(b.*a.*))*[^a]*
ToSqlExpression(`Regexp(Column("Trace"), "[^${a}]*((${a}.*${b}.*)|(${b}.*${a}.*))*[^${a}]")`)
}

function ResponsePattern(a, b)
{
// [^a]*(a.*b)*[^a]*
ToSqlExpression(`Regexp(Column("Trace"), "[^${a}]*(${a}.*${b})*[^${a}]")`)
}

function AlternateResponsePattern(a, b)
{
// [^a]*(a[^a]*b[^a]*)*[^a]*
ToSqlExpression(`Regexp(Column("Trace"), "[^${a}]*(${a}[^${a}]*${b}[^${a}]*)*[^${a}]")`)
}

function ChainResponsePattern(a, b)
{
// [^a]*(ab[^a]*)*[^a]*
ToSqlExpression(`Regexp(Column("Trace"), "[^${a}]*(${a}${b}[^${a}]*)*[^${a}]")`)
}

function PrecedencePattern(a, b)
{
// [^b]*(a.*b)*[^b]*
ToSqlExpression(`Regexp(Column("Trace"), "[^${b}]*(${a}.*${b})*[^${b}]")`)
}

let tracesDf = sdf
.GroupBy([caseIdColumn])
.Aggregate(["Trace": "Token"], [#{"Function": "list", "Ordering": [timeStampColumn], "Separator": ""}])
.WithColumn("RespondedExistenceNL", #sql{#expr{RespondedExistencePattern("n", "l")}})
.WithColumn("RespondedExistenceCA", #sql{#expr{RespondedExistencePattern("c", "a")}})
.WithColumn("ResponsePatternNL", #sql{#expr{ResponsePattern("n", "l")}})
.WithColumn("ResponsePatternCA", #sql{#expr{ResponsePattern("c", "a")}})
.WithColumn("AlternateResponsePatternNL", #sql{#expr{AlternateResponsePattern("n", "l")}})
.WithColumn("AlternateResponsePatternCA", #sql{#expr{AlternateResponsePattern("c", "a")}})
.WithColumn("ChainResponsePatternNL", #sql{#expr{ChainResponsePattern("n", "l")}})
.WithColumn("ChainResponsePatternCA", #sql{#expr{ChainResponsePattern("c", "a")}})
.WithColumn("PrecedencePatternNL", #sql{#expr{PrecedencePattern("n", "l")}})
.WithColumn("PrecedencePatternCA", #sql{#expr{PrecedencePattern("c", "a")}})
;

[tracesDf.Collect().ToCsv(), eventTypesDf.Collect().ToCsv()]
</syntaxhighlight>

SQL Scripting Commands

2025-01-14T15:49:52Z

MarHink: /* --#ImportSapQuery */

This page lists QPR ProcessAnalyzer commands that can be used in the SQL scripts. Each command precedes one or two SQL queries, which sets parameters for the command or defines the data used by the command.

<div style="display: flex;flex-wrap: wrap;">
<div style="flex: 1 0 230px;border:1px solid #dfdfdf;padding:0 1em 1em 1.5em;background-color:#F7FAFC;margin:10px 0px 0px 10px;">
=== Data Extraction ===
* [[#--.23CallWebService|CallWebService]]
* [[#--.23ImportOdbcQuery|ImportOdbcQuery]]
* [[#--.23ImportOleDbQuery|ImportOleDbQuery]]
* [[#--.23ImportSalesforceQuery|ImportSalesforceQuery]]
* [[#--.23ImportSapQuery|ImportSapQuery]]
* [[#--.23ImportSqlQuery|ImportSqlQuery]] (ADO.Net)
</div>

<div style="flex: 1 0 230px;border:1px solid #dfdfdf;padding:0 1em 1em 1.5em;background-color:#F7FAFC;margin:10px 0px 0px 10px;">

=== Data Output ===
* [[#--.23ImportDataTable|ImportDataTable]]
* [[#--.23SendEmail|SendEmail]]
* [[#--.23ShowReport|ShowReport]]
* [[#--.23WriteLog|WriteLog]]
</div>

<div style="flex: 1 0 230px;border:1px solid #dfdfdf;padding:0 1em 1em 1.5em;background-color:#F7FAFC;margin:10px 0px 0px 10px;">

=== Script Flow ===
* [[#--.23RunQuery|RunQuery]] ([[RunQuery Script Examples|examples]])
* [[#--.23Commit|Commit]]
* [[#--.23Exit|Exit]]
* [[#--.23GetAnalysis|GetAnalysis]]
* [[#--.23Run|Run]]
* [[#--.23StartBackground|StartBackground]]
</div>

</div>

= --#CallWebService =
Extracts data via Web Service. This command takes one SELECT query as parameter.

== Query ==
Configures the command using a SELECT statement returning two columns: the first column is for a key and the second one is for a value of that key. The values in both the key column and in the value column are of type NVARCHAR. The supported keys for this command are:
; Address
: Defines the URI of the service to call. Mandatory.
; Method
: Defines the HTTP method to use for the call. Must be any of the following: GET (default), POST, PUT, DELETE. Optional.
; Body
: Defines the message body text to send to the service. Default value is empty. Optional.
; Encoding
: Defines the encoding method to use. The supported options are listed in [https://msdn.microsoft.com/en-us/library/system.text.encoding%28v=vs.110%29.aspx https://msdn.microsoft.com/en-us/library/system.text.encoding%28v=vs.110%29.aspx]. Default value is UTF8. Optional.
; Timeout
: Number of milliseconds to wait before the request times out. Default value is 60000. Optional.
; ExecuteInClientSide
: Defines whether the web service call is made from the QPR ScriptLauncher or from the server. TRUE or 1, the call is executed in the ScriptLauncher. FALSE or 0, the call is executed in the server. Default value is FALSE. Optional.
; DefaultNetworkCredentials
: Optional. Defines the possibility to use default network credentials in web service calls:
: 1 = use the default network credentials.
: 0 = don't use the default network credentials.
: If CallWebService command is run in the server side (ExecuteInClientSide=False), the default network credentials can be used only if in the server configuration AllowForwardingNetworkCredentials is true (it is false by default). Otherwise, if the CallWebService command is run in the client side (ExecuteInClientSide=True), the default network credentials can always be used.
; CatchOperationExceptions
: Optional. Defines whether to stop the script execution or to continue to run the script from the next statement if an exception occurs when running the script:
: 1 = don't stop execution of the script, continue running the script from the next statement.
: 0 = stop execution of the current script and show the exception.
: The following script variables will be set and are shown in the script log:
: <code>@_ExceptionOccurred</code> If there was an exception, then this value is 1, otherwise 0. INT.
: <code>@_ExceptionType</code> If there was an exception, shows the C# class name for the exception, NVARCHAR(MAX), otherwise NULL.
: <code>@_ExceptionMessage</code> If there was an exception, contains a message that would have been displayed, NVARCHAR(MAX), otherwise NULL.
: <code>@_ExceptionDetails</code> If there was an exception, contains the details that would have been displayed, including the system stack trace, NVARCHAR(MAX), otherwise NULL.
;<nowiki><other parameters></nowiki>
: All the rest of the passed parameters not listed above are added as extra headers to the request. For example, ''Content-Type'' and ''Accept'' HTTP headers can be added. Optional.

== Result ==
The result of the request is passed to the script following the CallWebService operation in the following variables:
: <code>@_ResponseText</code> The response text received from the remote server. If there was an error in processing the request, this will contain the received error message. NVARCHAR(MAX).
: <code>@_ResponseStatusCode</code> The numeric status code received from the remote server. INT.
: <code>@_ResponseSuccess</code> True only if the request returned status code that represents a success. BIT.

See examples at the [[CallWebService Script Examples]] page.

= --#Commit =
[https://docs.microsoft.com/en-us/sql/t-sql/language-elements/commit-transaction-transact-sql?view=sql-server-ver15 Commits] the currently open SQL transaction in the sandbox database and starts a new transaction. The commit command can be executed at any point in the script. Note that the command does not have any parameters, i.e. there is no preceding SELECT statement before the --#Commit statement.

If the commit command is not used, the database transaction in the sandbox database is committed when the script is completed. On the other hand, if the script execution encounters an error, the SQL transaction is rolled back.

The commit command is useful in following circumstances:
* If the sandbox database is configured to allow storing permanent objects, commit can be used to preserve changes even if the script execution encounters an error.
* When the scripting is handling large amount of data, it's better to make commits during the script run, so that the database transaction log doesn't grow too large.
* Committing changes makes them visible for other users in the database.

Example:
<pre>
--#Commit
</pre>

= --#Exit =
Stops the execution of the script and gives a message to the user. This command takes one SELECT query as its parameter.

Configures the command using a SELECT statement returning two columns: the first column is for a key and the second one is for a value of that key. The values in both the key column and in the value column are of type NVARCHAR. The supported keys for this command are:<br/>
; Exit
: Defines whether to stop the script execution:
: 1 = stop execution of the current script and call the script defined by the RunScriptId parameter if it is given.
: 0 = if a value for the RunScriptId parameter is given, pause the execution of the current script and call the given script, then resume running the current script after the given script ends. If a value for RunScriptId is not given, do not pause or stop execution of the current script.
; MessageText
: Text to be shown to the user after the script execution is finished if the script finished because of the Exit command, i.e. when Exit=1. The default value is "Script execution finished.", which is shown also when the script finished normally, i.e. when Exit=0. The text is also written to the script log.
; RunScriptId
: Optional. The Id of the script to be run. Can be empty. Note that the script can call itself, so be careful not to create a looping script.
; CatchOperationExceptions
: Optional. Defines whether to stop the script execution or to continue to run the script from the next statement if an exception occurs when running the script:
: 1 = don't stop execution of the script, continue running the script from the next statement.
: 0 = stop execution of the current script and show the exception.
: The following script variables will be set and are shown in the script log:
: <code>@_ExceptionOccurred</code> If there was an exception, then this value is 1, otherwise 0. INT
: <code>@_ExceptionType</code> If there was an exception, shows the C# class name for the exception, NVARCHAR(MAX), otherwise NULL.
: <code>@_ExceptionMessage</code> If there was an exception, contains a message that would have been displayed, NVARCHAR(MAX), otherwise NULL.
: <code>@_ExceptionDetails</code> If there was an exception, contains the details that would have been displayed, including the system stack trace, NVARCHAR(MAX), otherwise NULL.

See examples at the [[Exit Script Examples]] page.

= --#GetAnalysis =

<div style="border:1px solid #dfdfdf;padding:0.5em 1em 0.5em 1em;background-color:#E7EAEC;margin:10px 0px 0px 10px;">
--#GetAnalysis command is deprecated and it will be removed in a future release. Use the more flexible [[SQL_Scripting_Commands#--.23RunQuery|--#RunQuery]] command instead.
</div>

Creates an analysis from the data which the preceding SQL statements given as parameters provide. This command can take several queries, one for every analysis to be performed. These queries and analysis results are independent from one another. Contains information about the scripts that are running and have been run.

Configures the command using a SELECT statement returning two columns: the first column is for a key and the second one is for a value of that key. The values in both the key column and in the value column are of type NVARCHAR. The supported keys for this command are:<br/>
; <Analysis Parameter>
: The --#GetAnalysis command supports the following analysis types:
* DataTableAnalysis=18: Reads a data table from SQL server and stores it in temporary table
* Etl=19
* EtlReport=20
* RunScript=25
* ExpressionAnalysis=33
; TargetTable
: The temporary table to which the analysis is to be stored. When the TargetTable parameter is used, the "Table" result type of the ForceAnalysisResultType parameter is also automatically used. If the specified temporary table already exists in the database then its contents are deleted before storing analysis.
; Show
: Optional. If TRUE or 1, the analysis is opened after the script is run. If the Show parameter is set to TRUE or 1 and the TargetTable parameter is used in the same GetAnalysis command, the analysis result is stored in the target table in tabular format.
; Title
: Optional. Name of the CSV file created when Show is TRUE or 1. Default value is the name of the analysis type.
; CatchOperationExceptions
: Optional. Defines whether to stop the script execution or to continue to run the script from the next statement if an exception occurs when running the script:
: 1 = don't stop execution of the script, continue running the script from the next statement.
: 0 = stop execution of the current script and show the exception.
: The following script variables will be set and are shown in the script log:
:<code>@_ExceptionOccurred</code> If there was an exception, then this value is 1, otherwise 0. INT
:<code>@_ExceptionType</code> If there was an exception, shows the C# class name for the exception, NVARCHAR(MAX), otherwise NULL.
:<code>@_ExceptionMessage</code> If there was an exception, contains a message that would have been displayed, NVARCHAR(MAX), otherwise NULL.
:<code>@_ExceptionDetails</code> If there was an exception, contains the details that would have been displayed, including the system stack trace, NVARCHAR(MAX), otherwise NULL.
; MaximumCount
: Used with Operation Log Analysis analysis type. Integer. The maximum amount of rows returned. Optional. Default value is 1000.

See examples at the [[GetAnalysis Script Examples]] page.

= --#ImportDataTable =
Imports data from an SQL query to a datatable. This command takes two SELECT queries as parameters.

== First Query ==
Configures the command using a SELECT statement returning two columns: the first column is for a key and the second one is for a value of that key. The values in both the key column and in the value column are of type NVARCHAR. The supported keys for this command are:<br/>
; ProjectId or ProjectName
: The id or the name of the project in which the target data table exists.
; DataTableId or DataTableName
: The id or the name of the existing/new target data table.
; Append
: Defines what to do with an existing contents of the target datatable. When value is 1, existing rows in the target datatable are not deleted (also new columns in the imported data are created to the datatable). When value is 0, existing rows in the target datatable are deleted before the import (columns are still preserved). Not used when creating a new data table.
; CatchOperationExceptions
: Optional. Defines whether to stop the script execution or to continue to run the script from the next statement if an exception occurs when running the script:
: 1 = don't stop execution of the script, continue running the script from the next statement.
: 0 = stop execution of the current script and show the exception.
: The following script variables will be set and are shown in the script log:
:<code>@_ExceptionOccurred</code> If there was an exception, then this value is 1, otherwise 0. INT
:<code>@_ExceptionType</code> If there was an exception, shows the C# class name for the exception, NVARCHAR(MAX), otherwise NULL.
:<code>@_ExceptionMessage</code> If there was an exception, contains a message that would have been displayed, NVARCHAR(MAX), otherwise NULL.
:<code>@_ExceptionDetails</code> If there was an exception, contains the details that would have been displayed, including the system stack trace, NVARCHAR(MAX), otherwise NULL.

== Second Query ==
; <nowiki><data></nowiki>
: The database query whose results are to be imported. Note that if the query doesn't return any data, the datatable is not created.

See examples at the [[ImportDataTable Script Examples]] page.

= --#ImportOdbcQuery =
Extracts data from an ODBC data source and imports it to QPR ProcessAnalyzer datatable or temporary table. Column names from the query result as used. If a column name contains illegal characters for table names, the illegal characters are converted to be underscore characters. Columns are extracted as text data. To use ImportOdbcQuery, define a SELECT statement returning two columns: the first column is for a key and the second one is for a value of that key. The values in both the key column and in the value column are of type NVARCHAR. The supported keys for this command are:

; TargetTable: The temporary table to which the data is to be imported. If not used, define the target using the ProjectId/ProjectName, DataTableId/DataTableName, and Append parameters described below.
; ProjectId / ProjectName
: The id or the name of the project in which the target data table exists.
; DataTableId / DataTableName
: The id or the name of the existing/new target data table.
; Append
: Defines what to do with an existing target data table and its contents. TRUE or any other Integer than "0" = the target data table and its existing contents are not deleted before import. If a user imports into a data table with 'Append' = FALSE or "0", the contents of the data table are deleted before the import. If a user imports into a temporary table (i.e. TargetTable) with 'Append' = FALSE or "0", then the whole temporary table is deleted before the import. Not used when creating a new data table.
; CatchOperationExceptions
: Optional. Defines whether to stop the script execution or to continue to run the script from the next statement if an exception occurs when running the script:
: 1 = don't stop execution of the script, continue running the script from the next statement.
: 0 = stop execution of the current script and show the exception.
: The following script variables will be set and are shown in the script log:
:<code>@_ExceptionOccurred</code> If there was an exception, then this value is 1, otherwise 0. INT
:<code>@_ExceptionType</code> If there was an exception, shows the C# class name for the exception, NVARCHAR(MAX), otherwise NULL.
:<code>@_ExceptionMessage</code> If there was an exception, contains a message that would have been displayed, NVARCHAR(MAX), otherwise NULL.
:<code>@_ExceptionDetails</code> If there was an exception, contains the details that would have been displayed, including the system stack trace, NVARCHAR(MAX), otherwise NULL.
; OdbcConnectionString
: The ODBC driver connection string that includes the settings needed to establish the initial connection. Mandatory. See [http://msdn.microsoft.com/en-us/library/system.data.odbc.odbcconnection.connectionstring%28v=vs.110%29.aspx?cs-save-lang=1&cs-lang=csharp#code-snippet-1 OdbcConnection.ConnectionString Property in Microsoft Development Network] for more information on the possible connection strings.
; OdbcConnectionStringKey
: [[Storing_Secrets_for_Scripts|Secret name]] for the connection string. Alternative for the OdbcConnectionString property.
; OdbcQueryString
: The SQL query string. Mandatory. Note that if the query doesn't return any data, the target data table or temporary table is not created.
; QueryExecutionTimeout
: Defines timeout in seconds for the ODBC command execution. If not specified, default value is 600 seconds.
; ExecuteInClientSide
: Defines whether the command is executed from the QPR ScriptLauncher or from the server. This parameter is used when there is no server connection available, for example. TRUE or 1, the query is executed in the QPR ScriptLauncher. FALSE or 0, the query is executed in the server. Supports only data table as the import destination. If 'TargetTable' has been defined as the import destination and the value of this parameter is given as TRUE or 1, you will receive an error message. Optional. Default value is FALSE.

See examples in the [[ImportOdbcQuery Script Examples]] page.

= --#ImportOleDbQuery =
Extracts data from an OLE DB data source and imports it to QPR ProcessAnalyzer datatable or a temporary table. Column names from the query result are used. It is possible to both create new datatables as well as modify existing datatables with this command. To use the ImportOleDbQuery, define a SELECT statement returning two columns: the first column is for a key and the second one is for a value of that key. The values in both the key column and in the value column are of type NVARCHAR. The supported keys for this command are:

; TargetTable
: The temporary table to which the data is to be imported. If not used, define the target using the ProjectId/ProjectName, DataTableId/DataTableName, and Append parameters described below.
; ProjectId / ProjectName
: The id or the name of the project in which the target data table exists.
; DataTableId / DataTableName
: The id or the name of the existing/new target data table.
; Append
: Defines what to do with an existing target data table and its contents. TRUE or any other Integer than "0" = the target data table and its existing contents are not deleted before import. If a user imports into a data table with 'Append' = FALSE or "0", the contents of the data table are deleted before the import. If a user imports into a temporary table(i.e. TargetTable) with 'Append' = FALSE or "0", then the whole temporary table is deleted before the import. Not used when creating a new data table.
; CatchOperationExceptions
: Optional. Defines whether to stop the script execution or to continue to run the script from the next statement if an exception occurs when running the script:
: 1 = don't stop execution of the script, continue running the script from the next statement.
: 0 = stop execution of the current script and show the exception.
: The following script variables will be set and are shown in the script log:
:<code>@_ExceptionOccurred</code> If there was an exception, then this value is 1, otherwise 0. INT
:<code>@_ExceptionType</code> If there was an exception, shows the C# class name for the exception, NVARCHAR(MAX), otherwise NULL.
:<code>@_ExceptionMessage</code> If there was an exception, contains a message that would have been displayed, NVARCHAR(MAX), otherwise NULL.
:<code>@_ExceptionDetails</code> If there was an exception, contains the details that would have been displayed, including the system stack trace, NVARCHAR(MAX), otherwise NULL.
; OleDbConnectionString
: The OLE DB connection string that includes the settings needed to establish the initial connection. Mandatory. See [http://msdn.microsoft.com/en-us/library/system.data.oledb.oledbconnection.connectionstring%28v=vs.110%29.aspx OleDbConnection.ConnectionString Property in Microsoft Development Network] for more information on the possible connection strings.
; OleDbQueryString
: The SQL query string. Mandatory. Note that if the query doesn't return any data, the target data table or temporary table is not created.
; QueryExecutionTimeout
: Defines timeout in seconds for the OLE DB command execution. If not specified, default value is 600 seconds.
; ExecuteInClientSide
: Defines whether the command is executed from the QPR ScriptLauncher or from the server. This parameter is used when there is no server connection available, for example. TRUE or 1, the query is executed in the QPR ScriptLauncher. FALSE or 0, the query is executed in the server. Supports only data table as the import destination. If 'TargetTable' has been defined as the import destination and the value of this parameter is given as TRUE or any other Integer than "0", you will receive an error message. Optional. Default value is FALSE.

See examples at the [[ImportOleDbQuery Script Examples]] page.

= --#ImportSalesforceQuery =
Extracts data from the Salesforce cloud using its REST API and imports the data to a datatable. The command takes one SELECT query as its parameter. If the query doesn't return any data, the target data table or temporary table is not created.

More information about the Salesforce REST API: https://developer.salesforce.com/docs/atlas.en-us.api_rest.meta/api_rest/intro_rest.htm.

== Query ==
Configures the command using a SELECT statement returning two columns: the first column is for a key and the second one is for a value of that key. The values in both the key column and in the value column are of type NVARCHAR. The supported keys for this command are:<br/>
; TargetTable
: Temporary table to which the data is imported. If not used, define the target using the ProjectId/ProjectName, DataTableId/DataTableName, and Append parameters described below.
; ProjectId / ProjectName
: Id or the name of the project in which the target datatable is located.
; DataTableId / DataTableName
: Id or the name of the target data table. If DataTableName is used, the ProjectId or ProjectName can also be used to define the project where the datatable is located.
; Append
: Defines what to do with an existing target data table contents. TRUE or 1, existing contents of the target datatable is not deleted in the import. When FALSE or 0, existing contents of the target datatable are deleted before importing new data. Not used when creating a new data table.
; CatchOperationExceptions
: Optional. Defines whether to stop the script execution or to continue to run the script from the next statement if an exception occurs when running the script:
: 1 = don't stop execution of the script, continue running the script from the next statement.
: 0 = stop execution of the current script and show the exception.
: The following script variables will be set and are shown in the script log:
:<code>@_ExceptionOccurred</code> If there was an exception, then this value is 1, otherwise 0. INT
:<code>@_ExceptionType</code> If there was an exception, shows the C# class name for the exception, NVARCHAR(MAX), otherwise NULL.
:<code>@_ExceptionMessage</code> If there was an exception, contains a message that would have been displayed, NVARCHAR(MAX), otherwise NULL.
:<code>@_ExceptionDetails</code> If there was an exception, contains the details that would have been displayed, including the system stack trace, NVARCHAR(MAX), otherwise NULL.
; SalesforceUser
: Username for the Salesforce cloud.
; SalesforcePW
: Password for the Salesforce cloud.
; SalesforcePWKey
: [[Storing_Secrets_for_Scripts|Secret name]] for the stored Salesforce password. Alternative for the SalesforcePW property.
; SalesforceUrl
: Optional. Salesforce web service url.
; SalesforceQueryMode
: Optional. Determines which Salesforce query function to use. One of the following values (1, 2 or 3) can be used:
: 1: '''QueryAll''' (default): Executes specified SOQL query, except unlike ''Query'', ''QueryAll'' returns records that are deleted because of a merge or delete. More information: https://developer.salesforce.com/docs/atlas.en-us.api_rest.meta/api_rest/resources_queryall.htm.
: 2: '''Query''': Executes the specified SOQL query. More information: https://developer.salesforce.com/docs/atlas.en-us.api_rest.meta/api_rest/resources_query.htm)
: 3: '''sObject Describe''': Completely describes the individual metadata at all levels for the specified object. For example, this can be used to retrieve the fields, URLs, and child relationships for the Account object. More information: https://developer.salesforce.com/docs/atlas.en-us.api_rest.meta/api_rest/resources_sobject_describe.htm).
; SalesforceQuery
: Query to run in the Salesforce cloud to fetch the data, defined as SOQL (Salesforce Object Query Language). More information: https://developer.salesforce.com/docs/atlas.en-us.236.0.soql_sosl.meta/soql_sosl/sforce_api_calls_soql_sosl_intro.htm.
; SalesforceQueryRetries
: Optional. Number of retries to attempt if the Salesforce query doesn't succeed. Default value is 3.
; SalesforceQueryRetryWait
: Optional. Number of milliseconds to wait between query retries. Default is 3000 ms.
; SalesforceBatchSize
: Optional. Data is queried from Salesforce in batches, and this setting determines the batch size. The value can be between 200 and 2000, and the default value is 500.

== Notes ==
If you get error ''INVALID_TYPE sObject type 'Objectname' is not supported'':
* Check that the object in question exists or that the object name is correct.
* Verify that the Salesforce user has rights to the object.
** You have to give access to the new custom objects and VisualForce pages from the user's profile, and you have to check the "Customize Application" checkbox under the same profile (https://developer.salesforce.com/forums/?id=906F00000008qG6IAI). Contact your Salesforce administrator.
* The Salesforce user may need extra license to access the object. Special 3rd party custom objects may need separate license. Contact your Salesforce application administrator.

See examples at the [[ImportSalesforceQuery Script Examples]] page.

= --#ImportSapQuery =
Extracts data from an SAP system and imports it to QPR ProcessAnalyzer datatable or a temporary table. Column names from the query result are used. If a column name contains illegal characters for table names, the illegal characters are converted to be underscore characters (e.g. "sap:Owner" -> "sap_Owner"). Columns are extracted as text data. Note that using this command requires [[QPR_ProcessAnalyzer_ScriptLauncher#Installing_SAP_NetWeaver_RFC_Library|installing SAP NetWeaver RFC Library]].

To use the ImportSapQuery command, define a SELECT statement returning two columns: the first column is for a key and the second one is for a value of that key. The values in both the key column and in the value column are of type NVARCHAR. The supported keys for this command are:
; TargetTable
: If this parameter is given, store the results into a temporary SQL table in the ETL sandbox. If the TargetTable parameter is not given, use either the ProjectId or ProjectName parameters.
; ProjectId / ProjectName
: The id or the name of the project in which the target data table exists.
; DataTableId / DataTableName
: The id or the name of the existing/new target data table.
; Append
: Defines what to do with an existing target data table and its contents. TRUE or any other Integer than "0" = the target data table and its existing contents are not deleted before import. If a user imports into a data table with 'Append' = FALSE or "0", the contents of the data table are deleted before the import. If a user imports into a temporary table (i.e. TargetTable) with 'Append' = FALSE or "0", then the whole temporary table is deleted before the import. Not used when creating a new data table.
; ConvertDataTypes
: List of SAP data types that are converted into respective data types supported by SQL Server instead of using NVARCHAR. Defined by listing the data type identifier characters in any order. Available data type identifying characters are '''IFPCDTNX'''. If not defined, all data is converted to NVARCHAR. Example: ''IFP'' (convert only numeric data types: Integer, Float, Packed number) ([[Importing_Data_from_SAP|more information]]).
; CatchOperationExceptions
: Optional. Defines whether to stop the script execution or to continue to run the script from the next statement if an exception occurs when running the script:
: 1 = don't stop execution of the script, continue running the script from the next statement.
: 0 = stop execution of the current script and show the exception.
: The following script variables will be set and are shown in the script log:
:<code>@_ExceptionOccurred</code> If there was an exception, then this value is 1, otherwise 0. INT
:<code>@_ExceptionType</code> If there was an exception, shows the C# class name for the exception, NVARCHAR(MAX), otherwise NULL.
:<code>@_ExceptionMessage</code> If there was an exception, contains a message that would have been displayed, NVARCHAR(MAX), otherwise NULL.
:<code>@_ExceptionDetails</code> If there was an exception, contains the details that would have been displayed, including the system stack trace, NVARCHAR(MAX), otherwise NULL.
;ImportChunkSize
:Specifies the size of the chunks of data used when importing the data from SAP to PA Server. The value represents the approximate maximum number of data cells in each chunk (consisting tables of <number of rows> * <number of columns> data cells. Default value is 200000. Smaller value causes big imports to be split into more chunks taking more time in total, but it makes importing of each chunk faster possibly helping in some timeout situations.
; SapUser
: SAP username used to connect to SAP. Mandatory. Corresponds to the "USER" constant on SAP side. See the [http://help.sap.com/saphelp_nw04/helpdata/en/e9/23c80d66d08c4c8c044a3ea11ca90f/content.htm SAP .NET Connector documentation] for more info.
; SapPW
: Password of the SAP user used to connect to SAP. Mandatory. Corresponds to the "PASSWD" constant on SAP side. See the [http://help.sap.com/saphelp_nw04/helpdata/en/e9/23c80d66d08c4c8c044a3ea11ca90f/content.htm SAP .NET Connector documentation] for more info.
; SapPWKey
: [[Storing_Secrets_for_Scripts|Secret name]] for the stored SAP password. Alternative for the SapPW property.
; SapClient
: The SAP backend client. Mandatory. Corresponds to the "CLIENT" constant on SAP side. See the [http://help.sap.com/saphelp_nw04/helpdata/en/e9/23c80d66d08c4c8c044a3ea11ca90f/content.htm SAP .NET Connector documentation] for more info.
; SapAppServerHost
: The hostname or IP of the specific SAP application server, to which all connections shall be opened. Mandatory if SapMessageServerHost is not defined. Corresponds to the "ASHOST" constant on SAP side. See the [http://help.sap.com/saphelp_nw04/helpdata/en/e9/23c80d66d08c4c8c044a3ea11ca90f/content.htm SAP .NET Connector documentation] for more info.
; SapMessageServerHost
: The hostname or IP of the SAP system’s message server (central instance). Mandatory if SapAppServerHost is not defined. Corresponds to the "MSHOST" constant on SAP side. See the [http://help.sap.com/saphelp_nw04/helpdata/en/e9/23c80d66d08c4c8c044a3ea11ca90f/content.htm SAP .NET Connector documentation] for more info.
; SapSystemNumber
: The SAP system’s system number. Mandatory if SapSystemID is not defined. Corresponds to the "SYSNR" constant on SAP side. See the [http://help.sap.com/saphelp_nw04/helpdata/en/e9/23c80d66d08c4c8c044a3ea11ca90f/content.htm SAP .NET Connector documentation] for more info.
; SapSystemID
: The SAP system’s three-letter system ID. Mandatory if SapSystemNumber is not defined. Corresponds to the "SYSID" constant on SAP side. See the [http://help.sap.com/saphelp_nw04/helpdata/en/e9/23c80d66d08c4c8c044a3ea11ca90f/content.htm SAP .NET Connector documentation] for more info.
; ExecuteInClientSide
: Defines whether the command is executed from the QPR ScriptLauncher or from the server. This parameter is used when there is no server connection available, for example. TRUE or 1, the query is executed in the QPR ScriptLauncher. FALSE or 0, the query is executed in the server. Supports only data table as the import destination. If 'TargetTable' has been defined as the import destination and the value of this parameter is given as TRUE or any other Integer than "0", you will receive an error message. Optional. Default value is FALSE.
; SapLanguage
: SAP language used. Default value is "EN". Optional. Corresponds to the "LANG" constant on SAP side. See the [http://help.sap.com/saphelp_nw04/helpdata/en/e9/23c80d66d08c4c8c044a3ea11ca90f/content.htm SAP .NET Connector documentation] for more info.
; SapPoolSize
: The maximum number of RFC connections that this destination will keep in its pool. Default value is "5". Optional. Corresponds to the "POOL_SIZE" constant on SAP side. See the [http://help.sap.com/saphelp_nw04/helpdata/en/e9/23c80d66d08c4c8c044a3ea11ca90f/content.htm SAP .NET Connector documentation] for more info.
; SapMaxPoolSize
: In order to prevent an unlimited number of connections to be opened, you can use this parameter. Default value is "10". Optional. Corresponds to the "MAX_POOL_SIZE" constant on SAP side. See the [http://help.sap.com/saphelp_nw04/helpdata/en/e9/23c80d66d08c4c8c044a3ea11ca90f/content.htm SAP .NET Connector documentation] for more info.
; SapConnectionIdleTimeout
: If a connection has been idle for more than SapIdleTimeout seconds, it will be closed and removed from the connection pool upon checking for idle connections or pools. Default value is "600". Optional. Corresponds to the "IDLE_TIMEOUT" constant on SAP side. See the [http://help.sap.com/saphelp_nw04/helpdata/en/e9/23c80d66d08c4c8c044a3ea11ca90f/content.htm SAP .NET Connector documentation] for more info.
; SapRouter
: List of host names and service names / port numbers for the SAPRouter in the following format: /H/hostname/S/portnumber. Optional. Corresponds to the "SAPROUTER" constant on SAP side. See the [http://help.sap.com/saphelp_nw04/helpdata/en/e9/23c80d66d08c4c8c044a3ea11ca90f/content.htm SAP .NET Connector documentation] for more info.
; SapLogonGroup
: The logon group from which the message server shall select an application server. Optional. Corresponds to the "GROUP" constant on SAP side. See the [http://help.sap.com/saphelp_nw04/helpdata/en/e9/23c80d66d08c4c8c044a3ea11ca90f/content.htm SAP .NET Connector documentation] for more info.
; SapQueryMode
: If this number is set to "1", then the query result will have the SAP Table field names as data table column names and actual data rows as rows. If this is set to "3", the query result will get the field descriptions from the SAP query using NO_DATA parameter, i.e. the returned columns are the following (in this order): Field, Type, Description, Length, Offset. Default value is "1". Optional. See the [http://help.sap.com/saphelp_nw04/helpdata/en/e9/23c80d66d08c4c8c044a3ea11ca90f/content.htm SAP .NET Connector documentation] for more info.
; SapQueryTable
: Name of the SAP table to be extracted. Specifies the value for the parameter QUERY_TABLE in tab: 'Import' or function module 'rfc_read_table' in SAP. Mandatory. See the [http://help.sap.com/saphelp_nw04/helpdata/en/e9/23c80d66d08c4c8c044a3ea11ca90f/content.htm SAP .NET Connector documentation] for more info. Note that if the query doesn't return any data, the target data table or temporary table is not created.
; SapRowcount
: The maximum amount of rows to fetch. Specifies the value for parameter ROWCOUNT in tab: 'Import' or function module 'rfc_read_table' in SAP. Optional. See the [http://help.sap.com/saphelp_nw04/helpdata/en/e9/23c80d66d08c4c8c044a3ea11ca90f/content.htm SAP .NET Connector documentation] for more info.
; SapRowskips
: The number of rows to skip. Specifies the value for parameter ROWSKIPS in tab: 'Import' or function module 'rfc_read_table'. in SAP. Optional. See the [http://help.sap.com/saphelp_nw04/helpdata/en/e9/23c80d66d08c4c8c044a3ea11ca90f/content.htm SAP .NET Connector documentation] for more info.
; SapWhereClause
: A comma separated list of WHERE clause elements passed for the SapQueryTable. Can be used with or without the SapWhereClauseSelect parameter. If used together with the SapWhereClauseSelect parameter, use the SapWhereClause parameter first. NOTE: The default maximum length for the Where Clause string is 72 characters in SAP, so the recommended maximum length of the SapWhereClause value is also 72 characters. In effect, specifies the value for parameter OPTIONS in tab: 'Import' or function module 'rfc_read_table' in SAP. Optional. See the [http://help.sap.com/saphelp_nw04/helpdata/en/e9/23c80d66d08c4c8c044a3ea11ca90f/content.htm SAP .NET Connector documentation] for more info.
; SapWhereClauseSelect
: The SELECT query to be executed in QPR ProcessAnalyzer sandbox. Used with or without the SapWhereClause parameter to pass WHERE clauses to SapQueryTable. If used together with the SapWhereClause parameter, use the SapWhereClause parameter first. The query is expected to return a table with at least one column, as the contents from the rows in the first column of the table are concatenated together to form the WHERE clause in SAP RFC_ReadTable. Therefore, it's recommended to first create the table with the WHERE clauses into a temporary table. In addition, it's recommended to have an order number column in the table and use that in the SELECT query to make sure the WHERE clause elements are concatenated in the correct order. The default maximum length for Where Clause string is 72 characters in SAP, so the recommended maximum length for the WHERE clause string in each row of the table is also 72. In effect, specifies the value for parameter OPTIONS in tab: 'Import' or function module 'rfc_read_table' in SAP. Optional. The contents up to the first 10 rows in the first column of the SELECT query are shown in the QPR ProcessAnalyzer [[QPR_ProcessAnalyzer_Logs#Script_Log|Script Log]]. See the [http://help.sap.com/saphelp_nw04/helpdata/en/e9/23c80d66d08c4c8c044a3ea11ca90f/content.htm SAP .NET Connector documentation] for more info.<br/>
; SapFieldNames
: A comma separated list of field names for columns to be imported. Default value is empty, resulting in all columns being imported. Specifies the value for parameter FIELDNAME in tab: 'Tables' for table 'FIELDS' for function module 'rfc_read_table' in SAP. Optional. See the [http://help.sap.com/saphelp_nw04/helpdata/en/e9/23c80d66d08c4c8c044a3ea11ca90f/content.htm SAP .NET Connector documentation] for more info.
; SapFunction
: If you define a value for this parameter, then the new value specifies the SAP function that is called inside the #ImportSapQuery command. Optional. The default value is RFC_READ_TABLE. Another possible value is BBP_RFC_READ_TABLE. See the [http://help.sap.com/saphelp_nw04/helpdata/en/e9/23c80d66d08c4c8c044a3ea11ca90f/content.htm SAP .NET Connector documentation] for more info.
; UseAnyAsColumnType
: Determines datatable column data types for the created columns. When ''true'', "Any" type of columns are created (resulting into SQL_variant columns in SQL server), and when ''false'', data types depend on the ConvertDataTypes parameter. Default value is true when running the import in SQL script, and otherwise default value is false.
;AliasUser
:
;AppServerService
:
;CharacterFaultIndicatorToken
:
;Codepage
:
;GatewayHost
:
;GatewayService
:
;IdleCheckTime
:
;LogonCheck
:
;MaxPoolWaitTime
:
;MessageServerService
:
;Name
:
;NoCompression
:
;OnCharacterConversionError
:
;PartnerCharSize
:
;PasswordChangeEnforced
:
;ProgramId
:
;R3Name
:
;RegistrationCount
:
;RepositoryDestination
:
;RepositoryPassword
:
;RepositorySncMyName
:
;RepositoryUser
:
;RepositoryX509Certificate
:
;SapSso2Ticket
:
;SncLibraryPath
:Full path including file name of the [[Importing_Data_from_SAP#SNC_encrypted_connection|SNC]] shared library to be used.
;SncMode
: Determines whether connections will be secured with [[Importing_Data_from_SAP#SNC_encrypted_connection|SNC]]. Value '''0''' doesn't use SNC (default) and value '''1''' uses SNC.
;SncMyName
:Token/identifier representing the external RFC program. In most cases this can be omitted. The installed [[Importing_Data_from_SAP#SNC_encrypted_connection|SNC]] solution usually knows its own SNC name. Only for solutions supporting “multiple identities”, you may Varies depending on the installed SNC solution (Secude, Kerberos, NTLM, etc). Example for Secude: p/secude:CN=ALEREMOT SAP Online Help 09.09.2014 SAP .NET Connector 3.0 41 need to specify the identity to be used for this particular destination/server. E, O=Mustermann-AG, C=DE
;SncPartnerName
:The backend's [[Importing_Data_from_SAP#SNC_encrypted_connection|SNC]]name.
;SncPartnerNames
:
;SncQop
:Quality of service to be used for SNC communication of this particular destination/server. One of the following values:
* 1: Digital signature
* 2: Digital signature and encryption
* 3: Digital signature, encryption, and user authentication
* 8: Default value defined by back-end system
* 9: Maximum value that the current security product supports
;SystemIds
:
;UseSapGui
:
;X509Certificate
:

See examples at the [[ImportSapQuery Script Examples]] page.

= --#ImportSqlQuery =
Extracts data from an ADO.Net source (which usually is an SQL Server database) and imports it to QPR ProcessAnalyzer datatable or a temporary table. Column names from the query result are used. It is possible to both create new Data Tables as well as modify existing datatables with this command. To use the ImportSqlQuery command, a SELECT statement returning two columns: the first column is for a key and the second one is for a value of that key. The values in both the key column and in the value column are of type NVARCHAR. The supported keys for this command are:

; TargetTable
: The temporary table to which the data is to be imported. If not used, define the target using the ProjectId/ProjectName, DataTableId/DataTableName, and Append parameters described below.
; ProjectId / ProjectName
: The id or the name of the project in which the target data table exists.
; DataTableId / DataTableName
: The id or the name of the existing/new target data table.
; Append
: Defines what to do with an existing target data table and its contents. TRUE or any other Integer than "0" = the target data table and its existing contents are not deleted before import. If a user imports into a data table with 'Append' = FALSE or "0", the contents of the data table are deleted before the import. If a user imports into a temporary table (i.e. TargetTable) with 'Append' = FALSE or "0", then the whole temporary table is deleted before the import. Not used when creating a new data table.
; CatchOperationExceptions
: Optional. Defines whether to stop the script execution or to continue to run the script from the next statement if an exception occurs when running the script:
: 1 = don't stop execution of the script, continue running the script from the next statement.
: 0 = stop execution of the current script and show the exception.
: The following script variables will be set and are shown in the script log:
:<code>@_ExceptionOccurred</code> If there was an exception, then this value is 1, otherwise 0. INT
:<code>@_ExceptionType</code> If there was an exception, shows the C# class name for the exception, NVARCHAR(MAX), otherwise NULL.
:<code>@_ExceptionMessage</code> If there was an exception, contains a message that would have been displayed, NVARCHAR(MAX), otherwise NULL.
:<code>@_ExceptionDetails</code> If there was an exception, contains the details that would have been displayed, including the system stack trace, NVARCHAR(MAX), otherwise NULL.
; SqlConnectionString
: The SQL connection string that includes the settings needed to establish the initial connection. Mandatory. See [http://msdn.microsoft.com/en-us/library/system.data.sqlclient.sqlconnection.connectionstring%28v=vs.110%29.aspx SqlConnection.ConnectionString Property in Microsoft Development Network] for more information on the connection parameters.
; SqlQueryString
: The SQL query string. Mandatory. Note that if the query doesn't return any data, the target data table or temporary table is not created.
; QueryExecutionTimeout
: Defines timeout in seconds for the SQL command execution. If not specified, default value is 600 seconds.
; ExecuteInClientSide
: Defines whether the command is executed from the QPR ScriptLauncher or from the server. This parameter is used when there is no server connection available, for example. TRUE or 1, the query is executed in the client side. FALSE or 0, the query is executed in the server side. Supports only data table as the import destination. If 'TargetTable' has been defined as the import destination and the value of this parameter is given as TRUE or any other Integer than "0", you will receive an error message. Optional. Default value is FALSE.

See examples at the [[ImportSqlQuery Script Examples]] page.

= --#Run =
Runs another script with specified parameters. This command can take multiple SELECT queries which are passed as parameters to the called script. The first SELECT configures the script call by defining the script id to be called.

== First Query ==
Configures the command using a SELECT statement returning two columns: the first column is for a key and the second one is for a value of that key. The values in both the key column and in the value column are of type NVARCHAR. The supported keys for this command are:<br/>
; ScriptId
: Mandatory. The Id of the called script.
; CatchOperationExceptions
: Optional. Defines whether to stop the script execution or to continue to run the script from the next statement if an exception occurs when running the script:
: 1 = don't stop execution of the script, continue running the script from the next statement.
: 0 = stop execution of the current script and show the exception.
: The following script variables will be set and are shown in the script log:
: <code>@_ExceptionOccurred</code> If there was an exception, then this value is 1, otherwise 0. INT
: <code>@_ExceptionType</code> If there was an exception, shows the C# class name for the exception, NVARCHAR(MAX), otherwise NULL.
: <code>@_ExceptionMessage</code> If there was an exception, contains a message that would have been displayed, NVARCHAR(MAX), otherwise NULL.
: <code>@_ExceptionDetails</code> If there was an exception, contains the details that would have been displayed, including the system stack trace, NVARCHAR(MAX), otherwise NULL.

== Following Queries ==
Subsequent queries are optional and they are used for passing parameters to the called script. Maximum number of arguments is 10. Each argument is created as a temporary table with names '''#_Arg1''', '''#_Arg2''', '''#_Arg10'''. In the created temporary tables, all columns are of the type SQL_VARIANT. If the column names have not been specified, then '''Value_0''', '''Value_1''' etc. are used.
The possible arguments are as follows:
* '''@_Argv''': Number of provided parameters (between 0 to 10) (type iNT)
* '''#_Arg1''', '''#_Arg2''', ... '''#_Arg10''': arguments passed to that script

Each argument exists in the called script until the next --#Run command is executed in that script. After the called script has finished, the main script continues its execution.

See examples at the [[Run Script Examples]] page.

= --#RunQuery =
Runs an [[Web_API:_Expression/query|expression language query]], and stores results to a [[QPR_ProcessAnalyzer_Project_Workspace#Datatables|datatable]] or to a temporary table in the scripting database. Following parameters can be used in the command:
* '''Configuration''': Expression language query to run, written in JSON as specified in [[Web_API:_Expression/query|Web API: Expression/query]]. Queries can be created by using a [[QPR_ProcessAnalyzer_Chart|chart]] where to open the '''Query''' (in the '''Advanced''' tab). It will show the query made by chart that's compatible with what can be specified in the ''Configuration'' parameter.
* '''TargetTable''': When specified, results are stored to a temporary table with that name in the scripting sandbox. The temporary table can be read using the subsequent commands. When the script ends, temporary tables are automatically removed.
* '''DatatableId''': When specified, data is stored to the defined existing datatable. When using datatable id, ProjectName or ProjectId parameter don't need to be defined.
* '''DataTableName''': When specified, data is stored to the datatable with that name, located in the same project as the script. If you want to use different project, specify either the ProjectName or ProjectId parameter.
* '''ProjectName''': Specifies a project by name where the results datatable is stored. Used together with the DataTableName parameter.
* '''ProjectId''': Specifies a project by id where the results datatable is stored. Used together with the DataTableName parameter.

See [[RunQuery Script Examples]].

= --#SendEmail =
Sends an e-mail and writes a message to script log whether sending the email was successful or not. Script execution continues even when the sending isn't successful.

Configures the command using a SELECT statement returning two columns: the first column is for a key and the second one is for a value of that key. The values in both the key column and in the value column are of type NVARCHAR. The supported keys for this command are:<br/>
; CatchOperationExceptions
: Optional. Defines whether to stop the script execution or to continue to run the script from the next statement if an exception occurs when running the script:
: 1 = don't stop execution of the script, continue running the script from the next statement.
: 0 = stop execution of the current script and show the exception.
: The following script variables will be set and are shown in the script log:
:<code>@_ExceptionOccurred</code> If there was an exception, then this value is 1, otherwise 0. INT
:<code>@_ExceptionType</code> If there was an exception, shows the C# class name for the exception, NVARCHAR(MAX), otherwise NULL.
:<code>@_ExceptionMessage</code> If there was an exception, contains a message that would have been displayed, NVARCHAR(MAX), otherwise NULL.
:<code>@_ExceptionDetails</code> If there was an exception, contains the details that would have been displayed, including the system stack trace, NVARCHAR(MAX), otherwise NULL.
'''E-mail Parameters'''
; EmailFrom
: Defines the from address for this e-mail message. Mandatory.
; EmailTo
: Defines the recipient(s) for this e-mail message given in a list separated by comma. Mandatory.
; EmailSubject
: Defines the subject of the email. Default value is empty. Optional.
; EmailBody
: Defines the message body. Default value is empty. Optional.
; EmailCc
: Defines the carbon copy recipient(s) for this e-mail message given in a list separated by comma. Optional.
; EmailBcc
: Defines the blind carbon copy recipient(s) for this e-mail message given in a list separated by comma. Optional.
; EmailIsBodyHtml
: Defines whether the e-mail message body is in HTML. TRUE or any other Integer than "0" = body is in HTML, FALSE or "0" = body is not in HTML. Default value is FALSE. Optional.
; EmailSender
: Defines the sender's address for this e-mail message. Default value is empty. Optional.
; EmailReplyTo
: Defines the ReplyTo address(es) for the mail message given in a list separated by comma. Optional.
; EmailPriority
: Defines the priority of this e-mail message. Possible values are "High", "Normal", and "Low". Default value is "Normal". Optional.
; EmailDeliveryNotification
: Defines the delivery notifications for this e-mail message. Possible values are "Delay", "Never", "None", "OnFailure", and "OnSuccess". Default value is "None". Optional.
; EmailBodyEncoding
: Defines the encoding used to encode the message body. Supported encodings are listed in the "Remarks" section at http://msdn.microsoft.com/en-us/library/System.Text.Encoding.aspx. UTF8 is used by default. Optional.
; EmailSubjectEncoding
: Defines the encoding used for the subject content for this e-mail message. Supported encodings are listed in the "Remarks" section at http://msdn.microsoft.com/en-us/library/System.Text.Encoding.aspx. UTF8 is used by default. Optional.
; EmailAttachmentQuery
: Defines a query to fetch the parameters for adding attachments to the email. Each row (except the header row) in the query result corresponds to one attachment. The result must contain the following columns in this order: Name of the attachment, Content for the attachment (Sent as-is without any modifications. Supports binary values.), Media type (supported types are text/plain, text/html, text/xml, and image/jpeg), and Creation time (SQL datetime). Names of the columns do not matter. If the result doesn't contain some of the columns, an error is written into the Progress log, and the email is not sent. Optional.

'''SMTP Server Parameters'''
; SmtpServer
: Defines the hostname or the IP address of the server. Mandatory for the first occurrence of the SendEmail command during script execution.
; SmtpPort
: Defines the port of the SMTP server. Default value is "25". Optional.
; SmtpAuthenticationUsername
: Defines the user name for the SMTP server. Note that the user name is in plain text and visible to all users who have access to the script. Optional.
; SmtpAuthenticationPassword
: Defines the password for the SMTP server. Note that the password is in plain text and visible to all users who have access to the script. Optional.
; SmtpEnableSSL
: Defines whether SSL should be enabled for the SMTP connection. TRUE or any other Integer than "0" = SSL is enabled, FALSE or "0" = SSL is not enabled. Default value is "FALSE". Optional.

See examples at the [[SendEmail Script Examples]] page.

= --#ShowReport =
Outputs result of an SQL query to a CSV file when running script from [[QPR_ProcessAnalyzer_ScriptLauncher|QPR ProcessAnalyzer ScriptLauncher]]. This command takes two SELECT queries as parameters.
== First Query ==
SQL query which results are shown.
; <nowiki><data></nowiki>
: Mandatory. The database query whose results are to be returned.

== Second Query ==
Configures the command using a SELECT statement returning two columns: the first column is for a key and the second one is for a value of that key. The values in both the key column and in the value column are of type NVARCHAR. The supported keys for this command are:<br/>
; <Analysis Parameter>
: Optional. The analysis parameters given for the operation. Some suggested parameters to be used:
:; Title
:: The name of the created CSV file.
:; MaximumCount
:: The maximum number of rows to show (0 = all, default = 1000).

; CatchOperationExceptions: Optional. Defines whether to stop the script execution or to continue to run the script from the next statement if an exception occurs when running the script:
: 1 = don't stop execution of the script, continue running the script from the next statement.
: 0 = stop execution of the current script and show the exception.
: The following script variables will be set and are shown in the script log:
:<code>@_ExceptionOccurred</code> If there was an exception, then this value is 1, otherwise 0. INT
:<code>@_ExceptionType</code> If there was an exception, shows the C# class name for the exception, NVARCHAR(MAX), otherwise NULL.
:<code>@_ExceptionMessage</code> If there was an exception, contains a message that would have been displayed, NVARCHAR(MAX), otherwise NULL.
:<code>@_ExceptionDetails</code> If there was an exception, contains the details that would have been displayed, including the system stack trace, NVARCHAR(MAX), otherwise NULL.

See examples at the [[ShowReport Script Examples]] page.

= --#StartBackground =
Continues the script run in background, i.e. the parent script execution completes and the rest of the script execution continues. When running a script in the background, it cannot output any results using the ShowReport command or GetAnalysis with the Show parameter. It's possible to terminate scripts that run in the background via the [[QPR_ProcessAnalyzer_Logs#Task_Log|Task log]]. No also that a script running in the background cannot execute in the client side mode.

Takes one SELECT query as a parameter. Following parameter is supported:

; Enabled
: Boolean value defining whether the script is run in background starting from this command. TRUE = run in background, FALSE = don't run in background. Default value is TRUE.

See examples at the [[StartBackground Script Examples]] page.

= --#WriteLog =
Adds the first column values from the preceding SQL statements to the log that is shown after the whole script execution is completed. In addition to the WriteLog command, you can also use the [https://docs.microsoft.com/en-us/sql/t-sql/language-elements/print-transact-sql?view=sql-server-ver15 Print SQL statement] to generate log entries into the script execution log. The difference to the WriteLog command is that the Print statement can use also variables.

See examples at the [[WriteLog Script Examples]] page.

__NOTOC__
[[Category: QPR ProcessAnalyzer]]

ExtractSap Function

2025-01-14T15:41:10Z

MarHink:

ExtractSap function imports data from an SAP R/3 system using its RFC interface and returns a DataFlow. The function is in the generic context and also in the project context. When using the secure strings, the function need to be called in the project context as the secure strings are project specific. Using the ExtractSap function requires installing [[QPR_ProcessAnalyzer_ScriptLauncher#Installing_SAP_NetWeaver_RFC_Library|SAP NetWeaver RFC Library]]. More information about [[Importing_Data_from_SAP|connecting to SAP]].

Parameters:
* '''UseGateway''': Boolean value indicating whether data extraction should be performed through the gateway (''true'') or by QPR ProcessAnalyzer Server (''false'', default value). The gateway may be needed to access on-premise systems that are not available in the public network.
* '''User''': SAP username used to connect to SAP. Mandatory. Corresponds to the "USER" constant in SAP.
* '''Password''': Password for the SAP user used to connect to SAP. Mandatory. Corresponds to the "PASSWD" constant in SAP.
* '''PasswordKey''': [[Storing_Secrets_for_Scripts|Secret name]] for the stored SAP password. Alternative for the Password property.
* '''Client''': The SAP backend client. Mandatory. Corresponds to the "CLIENT" constant in SAP.
* '''AppServerHost''': Hostname or IP of the specific SAP application server, to which all connections shall be opened. Mandatory if MessageServerHost is not defined. Corresponds to the "ASHOST" constant in SAP.
* '''MessageServerHost''': Hostname or IP of the SAP system’s message server (central instance). Mandatory if AppServerHost is not defined. Corresponds to the "MSHOST" constant in SAP.
* '''SystemNumber''': SAP system’s system number. Mandatory if SystemID is not defined. Corresponds to the "SYSNR" constant in SAP.
* '''SystemID''': SAP system’s three-letter system ID. Mandatory if SystemNumber is not defined. Corresponds to the "SYSID" constant in SAP.
* '''Options''': Array of clause elements. Specifies the value for parameter OPTIONS in tab 'Import' or function module 'rfc_read_table' in SAP. NOTE: maximum length for one string is 72 characters(SAP limit). Optional. Example: ["VBELN BETWEEN '0000017448'" ,"AND '0000017450'"].
* '''OptionString''': String of clause elements. Specifies the value for parameter OPTIONS in tab 'Import' or function module 'rfc_read_table' in SAP. Maximum length for one string is 72 characters(SAP limit). Over 72 character length OptionString is splited by ' AND ' and ' OR ' to several OPTIONS parameter to avoid SAP 72 characters limit. Optional. Example: "VBELN BETWEEN '0000017448' AND '0000017450'".
* '''FieldNames''': Comma separated list of field names for columns to be extracted. Default value is empty, which will extract all columns. Specifies the value for parameter FIELDNAME in tab 'Tables' for table 'FIELDS' for function module 'rfc_read_table' in SAP. Optional.
* '''Function''': SAP function name that is called in SAP. Optional. The default value is RFC_READ_TABLE. Another possible function name is BBP_RFC_READ_TABLE.
* '''Language''': SAP language used. Default value is "EN". Optional. Corresponds to the "LANG" constant in SAP.
* '''Rowcount''': Maximum amount of rows to fetch. Specifies the value for parameter ROWCOUNT in tab 'Import' or function module 'rfc_read_table' in SAP. Optional.
* '''Rowskips''': Number of rows to skip. Specifies the value for parameter ROWSKIPS in tab 'Import' or function module 'rfc_read_table'. in SAP. Optional.
* '''PoolSize''': Maximum number of RFC connections that this destination will keep in its pool. Default value is "5". Optional. Corresponds to the "POOL_SIZE" constant in SAP.
* '''IdleTimeout''': If a connection has been idle for more than the defined time in seconds, it will be closed and removed from the connection pool. Default value is "600". Optional. Corresponds to the "IDLE_TIMEOUT" constant in SAP.
* '''ImportChunkSize''': Specifies the size of the chunks of data used when importing the data from SAP to PA Server. The value represents the approximate maximum number of data cells in each chunk (consisting tables of <number of rows> * <number of columns> data cells. Default value is 200000. Smaller value causes big imports to be split into more chunks taking more time in total, but it makes importing of each chunk faster possibly helping in some timeout situations.
* '''Router''': List of host names and service names or port numbers for the SAPRouter in the following format: /H/hostname/S/portnumber. Optional. Corresponds to the "SAPROUTER" constant in SAP.
* '''LogonGroup''': The logon group from which the message server shall select an application server. Optional. Corresponds to the "GROUP" constant in SAP.
* '''Mode''': If this number is set to "1", then the query result will have the SAP Table field names as data table column names and actual data rows as rows. If this is set to "3", the query result will get the field descriptions from the SAP query using NO_DATA parameter, i.e. the returned columns are the following (in this order): Field, Type, Description, Length, Offset. Default value is "1". Optional.
* '''QueryTable''': Name of the SAP table to be extracted. Specifies the value for the parameter QUERY_TABLE in tab 'Import' or function module 'rfc_read_table' in SAP. Mandatory. Note that if the query doesn't return any data, the target data table or temporary table is not created.
* '''[[Importing_Data_from_SAP|ConvertDataTypes]]''': List of SAP data types that are converted into respective expression language data types. Defined by listing the data type identifier characters in any order. Available data type identifying characters are IFPCDTNX. If not defined, all data types are converted. Example: IFP (convert only numeric data types: Integer, Float, Packed number).
* '''Ping''': If this is set to true, SAP server is pinged before SAP query execution. Purpose is help troubleshooting SAP connection issues. Optional.
* '''TraceLevel''': Sets SAP trace level, which can be 0, 1, 2, 3 or 4. Setting higher trace level helps to troubleshoot SAP connection issues. ''sapnco.dll'' writes log file to the current working folder with for example name'nco_rfc_5484_2.trc'. Optional.
* '''AliasUser'''
* '''AppServerService'''
* '''CharacterFaultIndicatorToken'''
* '''Codepage'''
* '''GatewayHost'''
* '''GatewayService'''
* '''IdleCheckTime'''
* '''LogonCheck'''
* '''MaxPoolWaitTime'''
* '''MessageServerService'''
* '''Name'''
* '''NoCompression'''
* '''OnCharacterConversionError'''
* '''PartnerCharSize'''
* '''PasswordChangeEnforced'''
* '''ProgramId'''
* '''R3Name'''
* '''RegistrationCount'''
* '''RepositoryDestination'''
* '''RepositoryPassword'''
* '''RepositorySncMyName'''
* '''RepositoryUser'''
* '''RepositoryX509Certificate'''
* '''SapSso2Ticket'''
* '''SncLibraryPath''': Full path including file name of the SNC shared library to be used.
* '''SncMode''': Determines whether connections will be secured with SNC. Value 0 doesn't use SNC (default) and value 1 uses SNC.
* '''SncMyName''': Token/identifier representing the external RFC program. In most cases this can be omitted. The installed SNC solution usually knows its own SNC name. Only for solutions supporting “multiple identities”, you may Varies depending on the installed SNC solution (Secude, Kerberos, NTLM, etc). Example for Secude: p/secude:CN=ALEREMOT SAP Online Help 09.09.2014 SAP .NET Connector 3.0 41 need to specify the identity to be used for this particular destination/server. E, O=Mustermann-AG, C=DE
* '''SncPartnerName''': The backend's SNCname.
* '''SncPartnerNames'''
* '''SncQop''': Quality of service to be used for SNC communication of this particular destination/server. One of the following values:
** 1: Digital signature
** 2: Digital signature and encryption
** 3: Digital signature, encryption, and user authentication
** 8: Default value defined by back-end system
** 9: Maximum value that the current security product supports
* '''SystemIds'''
* '''UseSapGui'''
* '''X509Certificate'''

Examples:

An example of performing a SAP extraction and persisting the extracted data to a table having id 1. Note: Data table must exist already when using this approach.
<syntaxhighlight lang="typescript">
let sapConnection = #{
"AppServerHost": "sap01",
"SystemNumber": "00",
"User": "user1",
"PasswordKey": "SapPW1",
"Router": "",
"SystemID": "QPR",
"Client": "800",
"Language": "EN",
"PoolSize": 5,
"PoolSizeMax": 10,
"IdleTimeout": 600
};
let queryParameters = sapConnection.Extend(#{
"FieldNames": "VBELN,ERDAT,ERZET,ERNAM,NETWR,WAERK",
"QueryTable": "VBAK",
"Options": ["VBELN BETWEEN '0000017448'" ,"AND '0000017450'"]
});
let resultsFlow = ProjectById(1)
.ExtractSap(queryParameters);
DatatableById(1)
.Import(resultsFlow);
</syntaxhighlight>

An example of performing a SAP extraction and persisting the extracted data to a table named "SAPData" in project having id 1. If the data table does not yet exist, a new one is created into Snowflake.<syntaxhighlight lang="typescript">
let sapConnection = #{
"AppServerHost": "sap01",
"SystemNumber": "10",
"User": "exampleuser",
"Password": "examplepassword",
"Router": "/H/127.0.0.1/A/1234/H/",
"SystemID": "QPR",
"Client": "200",
"Language": "EN",
"PoolSize": 5,
"PoolSizeMax": 10,
"IdleTimeout": 600,
"AppServerHost": "127.0.0.1",
"LogonGroup": "GROUPXNAME",
"Mode": "1"
};
let queryParameters = sapConnection.Extend(#{
"FieldNames": "VBELN,ERDAT,ERZET,ERNAM,NETWR,WAERK",
"QueryTable": "VBAK",
"Options": ["VBELN EQ '0060000039'", "OR VBELN EQ '0060000040'"]
});
ProjectById(1)
.ExtractSap(queryParameters)
.Persist("SAPData", #{"ProjectId": 1, "Connection": CreateSnowflakeConnection(#{"ProjectId": 1})})
</syntaxhighlight>

An example of performing a SAP extract, a simple transformation for extracted data and import to a table named "TransformedSAPData" in project having id 1.

Note: The example uses ScriptLauncher as a gateway and works only with ScriptLauncher version 2023.1 or later with "UseLegacyClientSideImport"-setting in appsettings.json set to false (if the setting is available in the used version).<syntaxhighlight lang="typescript">
function ExtractTransformAndLoad(extractFunc, transformFunc, loadFunc)
{
let rawDataFlow = extractFunc();
let transformedDataFlow = ToDataFlow();

_system.Parallel.Run([
() => Catch({
let df;
while (!IsNullTop(df = rawDataFlow.Collect(#{"CollectChunk": true}))) {
transformedDataFlow.Append(transformFunc(df));
WriteLog(`A chunk having ${df.NRows} rows has been transformed.`);
}
if (rawDataFlow.HasError) {
transformedDataFlow.Fail("Error occurred during data extraction.");
}
else {
transformedDataFlow.Complete();
}
}, {
transformedDataFlow.Fail("Error occurred during transformation calculation.");
}),
() => {
loadFunc(transformedDataFlow);
}
]);
}

let sapConnection = #{
"AppServerHost": "sap01",
"SystemNumber": "10",
"User": "exampleuser",
"Password": "examplepassword",
"Router": "/H/127.0.0.1/A/1234/H/",
"SystemID": "QPR",
"Client": "200",
"Language": "EN",
"PoolSize": 5,
"PoolSizeMax": 10,
"IdleTimeout": 600,
"AppServerHost": "127.0.0.1",
"LogonGroup": "GROUPXNAME",
"Mode": "1"
};

ExtractTransformAndLoad(
() => ExtractSap(
sapConnection.Extend([
"FieldNames": "VBELN,ERDAT,ERZET,ERNAM,NETWR,WAERK",
"QueryTable": "VBAK",
"Options": ["VBELN BETWEEN '0000017448'" ,"AND '0060000042'"],
"UseGateway": true
])
),
df => df.SetColumns(["Test": () => `${Column("NETWR")} ${Column("WAERK")}`]),
dataFlow => {
dataFlow.Persist("TransformedSAPData", ["ProjectName": "TestData", "Append": 0]);
}
);

</syntaxhighlight>

Object-centric Process Mining Model

2024-12-10T09:02:27Z

MarHink: /* Object-centric model structure */

QPR ProcessAnalyzer supports object-centric process mining (OCPM) based on the OCEL 2.0 standard (https://www.ocel-standard.org). To use object-centric functionality, you need to transform data into the [[#Object-centric_model_structure|suitable format]] for the [[#Create_object-centric_model|object-centric model]]. Object-centric models can be analyzed in the object-centric flowchart and with (case-centric) charts because the object-centric model can be converted into a case-centric eventlog using [[#Object-centric_perspectives|perspectives]]. To use the OCPM functionality, Snowflake needs to be used as the calculation engine.

== Create object-centric model ==
Create a new object-centric model as follows:
# In the Workspace, open the project where to create the model.
# Select '''NEW"''' in the top right menu and select '''model'''.
# Define a name for the new model.
# Set '''Model type''' as '''Object-centric'''.
# Click '''Create'''.

== Configure object-centric model datatables ==
Datatables for the object-centric model need to exist in the same project as the model. Datatables can be set for the model as follows:
# In the Workspace, select the object-centric model and click '''Properties'''.
# In the model properties dialog, open the '''Datasource''' tab.
# Add a following kind of json configuration to the textbox:
<pre>
{
"Objects": "OCPM: objects",
"Events": "OCPM: events",
"ObjectToObject": "OCPM: object-object",
"EventToObject": "OCPM: event-object",
"ObjectTypes": {
"Invoice": "OCPM object: Invoice",
"Payment": "OCPM object: Payment",
"Purchase Order": "OCPM object: Purchase Order"
},
"EventTypes": {
"Approve Purchase Requisition": "OCPM event: Approve Purchase Requisition",
"Change PO Quantity": "OCPM event: Change PO Quantity",
"Create Purchase Order": "OCPM event: Create Purchase Order",
"Insert Invoice": "OCPM event: Insert Invoice",
"Insert Payment": "OCPM event: Insert Payment"
}
}
</pre>

The json configuration needs to have following properties:
* '''Objects''': Objects datatable name.
* '''Events''': Events datatable name.
* '''ObjectToObject''': Object-to-object relation datatable name.
* '''EventToObject''': Event-to-object relation datatable name.
* '''ObjectTypes''': Key-value-pairs of object type datatable names. Note that object names need to match with object names in the objects datatable.
* '''EventTypes''': Key-value-pairs of event type datatable names. Note that event names need to match with event names in the events datatable.

== Import from OCEL 2.0 JSON file ==
Object-centric model can be import from an OCEL 2.0 JSON file as follows:
# In the Workspace, open the project where to import the model.
# Select '''NEW''' in top right menu and select '''Import Model'''.
# Select the OCEL 2.0 JSON file from the disk and click '''Open'''.

An object-centric model and a list of datatables is created.

Example OCEL 2.0 eventlogs: https://www.ocel-standard.org/event-logs/overview/ (download the json format supported by QPR ProcessAnalyzer)

== Object-centric model structure ==
Object-centric model contains datatables described in the table below. Datatables can be named freely, as the model json configuration is used to define the datatable for each type of data. The datatables need to use column names specified in the table below because those are the column names assumed by the object-centric (i.e., column names cannot be selected freely).

{| class="wikitable"
!'''Datatable role'''
!'''Contained data'''
! '''Datatable columns'''
|-
||Objects
||Objects in the model (one row per object).
||
* '''OcelObjectId''': Unique id for the object (among all objects in the model).
* '''OcelObjectType''': Object type name (such as Order, Invoice, Delivery). Note that the model json configuration need to use same object type names.
|-
||Events
||Events in the model (one row per event).
||
* '''OcelEventId''': Unique id for the event (among all events in the model).
* '''OcelEventType''': Event type name (such as Order created, Invoice sent). Note that the model json configuration need to use same event type names.
* '''OcelEventTime''': Event timestamp.
|-
||Object-object relations
||Relations between objects (one row per relation).
||
* '''OcelObjectToObjectSourceId''': Source object id in the relation.
* '''OcelObjectToObjectTargetId''': Target object id in the relation.
* '''OcelObjectToObjectQualifier''': Describes the type of the relation as free-form text (not used currently).
|-
||Event-object relations
||Relations between events and objects (one row per relation).
||
* '''OcelEventToObjectSourceId''': Object id in the relation.
* '''OcelEventToObjectTargetId''': Event id in the relation.
* '''OcelEventToObjectQualifier''': Describes the type of the relation as free-form text (not used currently).
|-
||Object attributes (several datatables)
||Object attribute values, each object type in a separate table (one row per object).
||
* '''OcelObjectTypeObjectId''': Object id. Matches to the objects datatable ''OcelObjectId'' column.
* '''OcelObjectTypeTime''': Timestamp which the attribute value is valid from (not used currently).
* '''OcelObjectTypeChangedField''': Changed object attribute name (not used currently).
* '''<Object attributes>''': Columns for each of the object attribute values (column name is the object attribute name).
|-
||Event attributes (several datatables)
||Event attribute values, each event type in a separate table (one row per event).
||
* '''OcelEventTypeEventId''': Event id. Matches to the events datatable ''OcelEventId'' column.
* '''<Event attributes>''': Columns for each of the event attribute values (column name is the event attribute name).
|}

== Object-centric perspectives ==
Perspectives convert an object-centric model into the traditional case-centric eventlog, allowing to view and analyze object-centric models in analyses provided by charts. A single perspective is not able describe the object-centric model entirely, but just from a certain limited viewpoint. By using analyses with several perspectives, it's possible to get a more complete picture of the object-centric model. The perspective starts from a certain object type and traverses the object-object relations as many steps as desired.

To define a perspective, the following settings are defined in the chart settings:
* '''Base Object type''': Object of this type will be cases in the projected case-centric eventlog.
* '''Object Relation Steps''': Specifies how many object-object relations will be traversed in order to find events connected to the base objects. Value zero means that only those events are returned that are directly connected to the base objects.
* '''Show Event Types''': List of event type names which are included into the perspective eventlog. If no events are explicitly defined, all events will be included, but their event attributes are not included.

The resulting perspective eventlog will have the following columns:
* '''OcelObjectId''' (mapped to case id)
* '''OcelEventType''' (mapped to event type)
* '''OcelEventTime''' (mapped to timestamp)
* '''OcelEventId'''
* Object attributes of the base object type. Note that the object attribute values are "repeated" for all events belonging to the same object.
* Event attributes of the selected event types. Values are null for events that don't have the attribute.

The base object type attributes are available as case attributes. As the object attribute values may change over time in the OCEL 2.0 data, the last attribute value is used as the case attribute value. Note that other object type's attributes are not available as case attributes, so the object for which the attributes are used, need to be set as the base object.

== Save perspective to filter ==
It's possible to include the object-centric perspective to a stored filter. When a filter is selected, also the perspective in the filter is applied to the dashboard. This allows to quickly change perspectives for the entire dashboard. The chart specific perspective overrides the dashboard level perspective, so the dashboard level perspective is only applied for charts that don't have the chart specific perspective defined.

Perspective can be added to a filter as follows:
# Go to the ''Process Discovery'' dashboard.
# Open the ''Session variables'' dialog in the dots menu on top right.
# Paste the filter json to the ''Value'' of the ''Filter'' variable (it might be easiest to start with a filter without filter rules, and then add the filter rules using the UI).
# Click ''Done'' button for the dialog.
# Save the filter by hovering the ''Unsaved filter'' (filters dropdown list) in the header and click ''Save as new filter''.

Example: Filter json without any filter rules:
<pre>
{
"Items": [],
"Perspective": {
"ObjectType": "Container",
"RecursionDepth": 0
}
}
</pre>

Example: Filter json with a filter rule:
<pre>
{
"Items": [
{
"Type": "IncludeEvents",
"Items": [
{
"Type": "Attribute",
"Attribute": "OcelEventId",
"StringifiedValues": [ "0Event 1" ]
}
]
}
],
"Perspective": {
"ObjectType": "Container",
"RecursionDepth": 0
}
}
</pre>

== Differences to OCEL 2.0 standard ==
Object-centric models in QPR ProcessAnalyzer are mainly following the OCEL 2.0 standard, but there are the following differences:
* Changing of object attributes values over time is not supported.
* ''ocel_time'' field of each event type table is moved to events datatable (as every event has a timestemp).
* ''*_map_type'' columns are not needed as the model settings are used for the same purpose.
* Object type tables: If OcelObjectTypeChangedField is not null, all the other field values are copied from the previous entry except:
** ''OcelObjectTypeChangedField'' which has the names of the changed fields as a comma separated string.
** The actual changed field which has the new value.
** ''OcelObjectTypeTime'' which has the timestamp when the value changed.

System Library

2024-11-20T10:24:06Z

MarHink: /* Example */

System library is a collection of Expression Language functions and properties that provide additional tools for scripting Process Analyzer functionalities. System library is referenced in scripts via ''_system''-property, which provides additional properties dedicated for different areas of interests for scripting.

==ML.GeneratePredictionModel==
Documentation [[Create_Predicted_Eventlog#Create_prediction_script_in_QPR_ProcessAnalyzer|here]].

==ML.ApplyTransformations==
Documentation [[Create_Simulated_Eventlog#Create_simulation_script_in_QPR_ProcessAnalyzer|here]].

==Parallel.Run==
Runs given functions in parallel.

===Parameters===
*'''functions''':
**An array of functions to run in parallel.

===Return value ===
An array of results returned by the called functions, in the same order as the function generating them in the ''functions''-parameter.

===Example===
The following script uses _system.Parallel.Run to run three functions:

* SAP-extraction from VBAK-table in SAP (connection parameters defined in connectionParametersDict-dictionary).
*Transform the extracted data by adding a new column.
*Load the data into data table identified by dataTableId.
<syntaxhighlight lang="typescript">
function ExtractTransformAndLoad(extractFunc, transformFunc, loadFunc)
{
let rawDataFlow = extractFunc();
let transformedDataFlow = ToDataFlow();

_system.Parallel.Run([
() => Catch({
let df;
while (!IsNullTop(df = rawDataFlow.Collect(#{"CollectChunk": true}))) {
transformedDataFlow.Append(transformFunc(df));
WriteLog(`A chunk having ${df.NRows} rows has been transformed.`);
}
if (rawDataFlow.HasError) {
transformedDataFlow.Fail("Error occurred during data extraction.");
}
else {
transformedDataFlow.Complete();
}
}, {
transformedDataFlow.Fail("Error occurred during transformation calculation.");
}),
() => {
loadFunc(transformedDataFlow);
}
]);
}

ExtractTransformAndLoad(
() => ExtractSap(connectionParametersDict.Extend(
[
"FieldNames": "VBELN,ERDAT,ERZET,ERNAM,NETWR,WAERK",
"QueryTable": "VBAK",
"Options": ["VBELN BETWEEN '0000017448'" ,"AND '0060000042'"],
"UseGateway": true
])
),
df => df.SetColumns(["Test": () => `${Column("NETWR")} ${Column("WAERK")}`]),
dataFlow => {
DataTableById(dataTableId).Import(dataFlow, ["Append": 0]);
}
);

DataTableById(dataTableId).SqlDataFrame.OrderByColumns(["VBELN"], [true]).Collect()
</syntaxhighlight>

==RootCauses.FindForDataFrame==
Finds root causes for a particular process phenomenon by comparing properties of selected cases against those of all cases in given model.

Based on the similar in-memory function: [[FindRootCauses Function|EventLog.FindRootCauses]].

===Parameters===

*'''model''':
**Model object of a model whose data tables are used to calculate the root causes.
*'''parameters''':
**A parameter convertible to a StringDictionary object with the following supported key-values:
*** '''Filter''':
****Filter json (#30921#) that is applied to the event log before calculating root causes.
*** '''Selection''':
**** Selection json (#30927#) that defines selected cases to find root causes for. Is applied on top of the filtered event log (specified by Filter parameter).
****If nothing is selected, 100% of cases are counted as selected.
*** '''CaseAttributeTypes''':
****An array of strings with the names of case attributes included into the root causes.
****Only case attributes of type string, integer or boolean are included.
****If CaseAttributeTypes is null or empty string or not specified, all case attributes having type "String" are included into the root causes.
****If the model doesn't have a case attribute with the specified name, an error message is shown.
*** '''EventAttributeTypes''':
****An array of strings with the names of event attributes included into the root causes.
****Only event attributes of type string are included.
****If EventAttributeTypes is null, all event attributes having type "String" are included into the root causes.
****If EventAttributeTypes is empty array or not specified, event attributes not applied to the root causes.
****If the model doesn't have an event attribute with the specified name, an error message is shown.
****Analysis column Name should contain Event Attribute Name.
****Analysis column Value should contain Event Attribute Value and number of occurrences in case: <value> (count).
****Analysis column Type should have "EventAttributeValue" as its value.
***'''WeightingExpression''':
**** Expression providing weights for each case.
****Expression, if defined, must be any of the following types:
***** A string containing the [[SQL Expressions|SqlExpression]] to evaluate.
*****A SqlExpression object, created e.g., using ToSqlExpression-function (see also: [[QPR_ProcessAnalyzer_Expressions#In-memory_expression_blocks_in_SQL_expressions|In-memory expression blocks in SQL expressions]]).
****A row is filtered out of result if expression result is null.
***'''MaximumRowCount''':
****The maximum number of the most and least contributing root causes to return. Thus, the actual number of returned rows can be at most two times this value (if specified).
****If undefined, 200 is used.
**** If set to 0, all rows are returned.
***'''MinValueUsage''':
****The minimum total usage of a value included into the comparison. The number of cases having every returned value should be at least given percentage (a float value between 0.0 and 1.0) of all the compared cases.
*****If not defined or null, all values are included (=default).
***'''MaxNumUniqueValues''':
****Maximum number of unique values to include into the comparison for each attribute column. If the amount of unique values for any attribute exceeds this value, only given number of attributes are included that have the highest usage.
****If not defined or null, all values are included (=default).
***'''IncludeOthers''':
****Should the rest of the attribute values not included due to MinValueUsage or MaxNumUniqueValues filtering be included as an aggregated "Others" value?
****If given any string value, that value is used as the value for all the aggregated values.
****Default = undefined => others values will not be included into the results.
***'''ValueIfNull''': Value used to indicate null-values.
****Default = "(blank)".
****Must be not null.

===Return value===
Returns a [[SqlDataFrame in Expression Language|SqlDataFrame]] object with the following columns:

*Common columns:
**'''Type''':
***Type of the root cause:
****"CaseAttributeValue" for case attributes.
****"EventAttributeValue" for event attributes.
**'''Name''':
***When type is CaseAttributeValue, case attribute name.
***When type is EventAttributeValue, event attribute name.
**'''Value''':
***When type is CaseAttributeValue, case attribute value.
***When type is EventAttributeValue, event attribute value and number of occurrences in case.
** '''Total''':
***The total number of cases having the found root cause.
** '''Selected''':
***The number of cases that have the found root cause and belong to the selected cases.
**'''Compared''':
*** The number of cases that have the found root cause and don't belong to the selected cases.
**Columns when WeightingExpression not have value:
***'''Contribution''':
****The number of cases which contribute to the deviation from the average percentage of selected cases among all analyzed cases.
***'''ContributionPercentage''':
****The percent of cases which contribute to the deviation from the average percentage.
*** '''DifferencePercentage''':
****The deviation in percentage between selected cases with the found root cause and the average percentage of selected cases among all analyzed cases.
***'''SelectedPercentage''':
**** The percent of selected cases that have the found root cause out of all cases with that root cause.
**Columns when WeightingExpression have value:
***'''Contribution''':
****The sum of case weights which contribute to the deviation from the average percentage of selected case weights among all analyzed cases.
***'''ContributionPercentage''':
****The percent of case weights which contribute to the deviation from the average percentage.
*** '''DifferencePercentage''':
****The deviation in percentage between selected case weights with the found root cause and the average percentage of selected case weights among all analyzed cases.
***'''SelectedPercentage''':
****The percent of selected case weights that have the found root cause out of all case weights with that root cause.
*** '''SelectedWeight''':
**** The sum of weights that have the found root cause and belong to the selected cases.
*** '''ComparedWeight''':
****The sum of weights that have the found root cause and don't belong to the selected cases.
*** '''TotalWeight''':
****The sum of weights of all cases with that root cause.

===Example===
Calculate root cause analysis for given model using parameters read from the [[Web API: Expression/query|query configuration]].<syntaxhighlight lang="json">
{
"ProcessingMethod": "DataFrame",
"Root":"let m = _; _system.RootCauses.FindForDataFrame(m, _query.Configuration.Parameters.FindRootCausesParameters.Clone().Extend(#{\"Filter\": _query.Configuration.Filter}))",
"Parameters": {
"FindRootCausesParameters": {
"CaseAttributeTypes": ["Account Manager","Customer Group","Product Group","Region"],
"Selection": {"Items":[{"Type":"IncludeCases","Items":[{"Type":"CaseAttributeValue","Attribute":"Product Group","StringifiedValues":["0Hats"]}]}]},
"MaxNumUniqueValues": 2,
"MaximumRowCount": 1000,
"MinValueUsage": 0.20,
"WeightingExpression": "Column(\"Cost\")"
}
},
"Ordering": [
{"Name": "Contribution", "Direction": "Descending"}
]
}

</syntaxhighlight>

==Utils.GetSampledEvents==
Returns a [[SqlDataFrame in Expression Language|SqlDataFrame]] containing sampled events of given model where given filter is first applied.

===Parameters ===

*'''sourceModel''':
**ProcessAnalyzer model object of the model whose event data is to be filtered and sampled.
*'''sampledCaseCount''':
**The maximum number of cases to return (or null if all cases should be returned).
* '''filter''':
**JSON filter to be applied on the event data of the source model prior to performing the sampling.

===Return value===
[[SqlDataFrame in Expression Language|SqlDataFrame]] containing sampled events of given model where given filter is first applied.

===Example===
Get a sample of all the events of at most 1000 cases having "Hats" as "Product Group" case attribute value from model identified by modelId.<syntaxhighlight lang="typescript">
let eventDataSampleSdf = _system.Utils.GetSampledEvents(modelId, 1000, #{
"Items":[#{
"Type":"IncludeCases",
"Items":[#{
"Type":"CaseAttributeValue",
"Attribute":"Product Group",
"StringifiedValues":["0Hats"]
}]
}]
});
</syntaxhighlight>

==Utils.ModifyColumnTypes==
In-place modifies the column types of given columns in given data table.

===Parameters===
*'''dataTable''':
**ProcessAnalyzer model object of the model whose event data is to be filtered and sampled.
*'''columnTypesToSet''':
**Array of column name/type definitions to set.
***Only columns that are to be changed are required to be listed.
***Columns that don't exist in the data table will be skipped.

===Return value ===
Returns the modified data table object.

===Example===
Modify column "CaseId" to be of type string.<syntaxhighlight lang="typescript">
_system.Utils.ModifyColumnTypes(
eventsTable,
[#{"Name": "CaseId", "DataType": "String"}]
);

</syntaxhighlight>

==Utils.RunFunctionWithParallelLogging==
Runs given function that generates logging information into given data table in a way that all the logging will be included into the generated script run log as well (if run inside a script).

Internally polls the table every 5 seconds for new rows and adds all the newly added rows to script log.

===Parameters===

*'''logTable''':
**A DataTable used for logging.
*'''callbackFunc''':
** Function that uses given data table for logging it's current status.

===Return value===
The result of the callback function.

=== Example===
Run stored procedure named "StoredProcedureTest" in Snowflake, that generates new rows to log table identified by logTableId and log the generated rows into the script run log.<syntaxhighlight lang="typescript">
_system.Utils.RunFunctionWithParallelLogging(DataTableById(logTableId), () => {
CreateSnowflakeConnection().CallStoredProcedure("StoredProcedureTest", #{})
});

</syntaxhighlight>

System Library

2024-11-19T13:53:09Z

MarHink:

System library is a collection of Expression Language functions and properties that provide additional tools for scripting Process Analyzer functionalities. System library is referenced in scripts via ''_system''-property, which provides additional properties dedicated for different areas of interests for scripting.

==ML.GeneratePredictionModel==
Documentation [[Create_Predicted_Eventlog#Create_prediction_script_in_QPR_ProcessAnalyzer|here]].

==ML.ApplyTransformations==
Documentation [[Create_Simulated_Eventlog#Create_simulation_script_in_QPR_ProcessAnalyzer|here]].

==Parallel.Run==
Runs given functions in parallel.

===Parameters===
*'''functions''':
**An array of functions to run in parallel.

===Return value ===
An array of results returned by the called functions, in the same order as the function generating them in the ''functions''-parameter.

===Example===
The following script uses _system.Parallel.Run to run three functions:

* SAP-extraction from VBAK-table in SAP (connection parameters defined in connectionParametersDict-dictionary).
*Transform the extracted data by adding a new column.
*Load the data into data table identified by dataTableId.
<syntaxhighlight lang="typescript">
function ExtractTransformAndLoad(extractFunc, transformFunc, loadFunc)
{
let rawDataFlow = extractFunc();
let transformedDataFlow = ToDataFlow();

_system.Parallel.Run([
() => Catch({
let df;
while (!IsNullTop(df = rawDataFlow.Collect(#{"CollectChunk": true}))) {
transformedDataFlow.Append(transformFunc(df));
WriteLog(`A chunk having ${df.NRows} rows has been transformed.`);
}
if (rawDataFlow.HasError) {
transformedDataFlow.Fail("Error occurred during data extraction.");
}
else {
transformedDataFlow.Complete();
}
}, {
transformedDataFlow.Fail("Error occurred during transformation calculation.");
}),
() => {
loadFunc(transformedDataFlow);
}
]);
}

ExtractTransformAndLoad(
() => ExtractSap(connectionParametersDict.Extend(
[
"FieldNames": "VBELN,ERDAT,ERZET,ERNAM,NETWR,WAERK",
"QueryTable": "VBAK",
"Options": ["VBELN BETWEEN '0000017448'" ,"AND '0060000042'"],
"UseGateway": true
])
),
df => df.SetColumns(["Test": () => `${Column("NETWR")} ${Column("WAERK")}`]),
dataFlow => {
DataTableById(dataTableId).Import(dataFlow, ["Append": 0]);
}
);

DataTableById(dataTableId).SqlDataFrame.OrderByColumns(["VBELN"], [true]).Collect()
</syntaxhighlight>

==RootCauses.FindForDataFrame==
Finds root causes for a particular process phenomenon by comparing properties of selected cases against those of all cases in given model.

Based on the similar in-memory function: [[FindRootCauses Function|EventLog.FindRootCauses]].

===Parameters===

*'''model''':
**Model object of a model whose data tables are used to calculate the root causes.
*'''parameters''':
**A parameter convertible to a StringDictionary object with the following supported key-values:
*** '''Filter''':
****Filter json (#30921#) that is applied to the event log before calculating root causes.
*** '''Selection''':
**** Selection json (#30927#) that defines selected cases to find root causes for. Is applied on top of the filtered event log (specified by Filter parameter).
****If nothing is selected, 100% of cases are counted as selected.
*** '''CaseAttributeTypes''':
****An array of strings with the names of case attributes included into the root causes.
****Only case attributes of type string, integer or boolean are included.
****If CaseAttributeTypes is null or empty string or not specified, all case attributes having type "String" are included into the root causes.
****If the model doesn't have a case attribute with the specified name, an error message is shown.
*** '''EventAttributeTypes''':
****An array of strings with the names of event attributes included into the root causes.
****Only event attributes of type string are included.
****If EventAttributeTypes is null, all event attributes having type "String" are included into the root causes.
****If EventAttributeTypes is empty array or not specified, event attributes not applied to the root causes.
****If the model doesn't have an event attribute with the specified name, an error message is shown.
****Analysis column Name should contain Event Attribute Name.
****Analysis column Value should contain Event Attribute Value and number of occurrences in case: <value> (count).
****Analysis column Type should have "EventAttributeValue" as its value.
***'''WeightingExpression''':
**** Expression providing weights for each case.
****Expression, if defined, must be any of the following types:
***** A string containing the [[SQL Expressions|SqlExpression]] to evaluate.
*****A SqlExpression object, created e.g., using ToSqlExpression-function (see also: [[QPR_ProcessAnalyzer_Expressions#In-memory_expression_blocks_in_SQL_expressions|In-memory expression blocks in SQL expressions]]).
****A row is filtered out of result if expression result is null.
***'''MaximumRowCount''':
****The maximum number of the most and least contributing root causes to return. Thus, the actual number of returned rows can be at most two times this value (if specified).
****If undefined, 200 is used.
**** If set to 0, all rows are returned.
***'''MinValueUsage''':
****The minimum total usage of a value included into the comparison. The number of cases having every returned value should be at least given percentage (a float value between 0.0 and 1.0) of all the compared cases.
*****If not defined or null, all values are included (=default).
***'''MaxNumUniqueValues''':
****Maximum number of unique values to include into the comparison for each attribute column. If the amount of unique values for any attribute exceeds this value, only given number of attributes are included that have the highest usage.
****If not defined or null, all values are included (=default).
***'''IncludeOthers''':
****Should the rest of the attribute values not included due to MinValueUsage or MaxNumUniqueValues filtering be included as an aggregated "Others" value?
****If given any string value, that value is used as the value for all the aggregated values.
****Default = undefined => others values will not be included into the results.
***'''ValueIfNull''': Value used to indicate null-values.
****Default = "(blank)".
****Must be not null.

===Return value===
Returns a [[SqlDataFrame in Expression Language|SqlDataFrame]] object with the following columns:

*Common columns:
**'''Type''':
***Type of the root cause:
****"CaseAttributeValue" for case attributes.
****"EventAttributeValue" for event attributes.
**'''Name''':
***When type is CaseAttributeValue, case attribute name.
***When type is EventAttributeValue, event attribute name.
**'''Value''':
***When type is CaseAttributeValue, case attribute value.
***When type is EventAttributeValue, event attribute value and number of occurrences in case.
** '''Total''':
***The total number of cases having the found root cause.
** '''Selected''':
***The number of cases that have the found root cause and belong to the selected cases.
**'''Compared''':
*** The number of cases that have the found root cause and don't belong to the selected cases.
**Columns when WeightingExpression not have value:
***'''Contribution''':
****The number of cases which contribute to the deviation from the average percentage of selected cases among all analyzed cases.
***'''ContributionPercentage''':
****The percent of cases which contribute to the deviation from the average percentage.
*** '''DifferencePercentage''':
****The deviation in percentage between selected cases with the found root cause and the average percentage of selected cases among all analyzed cases.
***'''SelectedPercentage''':
**** The percent of selected cases that have the found root cause out of all cases with that root cause.
**Columns when WeightingExpression have value:
***'''Contribution''':
****The sum of case weights which contribute to the deviation from the average percentage of selected case weights among all analyzed cases.
***'''ContributionPercentage''':
****The percent of case weights which contribute to the deviation from the average percentage.
*** '''DifferencePercentage''':
****The deviation in percentage between selected case weights with the found root cause and the average percentage of selected case weights among all analyzed cases.
***'''SelectedPercentage''':
****The percent of selected case weights that have the found root cause out of all case weights with that root cause.
*** '''SelectedWeight''':
**** The sum of weights that have the found root cause and belong to the selected cases.
*** '''ComparedWeight''':
****The sum of weights that have the found root cause and don't belong to the selected cases.
*** '''TotalWeight''':
****The sum of weights of all cases with that root cause.

===Example===
Calculate root cause analysis for given model using parameters read from the [[Web API: Expression/query|query configuration]].<syntaxhighlight lang="json">
{
"ProcessingMethod": "DataFrame",
"Root":"let m = _; _system.RootCauses.FindForDataFrame(m, _query.Configuration.Parameters.FindRootCausesParameters.Clone().Extend(#{\"Filter\": _query.Configuration.Filter}))",
"Parameters": {
"FindRootCausesParameters": {
"CaseAttributeTypes": ["Account Manager","Customer Group","Product Group","Region"],
"Selection": {"Items":[{"Type":"IncludeCases","Items":[{"Type":"CaseAttributeValue","Attribute":"Product Group","StringifiedValues":["0Hats"]}]}]},
"MaxNumUniqueValues": 2,
"MaximumRowCount": 1000,
"MinValueUsage": 0.20,
"WeightingExpression": "Column(\"Cost\")"
}
},
"Ordering": [
{"Name": "Contribution", "Direction": "Descending"}
]
}

</syntaxhighlight>

==Utils.GetSampledEvents==
Returns a [[SqlDataFrame in Expression Language|SqlDataFrame]] containing sampled events of given model where given filter is first applied.

===Parameters ===

*'''sourceModel''':
**ProcessAnalyzer model object of the model whose event data is to be filtered and sampled.
*'''sampledCaseCount''':
**The maximum number of cases to return (or null if all cases should be returned).
* '''filter''':
**JSON filter to be applied on the event data of the source model prior to performing the sampling.

===Return value===
[[SqlDataFrame in Expression Language|SqlDataFrame]] containing sampled events of given model where given filter is first applied.

===Example===
Get a sample of all the events of at most 1000 cases having "Hats" as "Product Group" case attribute value from model identified by modelId.<syntaxhighlight lang="typescript">
let eventDataSampleSdf = _system.Utils.GetSampledEvents(modelId, 1000, #{
"Items":[#{
"Type":"IncludeCases",
"Items":[#{
"Type":"CaseAttributeValue",
"Attribute":"Product Group",
"StringifiedValues":["0Hats"]
}]
}]
});
</syntaxhighlight>

==Utils.ModifyColumnTypes==
In-place modifies the column types of given columns in given data table.

===Parameters===
*'''dataTable''':
**ProcessAnalyzer model object of the model whose event data is to be filtered and sampled.
*'''columnTypesToSet''':
**Array of column name/type definitions to set.
***Only columns that are to be changed are required to be listed.
***Columns that don't exist in the data table will be skipped.

===Return value ===
Returns the modified data table object.

===Example===
Modify column "CaseId" to be of type string.<syntaxhighlight lang="typescript">
_system.Utils.ModifyColumnTypes(
eventsTable,
#{#{"Name": "CaseId", "DataType": "String"}}
);

</syntaxhighlight>

==Utils.RunFunctionWithParallelLogging==
Runs given function that generates logging information into given data table in a way that all the logging will be included into the generated script run log as well (if run inside a script).

Internally polls the table every 5 seconds for new rows and adds all the newly added rows to script log.

===Parameters===

*'''logTable''':
**A DataTable used for logging.
*'''callbackFunc''':
** Function that uses given data table for logging it's current status.

===Return value===
The result of the callback function.

=== Example===
Run stored procedure named "StoredProcedureTest" in Snowflake, that generates new rows to log table identified by logTableId and log the generated rows into the script run log.<syntaxhighlight lang="typescript">
_system.Utils.RunFunctionWithParallelLogging(DataTableById(logTableId), () => {
CreateSnowflakeConnection().CallStoredProcedure("StoredProcedureTest", #{})
});

</syntaxhighlight>

System Library

2024-11-19T13:52:05Z

MarHink: /* ML.ApplyTransformations */

System library is a collection of Expression Language functions and properties that provide additional tools for scripting Process Analyzer functionalities. System library is referenced in scripts via ''_system''-property, which provides additional properties dedicated for different areas of interests for scripting.

The following hierarchy shows the properties and functions available in System Library:

* ML
** GeneratePredictionModel
**ApplyTransformations
*Parallel
**Run
*RootCauses
**FindRootCausesDataFrame
*Utils
**GetSampledEvents
**ModifyColumnTypes
**RunFunctionWithParallelLogging

==ML.GeneratePredictionModel==
Documentation [[Create_Predicted_Eventlog#Create_prediction_script_in_QPR_ProcessAnalyzer|here]].

==ML.ApplyTransformations==
Documentation [[Create_Simulated_Eventlog#Create_simulation_script_in_QPR_ProcessAnalyzer|here]].

==Parallel.Run==
Runs given functions in parallel.

===Parameters===
*'''functions''':
**An array of functions to run in parallel.

===Return value ===
An array of results returned by the called functions, in the same order as the function generating them in the ''functions''-parameter.

===Example===
The following script uses _system.Parallel.Run to run three functions:

* SAP-extraction from VBAK-table in SAP (connection parameters defined in connectionParametersDict-dictionary).
*Transform the extracted data by adding a new column.
*Load the data into data table identified by dataTableId.
<syntaxhighlight lang="typescript">
function ExtractTransformAndLoad(extractFunc, transformFunc, loadFunc)
{
let rawDataFlow = extractFunc();
let transformedDataFlow = ToDataFlow();

_system.Parallel.Run([
() => Catch({
let df;
while (!IsNullTop(df = rawDataFlow.Collect(#{"CollectChunk": true}))) {
transformedDataFlow.Append(transformFunc(df));
WriteLog(`A chunk having ${df.NRows} rows has been transformed.`);
}
if (rawDataFlow.HasError) {
transformedDataFlow.Fail("Error occurred during data extraction.");
}
else {
transformedDataFlow.Complete();
}
}, {
transformedDataFlow.Fail("Error occurred during transformation calculation.");
}),
() => {
loadFunc(transformedDataFlow);
}
]);
}

ExtractTransformAndLoad(
() => ExtractSap(connectionParametersDict.Extend(
[
"FieldNames": "VBELN,ERDAT,ERZET,ERNAM,NETWR,WAERK",
"QueryTable": "VBAK",
"Options": ["VBELN BETWEEN '0000017448'" ,"AND '0060000042'"],
"UseGateway": true
])
),
df => df.SetColumns(["Test": () => `${Column("NETWR")} ${Column("WAERK")}`]),
dataFlow => {
DataTableById(dataTableId).Import(dataFlow, ["Append": 0]);
}
);

DataTableById(dataTableId).SqlDataFrame.OrderByColumns(["VBELN"], [true]).Collect()
</syntaxhighlight>

==RootCauses.FindForDataFrame==
Finds root causes for a particular process phenomenon by comparing properties of selected cases against those of all cases in given model.

Based on the similar in-memory function: [[FindRootCauses Function|EventLog.FindRootCauses]].

===Parameters===

*'''model''':
**Model object of a model whose data tables are used to calculate the root causes.
*'''parameters''':
**A parameter convertible to a StringDictionary object with the following supported key-values:
*** '''Filter''':
****Filter json (#30921#) that is applied to the event log before calculating root causes.
*** '''Selection''':
**** Selection json (#30927#) that defines selected cases to find root causes for. Is applied on top of the filtered event log (specified by Filter parameter).
****If nothing is selected, 100% of cases are counted as selected.
*** '''CaseAttributeTypes''':
****An array of strings with the names of case attributes included into the root causes.
****Only case attributes of type string, integer or boolean are included.
****If CaseAttributeTypes is null or empty string or not specified, all case attributes having type "String" are included into the root causes.
****If the model doesn't have a case attribute with the specified name, an error message is shown.
*** '''EventAttributeTypes''':
****An array of strings with the names of event attributes included into the root causes.
****Only event attributes of type string are included.
****If EventAttributeTypes is null, all event attributes having type "String" are included into the root causes.
****If EventAttributeTypes is empty array or not specified, event attributes not applied to the root causes.
****If the model doesn't have an event attribute with the specified name, an error message is shown.
****Analysis column Name should contain Event Attribute Name.
****Analysis column Value should contain Event Attribute Value and number of occurrences in case: <value> (count).
****Analysis column Type should have "EventAttributeValue" as its value.
***'''WeightingExpression''':
**** Expression providing weights for each case.
****Expression, if defined, must be any of the following types:
***** A string containing the [[SQL Expressions|SqlExpression]] to evaluate.
*****A SqlExpression object, created e.g., using ToSqlExpression-function (see also: [[QPR_ProcessAnalyzer_Expressions#In-memory_expression_blocks_in_SQL_expressions|In-memory expression blocks in SQL expressions]]).
****A row is filtered out of result if expression result is null.
***'''MaximumRowCount''':
****The maximum number of the most and least contributing root causes to return. Thus, the actual number of returned rows can be at most two times this value (if specified).
****If undefined, 200 is used.
**** If set to 0, all rows are returned.
***'''MinValueUsage''':
****The minimum total usage of a value included into the comparison. The number of cases having every returned value should be at least given percentage (a float value between 0.0 and 1.0) of all the compared cases.
*****If not defined or null, all values are included (=default).
***'''MaxNumUniqueValues''':
****Maximum number of unique values to include into the comparison for each attribute column. If the amount of unique values for any attribute exceeds this value, only given number of attributes are included that have the highest usage.
****If not defined or null, all values are included (=default).
***'''IncludeOthers''':
****Should the rest of the attribute values not included due to MinValueUsage or MaxNumUniqueValues filtering be included as an aggregated "Others" value?
****If given any string value, that value is used as the value for all the aggregated values.
****Default = undefined => others values will not be included into the results.
***'''ValueIfNull''': Value used to indicate null-values.
****Default = "(blank)".
****Must be not null.

===Return value===
Returns a [[SqlDataFrame in Expression Language|SqlDataFrame]] object with the following columns:

*Common columns:
**'''Type''':
***Type of the root cause:
****"CaseAttributeValue" for case attributes.
****"EventAttributeValue" for event attributes.
**'''Name''':
***When type is CaseAttributeValue, case attribute name.
***When type is EventAttributeValue, event attribute name.
**'''Value''':
***When type is CaseAttributeValue, case attribute value.
***When type is EventAttributeValue, event attribute value and number of occurrences in case.
** '''Total''':
***The total number of cases having the found root cause.
** '''Selected''':
***The number of cases that have the found root cause and belong to the selected cases.
**'''Compared''':
*** The number of cases that have the found root cause and don't belong to the selected cases.
**Columns when WeightingExpression not have value:
***'''Contribution''':
****The number of cases which contribute to the deviation from the average percentage of selected cases among all analyzed cases.
***'''ContributionPercentage''':
****The percent of cases which contribute to the deviation from the average percentage.
*** '''DifferencePercentage''':
****The deviation in percentage between selected cases with the found root cause and the average percentage of selected cases among all analyzed cases.
***'''SelectedPercentage''':
**** The percent of selected cases that have the found root cause out of all cases with that root cause.
**Columns when WeightingExpression have value:
***'''Contribution''':
****The sum of case weights which contribute to the deviation from the average percentage of selected case weights among all analyzed cases.
***'''ContributionPercentage''':
****The percent of case weights which contribute to the deviation from the average percentage.
*** '''DifferencePercentage''':
****The deviation in percentage between selected case weights with the found root cause and the average percentage of selected case weights among all analyzed cases.
***'''SelectedPercentage''':
****The percent of selected case weights that have the found root cause out of all case weights with that root cause.
*** '''SelectedWeight''':
**** The sum of weights that have the found root cause and belong to the selected cases.
*** '''ComparedWeight''':
****The sum of weights that have the found root cause and don't belong to the selected cases.
*** '''TotalWeight''':
****The sum of weights of all cases with that root cause.

===Example===
Calculate root cause analysis for given model using parameters read from the [[Web API: Expression/query|query configuration]].<syntaxhighlight lang="json">
{
"ProcessingMethod": "DataFrame",
"Root":"let m = _; _system.RootCauses.FindForDataFrame(m, _query.Configuration.Parameters.FindRootCausesParameters.Clone().Extend(#{\"Filter\": _query.Configuration.Filter}))",
"Parameters": {
"FindRootCausesParameters": {
"CaseAttributeTypes": ["Account Manager","Customer Group","Product Group","Region"],
"Selection": {"Items":[{"Type":"IncludeCases","Items":[{"Type":"CaseAttributeValue","Attribute":"Product Group","StringifiedValues":["0Hats"]}]}]},
"MaxNumUniqueValues": 2,
"MaximumRowCount": 1000,
"MinValueUsage": 0.20,
"WeightingExpression": "Column(\"Cost\")"
}
},
"Ordering": [
{"Name": "Contribution", "Direction": "Descending"}
]
}

</syntaxhighlight>

==Utils.GetSampledEvents==
Returns a [[SqlDataFrame in Expression Language|SqlDataFrame]] containing sampled events of given model where given filter is first applied.

===Parameters ===

*'''sourceModel''':
**ProcessAnalyzer model object of the model whose event data is to be filtered and sampled.
*'''sampledCaseCount''':
**The maximum number of cases to return (or null if all cases should be returned).
* '''filter''':
**JSON filter to be applied on the event data of the source model prior to performing the sampling.

===Return value===
[[SqlDataFrame in Expression Language|SqlDataFrame]] containing sampled events of given model where given filter is first applied.

===Example===
Get a sample of all the events of at most 1000 cases having "Hats" as "Product Group" case attribute value from model identified by modelId.<syntaxhighlight lang="typescript">
let eventDataSampleSdf = _system.Utils.GetSampledEvents(modelId, 1000, #{
"Items":[#{
"Type":"IncludeCases",
"Items":[#{
"Type":"CaseAttributeValue",
"Attribute":"Product Group",
"StringifiedValues":["0Hats"]
}]
}]
});
</syntaxhighlight>

==Utils.ModifyColumnTypes==
In-place modifies the column types of given columns in given data table.

===Parameters===
*'''dataTable''':
**ProcessAnalyzer model object of the model whose event data is to be filtered and sampled.
*'''columnTypesToSet''':
**Array of column name/type definitions to set.
***Only columns that are to be changed are required to be listed.
***Columns that don't exist in the data table will be skipped.

===Return value ===
Returns the modified data table object.

===Example===
Modify column "CaseId" to be of type string.<syntaxhighlight lang="typescript">
_system.Utils.ModifyColumnTypes(
eventsTable,
#{#{"Name": "CaseId", "DataType": "String"}}
);

</syntaxhighlight>

==Utils.RunFunctionWithParallelLogging==
Runs given function that generates logging information into given data table in a way that all the logging will be included into the generated script run log as well (if run inside a script).

Internally polls the table every 5 seconds for new rows and adds all the newly added rows to script log.

===Parameters===

*'''logTable''':
**A DataTable used for logging.
*'''callbackFunc''':
** Function that uses given data table for logging it's current status.

===Return value===
The result of the callback function.

=== Example===
Run stored procedure named "StoredProcedureTest" in Snowflake, that generates new rows to log table identified by logTableId and log the generated rows into the script run log.<syntaxhighlight lang="typescript">
_system.Utils.RunFunctionWithParallelLogging(DataTableById(logTableId), () => {
CreateSnowflakeConnection().CallStoredProcedure("StoredProcedureTest", #{})
});

</syntaxhighlight>

System Library

2024-11-19T13:50:46Z

MarHink: /* ML.GeneratePredictionModel */

System library is a collection of Expression Language functions and properties that provide additional tools for scripting Process Analyzer functionalities. System library is referenced in scripts via ''_system''-property, which provides additional properties dedicated for different areas of interests for scripting.

The following hierarchy shows the properties and functions available in System Library:

* ML
** GeneratePredictionModel
**ApplyTransformations
*Parallel
**Run
*RootCauses
**FindRootCausesDataFrame
*Utils
**GetSampledEvents
**ModifyColumnTypes
**RunFunctionWithParallelLogging

==ML.GeneratePredictionModel==
Documentation [[Create_Predicted_Eventlog#Create_prediction_script_in_QPR_ProcessAnalyzer|here]].

==ML.ApplyTransformations==
Documentation here: [[Create Simulated Eventlog|ApplyTransformations]]

==Parallel.Run==
Runs given functions in parallel.

===Parameters===
*'''functions''':
**An array of functions to run in parallel.

===Return value ===
An array of results returned by the called functions, in the same order as the function generating them in the ''functions''-parameter.

===Example===
The following script uses _system.Parallel.Run to run three functions:

* SAP-extraction from VBAK-table in SAP (connection parameters defined in connectionParametersDict-dictionary).
*Transform the extracted data by adding a new column.
*Load the data into data table identified by dataTableId.
<syntaxhighlight lang="typescript">
function ExtractTransformAndLoad(extractFunc, transformFunc, loadFunc)
{
let rawDataFlow = extractFunc();
let transformedDataFlow = ToDataFlow();

_system.Parallel.Run([
() => Catch({
let df;
while (!IsNullTop(df = rawDataFlow.Collect(#{"CollectChunk": true}))) {
transformedDataFlow.Append(transformFunc(df));
WriteLog(`A chunk having ${df.NRows} rows has been transformed.`);
}
if (rawDataFlow.HasError) {
transformedDataFlow.Fail("Error occurred during data extraction.");
}
else {
transformedDataFlow.Complete();
}
}, {
transformedDataFlow.Fail("Error occurred during transformation calculation.");
}),
() => {
loadFunc(transformedDataFlow);
}
]);
}

ExtractTransformAndLoad(
() => ExtractSap(connectionParametersDict.Extend(
[
"FieldNames": "VBELN,ERDAT,ERZET,ERNAM,NETWR,WAERK",
"QueryTable": "VBAK",
"Options": ["VBELN BETWEEN '0000017448'" ,"AND '0060000042'"],
"UseGateway": true
])
),
df => df.SetColumns(["Test": () => `${Column("NETWR")} ${Column("WAERK")}`]),
dataFlow => {
DataTableById(dataTableId).Import(dataFlow, ["Append": 0]);
}
);

DataTableById(dataTableId).SqlDataFrame.OrderByColumns(["VBELN"], [true]).Collect()
</syntaxhighlight>

==RootCauses.FindForDataFrame==
Finds root causes for a particular process phenomenon by comparing properties of selected cases against those of all cases in given model.

Based on the similar in-memory function: [[FindRootCauses Function|EventLog.FindRootCauses]].

===Parameters===

*'''model''':
**Model object of a model whose data tables are used to calculate the root causes.
*'''parameters''':
**A parameter convertible to a StringDictionary object with the following supported key-values:
*** '''Filter''':
****Filter json (#30921#) that is applied to the event log before calculating root causes.
*** '''Selection''':
**** Selection json (#30927#) that defines selected cases to find root causes for. Is applied on top of the filtered event log (specified by Filter parameter).
****If nothing is selected, 100% of cases are counted as selected.
*** '''CaseAttributeTypes''':
****An array of strings with the names of case attributes included into the root causes.
****Only case attributes of type string, integer or boolean are included.
****If CaseAttributeTypes is null or empty string or not specified, all case attributes having type "String" are included into the root causes.
****If the model doesn't have a case attribute with the specified name, an error message is shown.
*** '''EventAttributeTypes''':
****An array of strings with the names of event attributes included into the root causes.
****Only event attributes of type string are included.
****If EventAttributeTypes is null, all event attributes having type "String" are included into the root causes.
****If EventAttributeTypes is empty array or not specified, event attributes not applied to the root causes.
****If the model doesn't have an event attribute with the specified name, an error message is shown.
****Analysis column Name should contain Event Attribute Name.
****Analysis column Value should contain Event Attribute Value and number of occurrences in case: <value> (count).
****Analysis column Type should have "EventAttributeValue" as its value.
***'''WeightingExpression''':
**** Expression providing weights for each case.
****Expression, if defined, must be any of the following types:
***** A string containing the [[SQL Expressions|SqlExpression]] to evaluate.
*****A SqlExpression object, created e.g., using ToSqlExpression-function (see also: [[QPR_ProcessAnalyzer_Expressions#In-memory_expression_blocks_in_SQL_expressions|In-memory expression blocks in SQL expressions]]).
****A row is filtered out of result if expression result is null.
***'''MaximumRowCount''':
****The maximum number of the most and least contributing root causes to return. Thus, the actual number of returned rows can be at most two times this value (if specified).
****If undefined, 200 is used.
**** If set to 0, all rows are returned.
***'''MinValueUsage''':
****The minimum total usage of a value included into the comparison. The number of cases having every returned value should be at least given percentage (a float value between 0.0 and 1.0) of all the compared cases.
*****If not defined or null, all values are included (=default).
***'''MaxNumUniqueValues''':
****Maximum number of unique values to include into the comparison for each attribute column. If the amount of unique values for any attribute exceeds this value, only given number of attributes are included that have the highest usage.
****If not defined or null, all values are included (=default).
***'''IncludeOthers''':
****Should the rest of the attribute values not included due to MinValueUsage or MaxNumUniqueValues filtering be included as an aggregated "Others" value?
****If given any string value, that value is used as the value for all the aggregated values.
****Default = undefined => others values will not be included into the results.
***'''ValueIfNull''': Value used to indicate null-values.
****Default = "(blank)".
****Must be not null.

===Return value===
Returns a [[SqlDataFrame in Expression Language|SqlDataFrame]] object with the following columns:

*Common columns:
**'''Type''':
***Type of the root cause:
****"CaseAttributeValue" for case attributes.
****"EventAttributeValue" for event attributes.
**'''Name''':
***When type is CaseAttributeValue, case attribute name.
***When type is EventAttributeValue, event attribute name.
**'''Value''':
***When type is CaseAttributeValue, case attribute value.
***When type is EventAttributeValue, event attribute value and number of occurrences in case.
** '''Total''':
***The total number of cases having the found root cause.
** '''Selected''':
***The number of cases that have the found root cause and belong to the selected cases.
**'''Compared''':
*** The number of cases that have the found root cause and don't belong to the selected cases.
**Columns when WeightingExpression not have value:
***'''Contribution''':
****The number of cases which contribute to the deviation from the average percentage of selected cases among all analyzed cases.
***'''ContributionPercentage''':
****The percent of cases which contribute to the deviation from the average percentage.
*** '''DifferencePercentage''':
****The deviation in percentage between selected cases with the found root cause and the average percentage of selected cases among all analyzed cases.
***'''SelectedPercentage''':
**** The percent of selected cases that have the found root cause out of all cases with that root cause.
**Columns when WeightingExpression have value:
***'''Contribution''':
****The sum of case weights which contribute to the deviation from the average percentage of selected case weights among all analyzed cases.
***'''ContributionPercentage''':
****The percent of case weights which contribute to the deviation from the average percentage.
*** '''DifferencePercentage''':
****The deviation in percentage between selected case weights with the found root cause and the average percentage of selected case weights among all analyzed cases.
***'''SelectedPercentage''':
****The percent of selected case weights that have the found root cause out of all case weights with that root cause.
*** '''SelectedWeight''':
**** The sum of weights that have the found root cause and belong to the selected cases.
*** '''ComparedWeight''':
****The sum of weights that have the found root cause and don't belong to the selected cases.
*** '''TotalWeight''':
****The sum of weights of all cases with that root cause.

===Example===
Calculate root cause analysis for given model using parameters read from the [[Web API: Expression/query|query configuration]].<syntaxhighlight lang="json">
{
"ProcessingMethod": "DataFrame",
"Root":"let m = _; _system.RootCauses.FindForDataFrame(m, _query.Configuration.Parameters.FindRootCausesParameters.Clone().Extend(#{\"Filter\": _query.Configuration.Filter}))",
"Parameters": {
"FindRootCausesParameters": {
"CaseAttributeTypes": ["Account Manager","Customer Group","Product Group","Region"],
"Selection": {"Items":[{"Type":"IncludeCases","Items":[{"Type":"CaseAttributeValue","Attribute":"Product Group","StringifiedValues":["0Hats"]}]}]},
"MaxNumUniqueValues": 2,
"MaximumRowCount": 1000,
"MinValueUsage": 0.20,
"WeightingExpression": "Column(\"Cost\")"
}
},
"Ordering": [
{"Name": "Contribution", "Direction": "Descending"}
]
}

</syntaxhighlight>

==Utils.GetSampledEvents==
Returns a [[SqlDataFrame in Expression Language|SqlDataFrame]] containing sampled events of given model where given filter is first applied.

===Parameters ===

*'''sourceModel''':
**ProcessAnalyzer model object of the model whose event data is to be filtered and sampled.
*'''sampledCaseCount''':
**The maximum number of cases to return (or null if all cases should be returned).
* '''filter''':
**JSON filter to be applied on the event data of the source model prior to performing the sampling.

===Return value===
[[SqlDataFrame in Expression Language|SqlDataFrame]] containing sampled events of given model where given filter is first applied.

===Example===
Get a sample of all the events of at most 1000 cases having "Hats" as "Product Group" case attribute value from model identified by modelId.<syntaxhighlight lang="typescript">
let eventDataSampleSdf = _system.Utils.GetSampledEvents(modelId, 1000, #{
"Items":[#{
"Type":"IncludeCases",
"Items":[#{
"Type":"CaseAttributeValue",
"Attribute":"Product Group",
"StringifiedValues":["0Hats"]
}]
}]
});
</syntaxhighlight>

==Utils.ModifyColumnTypes==
In-place modifies the column types of given columns in given data table.

===Parameters===
*'''dataTable''':
**ProcessAnalyzer model object of the model whose event data is to be filtered and sampled.
*'''columnTypesToSet''':
**Array of column name/type definitions to set.
***Only columns that are to be changed are required to be listed.
***Columns that don't exist in the data table will be skipped.

===Return value ===
Returns the modified data table object.

===Example===
Modify column "CaseId" to be of type string.<syntaxhighlight lang="typescript">
_system.Utils.ModifyColumnTypes(
eventsTable,
#{#{"Name": "CaseId", "DataType": "String"}}
);

</syntaxhighlight>

==Utils.RunFunctionWithParallelLogging==
Runs given function that generates logging information into given data table in a way that all the logging will be included into the generated script run log as well (if run inside a script).

Internally polls the table every 5 seconds for new rows and adds all the newly added rows to script log.

===Parameters===

*'''logTable''':
**A DataTable used for logging.
*'''callbackFunc''':
** Function that uses given data table for logging it's current status.

===Return value===
The result of the callback function.

=== Example===
Run stored procedure named "StoredProcedureTest" in Snowflake, that generates new rows to log table identified by logTableId and log the generated rows into the script run log.<syntaxhighlight lang="typescript">
_system.Utils.RunFunctionWithParallelLogging(DataTableById(logTableId), () => {
CreateSnowflakeConnection().CallStoredProcedure("StoredProcedureTest", #{})
});

</syntaxhighlight>

System Library

2024-11-19T13:49:54Z

MarHink: /* ML.GeneratePredictionModel */

System library is a collection of Expression Language functions and properties that provide additional tools for scripting Process Analyzer functionalities. System library is referenced in scripts via ''_system''-property, which provides additional properties dedicated for different areas of interests for scripting.

The following hierarchy shows the properties and functions available in System Library:

* ML
** GeneratePredictionModel
**ApplyTransformations
*Parallel
**Run
*RootCauses
**FindRootCausesDataFrame
*Utils
**GetSampledEvents
**ModifyColumnTypes
**RunFunctionWithParallelLogging

==ML.GeneratePredictionModel==
Documentation here: [[Create Predicted Eventlog|Create_Predicted_Eventlog#Create_prediction_script_in_QPR_ProcessAnalyzer]]

==ML.ApplyTransformations==
Documentation here: [[Create Simulated Eventlog|ApplyTransformations]]

==Parallel.Run==
Runs given functions in parallel.

===Parameters===
*'''functions''':
**An array of functions to run in parallel.

===Return value ===
An array of results returned by the called functions, in the same order as the function generating them in the ''functions''-parameter.

===Example===
The following script uses _system.Parallel.Run to run three functions:

* SAP-extraction from VBAK-table in SAP (connection parameters defined in connectionParametersDict-dictionary).
*Transform the extracted data by adding a new column.
*Load the data into data table identified by dataTableId.
<syntaxhighlight lang="typescript">
function ExtractTransformAndLoad(extractFunc, transformFunc, loadFunc)
{
let rawDataFlow = extractFunc();
let transformedDataFlow = ToDataFlow();

_system.Parallel.Run([
() => Catch({
let df;
while (!IsNullTop(df = rawDataFlow.Collect(#{"CollectChunk": true}))) {
transformedDataFlow.Append(transformFunc(df));
WriteLog(`A chunk having ${df.NRows} rows has been transformed.`);
}
if (rawDataFlow.HasError) {
transformedDataFlow.Fail("Error occurred during data extraction.");
}
else {
transformedDataFlow.Complete();
}
}, {
transformedDataFlow.Fail("Error occurred during transformation calculation.");
}),
() => {
loadFunc(transformedDataFlow);
}
]);
}

ExtractTransformAndLoad(
() => ExtractSap(connectionParametersDict.Extend(
[
"FieldNames": "VBELN,ERDAT,ERZET,ERNAM,NETWR,WAERK",
"QueryTable": "VBAK",
"Options": ["VBELN BETWEEN '0000017448'" ,"AND '0060000042'"],
"UseGateway": true
])
),
df => df.SetColumns(["Test": () => `${Column("NETWR")} ${Column("WAERK")}`]),
dataFlow => {
DataTableById(dataTableId).Import(dataFlow, ["Append": 0]);
}
);

DataTableById(dataTableId).SqlDataFrame.OrderByColumns(["VBELN"], [true]).Collect()
</syntaxhighlight>

==RootCauses.FindForDataFrame==
Finds root causes for a particular process phenomenon by comparing properties of selected cases against those of all cases in given model.

Based on the similar in-memory function: [[FindRootCauses Function|EventLog.FindRootCauses]].

===Parameters===

*'''model''':
**Model object of a model whose data tables are used to calculate the root causes.
*'''parameters''':
**A parameter convertible to a StringDictionary object with the following supported key-values:
*** '''Filter''':
****Filter json (#30921#) that is applied to the event log before calculating root causes.
*** '''Selection''':
**** Selection json (#30927#) that defines selected cases to find root causes for. Is applied on top of the filtered event log (specified by Filter parameter).
****If nothing is selected, 100% of cases are counted as selected.
*** '''CaseAttributeTypes''':
****An array of strings with the names of case attributes included into the root causes.
****Only case attributes of type string, integer or boolean are included.
****If CaseAttributeTypes is null or empty string or not specified, all case attributes having type "String" are included into the root causes.
****If the model doesn't have a case attribute with the specified name, an error message is shown.
*** '''EventAttributeTypes''':
****An array of strings with the names of event attributes included into the root causes.
****Only event attributes of type string are included.
****If EventAttributeTypes is null, all event attributes having type "String" are included into the root causes.
****If EventAttributeTypes is empty array or not specified, event attributes not applied to the root causes.
****If the model doesn't have an event attribute with the specified name, an error message is shown.
****Analysis column Name should contain Event Attribute Name.
****Analysis column Value should contain Event Attribute Value and number of occurrences in case: <value> (count).
****Analysis column Type should have "EventAttributeValue" as its value.
***'''WeightingExpression''':
**** Expression providing weights for each case.
****Expression, if defined, must be any of the following types:
***** A string containing the [[SQL Expressions|SqlExpression]] to evaluate.
*****A SqlExpression object, created e.g., using ToSqlExpression-function (see also: [[QPR_ProcessAnalyzer_Expressions#In-memory_expression_blocks_in_SQL_expressions|In-memory expression blocks in SQL expressions]]).
****A row is filtered out of result if expression result is null.
***'''MaximumRowCount''':
****The maximum number of the most and least contributing root causes to return. Thus, the actual number of returned rows can be at most two times this value (if specified).
****If undefined, 200 is used.
**** If set to 0, all rows are returned.
***'''MinValueUsage''':
****The minimum total usage of a value included into the comparison. The number of cases having every returned value should be at least given percentage (a float value between 0.0 and 1.0) of all the compared cases.
*****If not defined or null, all values are included (=default).
***'''MaxNumUniqueValues''':
****Maximum number of unique values to include into the comparison for each attribute column. If the amount of unique values for any attribute exceeds this value, only given number of attributes are included that have the highest usage.
****If not defined or null, all values are included (=default).
***'''IncludeOthers''':
****Should the rest of the attribute values not included due to MinValueUsage or MaxNumUniqueValues filtering be included as an aggregated "Others" value?
****If given any string value, that value is used as the value for all the aggregated values.
****Default = undefined => others values will not be included into the results.
***'''ValueIfNull''': Value used to indicate null-values.
****Default = "(blank)".
****Must be not null.

===Return value===
Returns a [[SqlDataFrame in Expression Language|SqlDataFrame]] object with the following columns:

*Common columns:
**'''Type''':
***Type of the root cause:
****"CaseAttributeValue" for case attributes.
****"EventAttributeValue" for event attributes.
**'''Name''':
***When type is CaseAttributeValue, case attribute name.
***When type is EventAttributeValue, event attribute name.
**'''Value''':
***When type is CaseAttributeValue, case attribute value.
***When type is EventAttributeValue, event attribute value and number of occurrences in case.
** '''Total''':
***The total number of cases having the found root cause.
** '''Selected''':
***The number of cases that have the found root cause and belong to the selected cases.
**'''Compared''':
*** The number of cases that have the found root cause and don't belong to the selected cases.
**Columns when WeightingExpression not have value:
***'''Contribution''':
****The number of cases which contribute to the deviation from the average percentage of selected cases among all analyzed cases.
***'''ContributionPercentage''':
****The percent of cases which contribute to the deviation from the average percentage.
*** '''DifferencePercentage''':
****The deviation in percentage between selected cases with the found root cause and the average percentage of selected cases among all analyzed cases.
***'''SelectedPercentage''':
**** The percent of selected cases that have the found root cause out of all cases with that root cause.
**Columns when WeightingExpression have value:
***'''Contribution''':
****The sum of case weights which contribute to the deviation from the average percentage of selected case weights among all analyzed cases.
***'''ContributionPercentage''':
****The percent of case weights which contribute to the deviation from the average percentage.
*** '''DifferencePercentage''':
****The deviation in percentage between selected case weights with the found root cause and the average percentage of selected case weights among all analyzed cases.
***'''SelectedPercentage''':
****The percent of selected case weights that have the found root cause out of all case weights with that root cause.
*** '''SelectedWeight''':
**** The sum of weights that have the found root cause and belong to the selected cases.
*** '''ComparedWeight''':
****The sum of weights that have the found root cause and don't belong to the selected cases.
*** '''TotalWeight''':
****The sum of weights of all cases with that root cause.

===Example===
Calculate root cause analysis for given model using parameters read from the [[Web API: Expression/query|query configuration]].<syntaxhighlight lang="json">
{
"ProcessingMethod": "DataFrame",
"Root":"let m = _; _system.RootCauses.FindForDataFrame(m, _query.Configuration.Parameters.FindRootCausesParameters.Clone().Extend(#{\"Filter\": _query.Configuration.Filter}))",
"Parameters": {
"FindRootCausesParameters": {
"CaseAttributeTypes": ["Account Manager","Customer Group","Product Group","Region"],
"Selection": {"Items":[{"Type":"IncludeCases","Items":[{"Type":"CaseAttributeValue","Attribute":"Product Group","StringifiedValues":["0Hats"]}]}]},
"MaxNumUniqueValues": 2,
"MaximumRowCount": 1000,
"MinValueUsage": 0.20,
"WeightingExpression": "Column(\"Cost\")"
}
},
"Ordering": [
{"Name": "Contribution", "Direction": "Descending"}
]
}

</syntaxhighlight>

==Utils.GetSampledEvents==
Returns a [[SqlDataFrame in Expression Language|SqlDataFrame]] containing sampled events of given model where given filter is first applied.

===Parameters ===

*'''sourceModel''':
**ProcessAnalyzer model object of the model whose event data is to be filtered and sampled.
*'''sampledCaseCount''':
**The maximum number of cases to return (or null if all cases should be returned).
* '''filter''':
**JSON filter to be applied on the event data of the source model prior to performing the sampling.

===Return value===
[[SqlDataFrame in Expression Language|SqlDataFrame]] containing sampled events of given model where given filter is first applied.

===Example===
Get a sample of all the events of at most 1000 cases having "Hats" as "Product Group" case attribute value from model identified by modelId.<syntaxhighlight lang="typescript">
let eventDataSampleSdf = _system.Utils.GetSampledEvents(modelId, 1000, #{
"Items":[#{
"Type":"IncludeCases",
"Items":[#{
"Type":"CaseAttributeValue",
"Attribute":"Product Group",
"StringifiedValues":["0Hats"]
}]
}]
});
</syntaxhighlight>

==Utils.ModifyColumnTypes==
In-place modifies the column types of given columns in given data table.

===Parameters===
*'''dataTable''':
**ProcessAnalyzer model object of the model whose event data is to be filtered and sampled.
*'''columnTypesToSet''':
**Array of column name/type definitions to set.
***Only columns that are to be changed are required to be listed.
***Columns that don't exist in the data table will be skipped.

===Return value ===
Returns the modified data table object.

===Example===
Modify column "CaseId" to be of type string.<syntaxhighlight lang="typescript">
_system.Utils.ModifyColumnTypes(
eventsTable,
#{#{"Name": "CaseId", "DataType": "String"}}
);

</syntaxhighlight>

==Utils.RunFunctionWithParallelLogging==
Runs given function that generates logging information into given data table in a way that all the logging will be included into the generated script run log as well (if run inside a script).

Internally polls the table every 5 seconds for new rows and adds all the newly added rows to script log.

===Parameters===

*'''logTable''':
**A DataTable used for logging.
*'''callbackFunc''':
** Function that uses given data table for logging it's current status.

===Return value===
The result of the callback function.

=== Example===
Run stored procedure named "StoredProcedureTest" in Snowflake, that generates new rows to log table identified by logTableId and log the generated rows into the script run log.<syntaxhighlight lang="typescript">
_system.Utils.RunFunctionWithParallelLogging(DataTableById(logTableId), () => {
CreateSnowflakeConnection().CallStoredProcedure("StoredProcedureTest", #{})
});

</syntaxhighlight>

System Library

2024-11-19T13:48:49Z

MarHink:

System library is a collection of Expression Language functions and properties that provide additional tools for scripting Process Analyzer functionalities. System library is referenced in scripts via ''_system''-property, which provides additional properties dedicated for different areas of interests for scripting.

The following hierarchy shows the properties and functions available in System Library:

* ML
** GeneratePredictionModel
**ApplyTransformations
*Parallel
**Run
*RootCauses
**FindRootCausesDataFrame
*Utils
**GetSampledEvents
**ModifyColumnTypes
**RunFunctionWithParallelLogging

==ML.GeneratePredictionModel==
Documentation here: [[Create Predicted Eventlog|GeneratePredictionModel]]

==ML.ApplyTransformations==
Documentation here: [[Create Simulated Eventlog|ApplyTransformations]]

==Parallel.Run==
Runs given functions in parallel.

===Parameters===
*'''functions''':
**An array of functions to run in parallel.

===Return value ===
An array of results returned by the called functions, in the same order as the function generating them in the ''functions''-parameter.

===Example===
The following script uses _system.Parallel.Run to run three functions:

* SAP-extraction from VBAK-table in SAP (connection parameters defined in connectionParametersDict-dictionary).
*Transform the extracted data by adding a new column.
*Load the data into data table identified by dataTableId.
<syntaxhighlight lang="typescript">
function ExtractTransformAndLoad(extractFunc, transformFunc, loadFunc)
{
let rawDataFlow = extractFunc();
let transformedDataFlow = ToDataFlow();

_system.Parallel.Run([
() => Catch({
let df;
while (!IsNullTop(df = rawDataFlow.Collect(#{"CollectChunk": true}))) {
transformedDataFlow.Append(transformFunc(df));
WriteLog(`A chunk having ${df.NRows} rows has been transformed.`);
}
if (rawDataFlow.HasError) {
transformedDataFlow.Fail("Error occurred during data extraction.");
}
else {
transformedDataFlow.Complete();
}
}, {
transformedDataFlow.Fail("Error occurred during transformation calculation.");
}),
() => {
loadFunc(transformedDataFlow);
}
]);
}

ExtractTransformAndLoad(
() => ExtractSap(connectionParametersDict.Extend(
[
"FieldNames": "VBELN,ERDAT,ERZET,ERNAM,NETWR,WAERK",
"QueryTable": "VBAK",
"Options": ["VBELN BETWEEN '0000017448'" ,"AND '0060000042'"],
"UseGateway": true
])
),
df => df.SetColumns(["Test": () => `${Column("NETWR")} ${Column("WAERK")}`]),
dataFlow => {
DataTableById(dataTableId).Import(dataFlow, ["Append": 0]);
}
);

DataTableById(dataTableId).SqlDataFrame.OrderByColumns(["VBELN"], [true]).Collect()
</syntaxhighlight>

==RootCauses.FindForDataFrame==
Finds root causes for a particular process phenomenon by comparing properties of selected cases against those of all cases in given model.

Based on the similar in-memory function: [[FindRootCauses Function|EventLog.FindRootCauses]].

===Parameters===

*'''model''':
**Model object of a model whose data tables are used to calculate the root causes.
*'''parameters''':
**A parameter convertible to a StringDictionary object with the following supported key-values:
*** '''Filter''':
****Filter json (#30921#) that is applied to the event log before calculating root causes.
*** '''Selection''':
**** Selection json (#30927#) that defines selected cases to find root causes for. Is applied on top of the filtered event log (specified by Filter parameter).
****If nothing is selected, 100% of cases are counted as selected.
*** '''CaseAttributeTypes''':
****An array of strings with the names of case attributes included into the root causes.
****Only case attributes of type string, integer or boolean are included.
****If CaseAttributeTypes is null or empty string or not specified, all case attributes having type "String" are included into the root causes.
****If the model doesn't have a case attribute with the specified name, an error message is shown.
*** '''EventAttributeTypes''':
****An array of strings with the names of event attributes included into the root causes.
****Only event attributes of type string are included.
****If EventAttributeTypes is null, all event attributes having type "String" are included into the root causes.
****If EventAttributeTypes is empty array or not specified, event attributes not applied to the root causes.
****If the model doesn't have an event attribute with the specified name, an error message is shown.
****Analysis column Name should contain Event Attribute Name.
****Analysis column Value should contain Event Attribute Value and number of occurrences in case: <value> (count).
****Analysis column Type should have "EventAttributeValue" as its value.
***'''WeightingExpression''':
**** Expression providing weights for each case.
****Expression, if defined, must be any of the following types:
***** A string containing the [[SQL Expressions|SqlExpression]] to evaluate.
*****A SqlExpression object, created e.g., using ToSqlExpression-function (see also: [[QPR_ProcessAnalyzer_Expressions#In-memory_expression_blocks_in_SQL_expressions|In-memory expression blocks in SQL expressions]]).
****A row is filtered out of result if expression result is null.
***'''MaximumRowCount''':
****The maximum number of the most and least contributing root causes to return. Thus, the actual number of returned rows can be at most two times this value (if specified).
****If undefined, 200 is used.
**** If set to 0, all rows are returned.
***'''MinValueUsage''':
****The minimum total usage of a value included into the comparison. The number of cases having every returned value should be at least given percentage (a float value between 0.0 and 1.0) of all the compared cases.
*****If not defined or null, all values are included (=default).
***'''MaxNumUniqueValues''':
****Maximum number of unique values to include into the comparison for each attribute column. If the amount of unique values for any attribute exceeds this value, only given number of attributes are included that have the highest usage.
****If not defined or null, all values are included (=default).
***'''IncludeOthers''':
****Should the rest of the attribute values not included due to MinValueUsage or MaxNumUniqueValues filtering be included as an aggregated "Others" value?
****If given any string value, that value is used as the value for all the aggregated values.
****Default = undefined => others values will not be included into the results.
***'''ValueIfNull''': Value used to indicate null-values.
****Default = "(blank)".
****Must be not null.

===Return value===
Returns a [[SqlDataFrame in Expression Language|SqlDataFrame]] object with the following columns:

*Common columns:
**'''Type''':
***Type of the root cause:
****"CaseAttributeValue" for case attributes.
****"EventAttributeValue" for event attributes.
**'''Name''':
***When type is CaseAttributeValue, case attribute name.
***When type is EventAttributeValue, event attribute name.
**'''Value''':
***When type is CaseAttributeValue, case attribute value.
***When type is EventAttributeValue, event attribute value and number of occurrences in case.
** '''Total''':
***The total number of cases having the found root cause.
** '''Selected''':
***The number of cases that have the found root cause and belong to the selected cases.
**'''Compared''':
*** The number of cases that have the found root cause and don't belong to the selected cases.
**Columns when WeightingExpression not have value:
***'''Contribution''':
****The number of cases which contribute to the deviation from the average percentage of selected cases among all analyzed cases.
***'''ContributionPercentage''':
****The percent of cases which contribute to the deviation from the average percentage.
*** '''DifferencePercentage''':
****The deviation in percentage between selected cases with the found root cause and the average percentage of selected cases among all analyzed cases.
***'''SelectedPercentage''':
**** The percent of selected cases that have the found root cause out of all cases with that root cause.
**Columns when WeightingExpression have value:
***'''Contribution''':
****The sum of case weights which contribute to the deviation from the average percentage of selected case weights among all analyzed cases.
***'''ContributionPercentage''':
****The percent of case weights which contribute to the deviation from the average percentage.
*** '''DifferencePercentage''':
****The deviation in percentage between selected case weights with the found root cause and the average percentage of selected case weights among all analyzed cases.
***'''SelectedPercentage''':
****The percent of selected case weights that have the found root cause out of all case weights with that root cause.
*** '''SelectedWeight''':
**** The sum of weights that have the found root cause and belong to the selected cases.
*** '''ComparedWeight''':
****The sum of weights that have the found root cause and don't belong to the selected cases.
*** '''TotalWeight''':
****The sum of weights of all cases with that root cause.

===Example===
Calculate root cause analysis for given model using parameters read from the [[Web API: Expression/query|query configuration]].<syntaxhighlight lang="json">
{
"ProcessingMethod": "DataFrame",
"Root":"let m = _; _system.RootCauses.FindForDataFrame(m, _query.Configuration.Parameters.FindRootCausesParameters.Clone().Extend(#{\"Filter\": _query.Configuration.Filter}))",
"Parameters": {
"FindRootCausesParameters": {
"CaseAttributeTypes": ["Account Manager","Customer Group","Product Group","Region"],
"Selection": {"Items":[{"Type":"IncludeCases","Items":[{"Type":"CaseAttributeValue","Attribute":"Product Group","StringifiedValues":["0Hats"]}]}]},
"MaxNumUniqueValues": 2,
"MaximumRowCount": 1000,
"MinValueUsage": 0.20,
"WeightingExpression": "Column(\"Cost\")"
}
},
"Ordering": [
{"Name": "Contribution", "Direction": "Descending"}
]
}

</syntaxhighlight>

==Utils.GetSampledEvents==
Returns a [[SqlDataFrame in Expression Language|SqlDataFrame]] containing sampled events of given model where given filter is first applied.

===Parameters ===

*'''sourceModel''':
**ProcessAnalyzer model object of the model whose event data is to be filtered and sampled.
*'''sampledCaseCount''':
**The maximum number of cases to return (or null if all cases should be returned).
* '''filter''':
**JSON filter to be applied on the event data of the source model prior to performing the sampling.

===Return value===
[[SqlDataFrame in Expression Language|SqlDataFrame]] containing sampled events of given model where given filter is first applied.

===Example===
Get a sample of all the events of at most 1000 cases having "Hats" as "Product Group" case attribute value from model identified by modelId.<syntaxhighlight lang="typescript">
let eventDataSampleSdf = _system.Utils.GetSampledEvents(modelId, 1000, #{
"Items":[#{
"Type":"IncludeCases",
"Items":[#{
"Type":"CaseAttributeValue",
"Attribute":"Product Group",
"StringifiedValues":["0Hats"]
}]
}]
});
</syntaxhighlight>

==Utils.ModifyColumnTypes==
In-place modifies the column types of given columns in given data table.

===Parameters===
*'''dataTable''':
**ProcessAnalyzer model object of the model whose event data is to be filtered and sampled.
*'''columnTypesToSet''':
**Array of column name/type definitions to set.
***Only columns that are to be changed are required to be listed.
***Columns that don't exist in the data table will be skipped.

===Return value ===
Returns the modified data table object.

===Example===
Modify column "CaseId" to be of type string.<syntaxhighlight lang="typescript">
_system.Utils.ModifyColumnTypes(
eventsTable,
#{#{"Name": "CaseId", "DataType": "String"}}
);

</syntaxhighlight>

==Utils.RunFunctionWithParallelLogging==
Runs given function that generates logging information into given data table in a way that all the logging will be included into the generated script run log as well (if run inside a script).

Internally polls the table every 5 seconds for new rows and adds all the newly added rows to script log.

===Parameters===

*'''logTable''':
**A DataTable used for logging.
*'''callbackFunc''':
** Function that uses given data table for logging it's current status.

===Return value===
The result of the callback function.

=== Example===
Run stored procedure named "StoredProcedureTest" in Snowflake, that generates new rows to log table identified by logTableId and log the generated rows into the script run log.<syntaxhighlight lang="typescript">
_system.Utils.RunFunctionWithParallelLogging(DataTableById(logTableId), () => {
CreateSnowflakeConnection().CallStoredProcedure("StoredProcedureTest", #{})
});

</syntaxhighlight>

QPR ProcessAnalyzer Wiki

2024-11-19T13:42:43Z

MarHink: Added system library link

<div class="downloadButton" style="width:190px;float:right;margin: 3px 12px 0px 15px;">[[Online_Learning_Platform|Online Learning<br />Platform]]</div>

Welcome to QPR ProcessAnalyzer Wiki! QPR ProcessAnalyzer is a software for turning event and transactional data into visual process analysis and intelligence. Topics in this documentation are divided based on user roles for process analysts, developers and administrators.

<div style="height:5px;"></div>
== For Process Analysts ==
This section contains information how to get started with QPR ProcessAnalyzer and how to create your first dashboards! This section also describes how to use filters and how to make different kinds of analyses with QPR ProcessAnalyzer.

<div style="display: flex;flex-wrap: wrap;">
<div style="flex: 1 0 210px;border:1px solid #dfdfdf;padding:0 1em 1em 1.5em;background-color:#F7FAFC;margin:10px 0px 0px 10px;">
=== Getting Started ===
* [[Getting Started with QPR ProcessAnalyzer]]
* [[QPR_ProcessAnalyzer_Native_App_in_Snowflake|Snowflake Native App]]
* [[Introduction to Process Mining|Introduction to Process Mining]]
* [[Process_Mining_Concepts|Process Mining Concepts]]
* [[Log_in_QPR_ProcessAnalyzer|Log in QPR ProcessAnalyzer]]
* [[Languages and Localization|Language and Localization Settings]]
* [[User Settings|User Settings]]
* [[Navigation_Menu|Navigation Menu Functions]]
</div>

<div style="flex: 1 0 210px;border:1px solid #dfdfdf;padding:0 1em 1em 1.5em;background-color:#F7FAFC;margin:10px 0px 0px 10px;">

=== Working with Dashboards ===
* [[QPR_ProcessAnalyzer_Project_Workspace|Project Workspace]]: [[QPR_ProcessAnalyzer_Project_Workspace#Projects|Projects]], [[QPR_ProcessAnalyzer_Project_Workspace#Dashboards|Dashboards]], [[QPR_ProcessAnalyzer_Project_Workspace#Models|Models]], [[QPR_ProcessAnalyzer_Project_Workspace#Datatables|Datatables]], [[Managing_Scripts|Scripts]], [[QPR_ProcessAnalyzer_Project_Workspace#Recycle_Bin|Recycle Bin]]
* [[Filtering_in_QPR_ProcessAnalyzer|Using Filters]]
* [[QPR ProcessAnalyzer Dashboard Designer|Creating Dashboards]]
* [[AI Assistant for QPR ProcessAnalyzer|AI Assistant]] (powered by generative AI)
* [[Dashboard Variables|Dynamic Variables in Dashboards]]
* [[Business Calendar|Business Calendar to Calculate Durations]]
* [[Best Practices for Designing Dashboards|Best Practices for Designing Dashboards]]
</div>

<div style="flex: 1 0 370px;border:1px solid #dfdfdf;padding:0 1em 1em 1.5em;background-color:#F7FAFC;margin:10px 0px 0px 10px;">

=== Analyses and Visualizations ===
<div style="float:left; width:50%;">
* [[Process Flowchart|Process Flowchart]]
* [[Object-Centric_Flowchart|Object-Centric Flowchart]]
* [[QPR ProcessAnalyzer Chart|Chart]] / [[Snowflake Chart|Snowflake Chart]]
** [[QPR ProcessAnalyzer Graphs|Graphs]]
** [[QPR_ProcessAnalyzer_Table|Table]]
** [[QPR_ProcessAnalyzer_Pivot_Table|Pivot Table]]
** [[QPR_ProcessAnalyzer_KPI_Card|KPI Card]]
** [[Measure,_Dimension_and_Column_Settings|Measure Settings]]
** [[Chart_On-screen_Settings|On-screen Settings]]
** [[Chart_Linked_Settings|Linked Settings]]
** [[Actions_to_Run_Script_in_Table|Run Script Actions]]
</div>
<div style="float:left; width:50%;">
* [[Root Causes|Root Causes Analysis]]
* [[Clustering Analysis|Clustering Analysis]]
* [[Conformance Analysis|Conformance Analysis]]
* [[Design Diagram|Design Diagram]] / [[QPR ProcessAnalyzer BPMN Editor|BPMN Editor]]
* [[Gantt_Chart|Gantt Chart]]
* [[Sankey_Chart|Sankey Chart]]
* [[Label and Link]]
* [[Image|Image]]
* [[Filter_Selectors|Filter Selectors]] / [[Dropdown_List_Selector|Dropdown List Selector]]
</div>
</div>

</div>

== For Citizen Developers ==
This section describes how to build ETL scripts that transform the source data into process mining models. There is also a detailed description how the process mining models can be configured so that they are optimal for the desired analyses. Finally, there is reference documentation for all expression language related functionality, that can be used both when writing custom KPI's in dashboards and in the ETL scripts.

<div style="display: flex;flex-wrap: wrap;">
<div style="flex: 1 0 370px;border:1px solid #dfdfdf;padding:0 1em 1em 1.5em;background-color:#F7FAFC;margin:10px 0px 0px 10px;">
=== Expression Language Reference ===
* [[SQL_Expressions|SQL Expressions for Snowflake]]
* In-memory expressions: [[QPR_ProcessAnalyzer_Expressions|Basic Syntax and Operations]] / [[Generic Functions in QPR ProcessAnalyzer|Generic Functions]]
* [[Generic_Objects_in_Expression_Language|Generic types]] ([[Generic_Properties_in_Expression_Language|Generic Properties]], [[Generic_Objects_in_Expression_Language#Array|Array]], [[Generic_Objects_in_Expression_Language#DateTime|DateTime]], [[Generic_Objects_in_Expression_Language#String|String]], [[Generic_Objects_in_Expression_Language#Timespan|Timespan]], [[Generic_Objects_in_Expression_Language#Dictionary|Dictionary]])
* [[Process_Mining_Objects_in_Expression_Language|In-memory models API]] ([[Process_Mining_Objects_in_Expression_Language#AttributeType|AttributeType]], [[Process_Mining_Objects_in_Expression_Language#Case|Case]], [[Process_Mining_Objects_in_Expression_Language#Event|Event]], [[Process_Mining_Objects_in_Expression_Language#EventLog|EventLog]], [[Process_Mining_Objects_in_Expression_Language#EventType|EventType]], [[Process_Mining_Objects_in_Expression_Language#Flow|Flow]], [[Process_Mining_Objects_in_Expression_Language#FlowOccurrence|FlowOccurrence]], [[Process_Mining_Objects_in_Expression_Language#Variation|Variation]])
* Configuration objects ([[Dashboard_in_Expression_Language|Dashboard]], [[Datatable_in_Expression_Language|Datatable]], [[QPR_ProcessAnalyzer_Objects_in_Expression_Language#Model|Model]], [[QPR_ProcessAnalyzer_Objects_in_Expression_Language#Filter|Filter]], [[Diagram_in_Expression_Language|Diagram]], [[QPR_ProcessAnalyzer_Objects_in_Expression_Language#Script|Script]], [[QPR_ProcessAnalyzer_Objects_in_Expression_Language#Project|Project]], [[QPR_ProcessAnalyzer_Objects_in_Expression_Language#User.2FGroup|User/Group]])
* Tabular data ([[DataFrame in Expression Language|DataFrame]], [[SqlDataFrame in Expression Language|SqlDataFrame]], [[DataFlow_in_Expression_Language|DataFlow]])
* [[Machine_Learning_Functions_in_Expression_Language|Machine Learning API]] / [[Conformance_Checking|Conformance Checking]]
* [[Filtering_in_QPR_ProcessAnalyzer_Queries|Filter Rules JSON]]
* [[System Library]]
</div>

<div style="flex: 1 0 250px;border:1px solid #dfdfdf;padding:0 1em 1em 1.5em;background-color:#F7FAFC;margin:10px 0px 0px 10px;">

=== Create Process Mining Models ===
* [[Creating Process Mining Model|Walkthrough: Creating Process Mining Model]]
* [[Exporting_and_Importing_Data_in_QPR_ProcessAnalyzer|Importing Data from CSV, XES and PACM files]]
* [[Object-centric_Process_Mining_Model|Object-centric Process Mining]]
* [[Event Ordering for Identical Timestamps|Event Ordering for Identical Timestamps]]
* [[Managing Time Zones and Local Time|Time Zones and Local Time]]
* [[Best_Practices_for_Designing_Models|Best Practices for Designing Models]]
* [[Create_Predicted_Eventlog|Create Predicted Eventlog]]
* [[Create Simulated Eventlog]]
==== For In-memory Models ====
* [[Calculated Attributes in QPR ProcessAnalyzer|Calculated Case and Event Attributes]]
* [[Email Notifications|Email Notifications]]
* [[QPR ProcessAnalyzer Model Datasources|Model datasources]] ([[QPR ProcessAnalyzer Model Datasources#Loading Data from Datatables|Datatable]], [[QPR ProcessAnalyzer Model Datasources#Loading Script|Loading Script]], [[QPR ProcessAnalyzer Model Datasources#ODBC_Datasource|ODBC]])
* [[Case Level Permissions|Case Level Permissions]]
* [[Automatic Model Loading on Server Startup|Keeping Models Always Available]]
</div>

<div style="flex: 1 0 250px;border:1px solid #dfdfdf;padding:0 1em 1em 1.5em;background-color:#F7FAFC;margin:10px 0px 0px 10px;">

=== Data Integrations and Connectors ===
* [[Managing Scripts|Managing Scripts in Workspace]]
* [[Datatable_Properties_Dialog|Managing Datatables in Workspace]]
* [[SQL Scripting for ETL|Writing SQL Scripts]]
* [[SQL Scripting Commands|SQL Scripting Commands Reference]]
* [[Storing Secrets for Scripts|Storing Secrets]]
* [[QPR ProcessAnalyzer ScriptLauncher|Installing and using QPR ScriptLauncher]]
* [[Importing_Data_from_SAP|How to Import Data from SAP]]
* [[Anonymize data|Anonymize data]]
* [[Expression Script Examples|Expression Script Examples]]
* [[QPR ProcessAnalyzer API|QPR ProcessAnalyzer REST API]]
* [[Sample Eventlog Files|Sample Eventlogs]]
* [[QPR TaskRecorder]]
</div>
</div>

== For System Administrators ==
This section starts with the planning of QPR ProcessAnalyzer installation. After all requirements have been fulfilled, you can continue with the installation and configuration of QPR ProcessAnalyzer. Finally, learn how to manage users and perform other administrative tasks.

<div style="display: flex;flex-wrap: wrap;">
<div style="flex: 1 0 290px;border:1px solid #dfdfdf;padding:0 1em 1em 1.5em;background-color:#F7FAFC;margin:10px 0px 0px 10px;">
=== Planning Installation ===
* [[QPR ProcessAnalyzer System Requirements|System Requirements]]
* [[QPR ProcessAnalyzer System Architecture|System Architecture]]
* [[User Session Management|User Session Management]]
* [[QPR ProcessAnalyzer Release Notes|QPR ProcessAnalyzer Release Notes]]
* [[QPR_TaskRecorder_Release_Notes|QPR TaskRecorder Release Notes]]
* [[QPR_ProcessAnalyzer_Downloads|Downloads Page]]
</div>

<div style="flex: 1 0 290px;border:1px solid #dfdfdf;padding:0 1em 1em 1.5em;background-color:#F7FAFC;margin:10px 0px 0px 10px;">

=== Installing and Configuring ===
* [[Installing QPR ProcessAnalyzer Server|Install QPR ProcessAnalyzer Server]] (or [[Updating_QPR_ProcessAnalyzer_Server|update existing]])
* [[Snowflake_Connection_Configuration|Configure Snowflake Connection]]
* [[Setting_up_Scripting_Sandbox|Setting up SQL Scripting Sandbox]]
* [[SAML_2.0_Federated_Authentication|SAML 2.0 Authentication]]
* [[QPR ProcessAnalyzer Security Hardening|Security Hardening]]
* [[Activate_QPR_ProcessAnalyzer_using_ActivationUtility|Activation without Internet Connection]]
</div>

<div style="flex: 1 0 290px;border:1px solid #dfdfdf;padding:0 1em 1em 1.5em;background-color:#F7FAFC;margin:10px 0px 0px 10px;">

=== Administrating System ===
* [[Roles and Permissions|User Roles and Permissions]]
* [[Manage Users and Groups|Managing Users]]
* [[PA_Configuration_database_table|System Configurations]]
* [[QPR ProcessAnalyzer Logs|Logs for Audit and Troubleshooting]]
* [[In-memory_Models_Management|In-memory Models Management]]
</div>
</div>

== Agreements ==
See the [[QPR End User Software License Agreement]] and [[QPR Software as a Service Agreement]].

__NOTOC__

System Library

2024-11-19T13:35:36Z

MarHink: /* Example */

System library is a collection of Expression Language functions and properties that provide additional tools for scripting Process Analyzer functionalities. System library is referenced in scripts via ''_system''-property, which provides additional properties dedicated for different areas of interests for scripting.

The following hierarchy shows the properties and functions available in System Library:

* ML
** GeneratePredictionModel (documentation here: [[Create Predicted Eventlog|GeneratePredictionModel]])
** ApplyTransformations (documentation here: [[Create Simulated Eventlog|ApplyTransformations]])
* Parallel
** Run
* RootCauses
** FindRootCausesDataFrame
* Utils
** GetSampledEvents
** ModifyColumnTypes
** RunFunctionWithParallelLogging

== Parallel.Run ==
Runs given functions in parallel.

=== Parameters ===
* '''functions''':
** An array of functions to run in parallel.

=== Return value ===
An array of results returned by the called functions, in the same order as the function generating them in the ''functions''-parameter.

=== Example ===
The following script uses _system.Parallel.Run to run three functions:

* SAP-extraction from VBAK-table in SAP (connection parameters defined in connectionParametersDict-dictionary).
* Transform the extracted data by adding a new column.
* Load the data into data table identified by dataTableId.
<syntaxhighlight lang="typescript">
function ExtractTransformAndLoad(extractFunc, transformFunc, loadFunc)
{
let rawDataFlow = extractFunc();
let transformedDataFlow = ToDataFlow();

_system.Parallel.Run([
() => Catch({
let df;
while (!IsNullTop(df = rawDataFlow.Collect(#{"CollectChunk": true}))) {
transformedDataFlow.Append(transformFunc(df));
WriteLog(`A chunk having ${df.NRows} rows has been transformed.`);
}
if (rawDataFlow.HasError) {
transformedDataFlow.Fail("Error occurred during data extraction.");
}
else {
transformedDataFlow.Complete();
}
}, {
transformedDataFlow.Fail("Error occurred during transformation calculation.");
}),
() => {
loadFunc(transformedDataFlow);
}
]);
}

ExtractTransformAndLoad(
() => ExtractSap(connectionParametersDict.Extend(
[
"FieldNames": "VBELN,ERDAT,ERZET,ERNAM,NETWR,WAERK",
"QueryTable": "VBAK",
"Options": ["VBELN BETWEEN '0000017448'" ,"AND '0060000042'"],
"UseGateway": true
])
),
df => df.SetColumns(["Test": () => `${Column("NETWR")} ${Column("WAERK")}`]),
dataFlow => {
DataTableById(dataTableId).Import(dataFlow, ["Append": 0]);
}
);

DataTableById(dataTableId).SqlDataFrame.OrderByColumns(["VBELN"], [true]).Collect()
</syntaxhighlight>

== RootCauses.FindForDataFrame ==
Finds root causes for a particular process phenomenon by comparing properties of selected cases against those of all cases in given model.

Based on the similar in-memory function: [[FindRootCauses Function|EventLog.FindRootCauses]].

=== Parameters ===

* '''model''':
** Model object of a model whose data tables are used to calculate the root causes.
* '''parameters''':
** A parameter convertible to a StringDictionary object with the following supported key-values:
*** '''Filter''':
**** Filter json (#30921#) that is applied to the event log before calculating root causes.
*** '''Selection''':
**** Selection json (#30927#) that defines selected cases to find root causes for. Is applied on top of the filtered event log (specified by Filter parameter).
**** If nothing is selected, 100% of cases are counted as selected.
*** '''CaseAttributeTypes''':
**** An array of strings with the names of case attributes included into the root causes.
**** Only case attributes of type string, integer or boolean are included.
**** If CaseAttributeTypes is null or empty string or not specified, all case attributes having type "String" are included into the root causes.
**** If the model doesn't have a case attribute with the specified name, an error message is shown.
*** '''EventAttributeTypes''':
**** An array of strings with the names of event attributes included into the root causes.
**** Only event attributes of type string are included.
**** If EventAttributeTypes is null, all event attributes having type "String" are included into the root causes.
**** If EventAttributeTypes is empty array or not specified, event attributes not applied to the root causes.
**** If the model doesn't have an event attribute with the specified name, an error message is shown.
**** Analysis column Name should contain Event Attribute Name.
**** Analysis column Value should contain Event Attribute Value and number of occurrences in case: <value> (count).
**** Analysis column Type should have "EventAttributeValue" as its value.
*** '''WeightingExpression''':
**** Expression providing weights for each case.
**** Expression, if defined, must be any of the following types:
***** A string containing the [[SQL Expressions|SqlExpression]] to evaluate.
***** A SqlExpression object, created e.g., using ToSqlExpression-function (see also: [[QPR_ProcessAnalyzer_Expressions#In-memory_expression_blocks_in_SQL_expressions|In-memory expression blocks in SQL expressions]]).
**** A row is filtered out of result if expression result is null.
*** '''MaximumRowCount''':
**** The maximum number of the most and least contributing root causes to return. Thus, the actual number of returned rows can be at most two times this value (if specified).
**** If undefined, 200 is used.
**** If set to 0, all rows are returned.
*** '''MinValueUsage''':
**** The minimum total usage of a value included into the comparison. The number of cases having every returned value should be at least given percentage (a float value between 0.0 and 1.0) of all the compared cases.
***** If not defined or null, all values are included (=default).
*** '''MaxNumUniqueValues''':
**** Maximum number of unique values to include into the comparison for each attribute column. If the amount of unique values for any attribute exceeds this value, only given number of attributes are included that have the highest usage.
**** If not defined or null, all values are included (=default).
*** '''IncludeOthers''':
**** Should the rest of the attribute values not included due to MinValueUsage or MaxNumUniqueValues filtering be included as an aggregated "Others" value?
**** If given any string value, that value is used as the value for all the aggregated values.
**** Default = undefined => others values will not be included into the results.
*** '''ValueIfNull''': Value used to indicate null-values.
**** Default = "(blank)".
**** Must be not null.

=== Return value ===
Returns a [[SqlDataFrame in Expression Language|SqlDataFrame]] object with the following columns:

* Common columns:
** '''Type''':
*** Type of the root cause:
**** "CaseAttributeValue" for case attributes.
**** "EventAttributeValue" for event attributes.
** '''Name''':
*** When type is CaseAttributeValue, case attribute name.
*** When type is EventAttributeValue, event attribute name.
** '''Value''':
*** When type is CaseAttributeValue, case attribute value.
*** When type is EventAttributeValue, event attribute value and number of occurrences in case.
** '''Total''':
*** The total number of cases having the found root cause.
** '''Selected''':
*** The number of cases that have the found root cause and belong to the selected cases.
** '''Compared''':
*** The number of cases that have the found root cause and don't belong to the selected cases.
** Columns when WeightingExpression not have value:
*** '''Contribution''':
**** The number of cases which contribute to the deviation from the average percentage of selected cases among all analyzed cases.
*** '''ContributionPercentage''':
**** The percent of cases which contribute to the deviation from the average percentage.
*** '''DifferencePercentage''':
**** The deviation in percentage between selected cases with the found root cause and the average percentage of selected cases among all analyzed cases.
*** '''SelectedPercentage''':
**** The percent of selected cases that have the found root cause out of all cases with that root cause.
** Columns when WeightingExpression have value:
*** '''Contribution''':
**** The sum of case weights which contribute to the deviation from the average percentage of selected case weights among all analyzed cases.
*** '''ContributionPercentage''':
**** The percent of case weights which contribute to the deviation from the average percentage.
*** '''DifferencePercentage''':
**** The deviation in percentage between selected case weights with the found root cause and the average percentage of selected case weights among all analyzed cases.
*** '''SelectedPercentage''':
**** The percent of selected case weights that have the found root cause out of all case weights with that root cause.
*** '''SelectedWeight''':
**** The sum of weights that have the found root cause and belong to the selected cases.
*** '''ComparedWeight''':
**** The sum of weights that have the found root cause and don't belong to the selected cases.
*** '''TotalWeight''':
**** The sum of weights of all cases with that root cause.

=== Example ===
Calculate root cause analysis for given model using parameters read from the [[Web API: Expression/query|query configuration]].<syntaxhighlight lang="json">
{
"ProcessingMethod": "DataFrame",
"Root":"let m = _; _system.RootCauses.FindForDataFrame(m, _query.Configuration.Parameters.FindRootCausesParameters.Clone().Extend(#{\"Filter\": _query.Configuration.Filter}))",
"Parameters": {
"FindRootCausesParameters": {
"CaseAttributeTypes": ["Account Manager","Customer Group","Product Group","Region"],
"Selection": {"Items":[{"Type":"IncludeCases","Items":[{"Type":"CaseAttributeValue","Attribute":"Product Group","StringifiedValues":["0Hats"]}]}]},
"MaxNumUniqueValues": 2,
"MaximumRowCount": 1000,
"MinValueUsage": 0.20,
"WeightingExpression": "Column(\"Cost\")"
}
},
"Ordering": [
{"Name": "Contribution", "Direction": "Descending"}
]
}

</syntaxhighlight>

== Utils.GetSampledEvents ==
Returns a [[SqlDataFrame in Expression Language|SqlDataFrame]] containing sampled events of given model where given filter is first applied.

=== Parameters ===

* '''sourceModel''':
** ProcessAnalyzer model object of the model whose event data is to be filtered and sampled.
* '''sampledCaseCount''':
** The maximum number of cases to return (or null if all cases should be returned).
* '''filter''':
** JSON filter to be applied on the event data of the source model prior to performing the sampling.

=== Return value ===
[[SqlDataFrame in Expression Language|SqlDataFrame]] containing sampled events of given model where given filter is first applied.

=== Example ===
Get a sample of all the events of at most 1000 cases having "Hats" as "Product Group" case attribute value from model identified by modelId.<syntaxhighlight lang="typescript">
let eventDataSampleSdf = _system.Utils.GetSampledEvents(modelId, 1000, #{
"Items":[#{
"Type":"IncludeCases",
"Items":[#{
"Type":"CaseAttributeValue",
"Attribute":"Product Group",
"StringifiedValues":["0Hats"]
}]
}]
});
</syntaxhighlight>

== Utils.ModifyColumnTypes ==
In-place modifies the column types of given columns in given data table.

=== Parameters ===
* '''dataTable''':
** ProcessAnalyzer model object of the model whose event data is to be filtered and sampled.
* '''columnTypesToSet''':
** Array of column name/type definitions to set.
*** Only columns that are to be changed are required to be listed.
*** Columns that don't exist in the data table will be skipped.

=== Return value ===
Returns the modified data table object.

=== Example ===
Modify column "CaseId" to be of type string.<syntaxhighlight lang="typescript">
_system.Utils.ModifyColumnTypes(
eventsTable,
#{#{"Name": "CaseId", "DataType": "String"}}
);

</syntaxhighlight>

== Utils.RunFunctionWithParallelLogging ==
Runs given function that generates logging information into given data table in a way that all the logging will be included into the generated script run log as well (if run inside a script).

Internally polls the table every 5 seconds for new rows and adds all the newly added rows to script log.

=== Parameters ===

* '''logTable''':
** A DataTable used for logging.
* '''callbackFunc''':
** Function that uses given data table for logging it's current status.

=== Return value ===
The result of the callback function.

=== Example ===
Run stored procedure named "StoredProcedureTest" in Snowflake, that generates new rows to log table identified by logTableId and log the generated rows into the script run log.<syntaxhighlight lang="typescript">
_system.Utils.RunFunctionWithParallelLogging(DataTableById(logTableId), () => {
CreateSnowflakeConnection().CallStoredProcedure("StoredProcedureTest", #{})
});

</syntaxhighlight>

System Library

2024-11-19T13:26:55Z

MarHink:

System library is a collection of Expression Language functions and properties that provide additional tools for scripting Process Analyzer functionalities. System library is referenced in scripts via ''_system''-property, which provides additional properties dedicated for different areas of interests for scripting.

The following hierarchy shows the properties and functions available in System Library:

* ML
** GeneratePredictionModel (documentation here: [[Create Predicted Eventlog|GeneratePredictionModel]])
** ApplyTransformations (documentation here: [[Create Simulated Eventlog|ApplyTransformations]])
* Parallel
** Run
* RootCauses
** FindRootCausesDataFrame
* Utils
** GetSampledEvents
** ModifyColumnTypes
** RunFunctionWithParallelLogging

== Parallel.Run ==
Runs given functions in parallel.

=== Parameters ===
* '''functions''':
** An array of functions to run in parallel.

=== Return value ===
An array of results returned by the called functions, in the same order as the function generating them in the ''functions''-parameter.

=== Example ===
The following script uses _system.Parallel.Run to run three functions:

* SAP-extraction from VBAK-table in SAP (connection parameters defined in connectionParametersDict-dictionary).
* Transform the extracted data by adding a new column.
* Load the data into data table identified by dataTableId.
<syntaxhighlight lang="typescript">
function ExtractTransformAndLoad(extractFunc, transformFunc, loadFunc)
{
let rawDataFlow = extractFunc();
let transformedDataFlow = ToDataFlow();

_system.Parallel.Run([
() => Catch({
let df;
while (!IsNullTop(df = rawDataFlow.Collect(#{"CollectChunk": true}))) {
transformedDataFlow.Append(transformFunc(df));
WriteLog(`A chunk having ${df.NRows} rows has been transformed.`);
}
if (rawDataFlow.HasError) {
transformedDataFlow.Fail("Error occurred during data extraction.");
}
else {
transformedDataFlow.Complete();
}
}, {
transformedDataFlow.Fail("Error occurred during transformation calculation.");
}),
() => {
loadFunc(transformedDataFlow);
}
]);
}

ExtractTransformAndLoad(
() => ExtractSap(connectionParametersDict.Extend(
[
"FieldNames": "VBELN,ERDAT,ERZET,ERNAM,NETWR,WAERK",
"QueryTable": "VBAK",
"Options": ["VBELN BETWEEN '0000017448'" ,"AND '0060000042'"],
"UseGateway": true
])
),
df => df.SetColumns(["Test": () => `${Column("NETWR")} ${Column("WAERK")}`]),
dataFlow => {
DataTableById(dataTableId).Import(dataFlow, ["Append": 0]);
}
);

DataTableById(dataTableId).SqlDataFrame.OrderByColumns(["VBELN"], [true]).Collect()
</syntaxhighlight>

== RootCauses.FindForDataFrame ==
Finds root causes for a particular process phenomenon by comparing properties of selected cases against those of all cases in given model.

Based on the similar in-memory function: [[FindRootCauses Function|EventLog.FindRootCauses]].

=== Parameters ===

* '''model''':
** Model object of a model whose data tables are used to calculate the root causes.
* '''parameters''':
** A parameter convertible to a StringDictionary object with the following supported key-values:
*** '''Filter''':
**** Filter json (#30921#) that is applied to the event log before calculating root causes.
*** '''Selection''':
**** Selection json (#30927#) that defines selected cases to find root causes for. Is applied on top of the filtered event log (specified by Filter parameter).
**** If nothing is selected, 100% of cases are counted as selected.
*** '''CaseAttributeTypes''':
**** An array of strings with the names of case attributes included into the root causes.
**** Only case attributes of type string, integer or boolean are included.
**** If CaseAttributeTypes is null or empty string or not specified, all case attributes having type "String" are included into the root causes.
**** If the model doesn't have a case attribute with the specified name, an error message is shown.
*** '''EventAttributeTypes''':
**** An array of strings with the names of event attributes included into the root causes.
**** Only event attributes of type string are included.
**** If EventAttributeTypes is null, all event attributes having type "String" are included into the root causes.
**** If EventAttributeTypes is empty array or not specified, event attributes not applied to the root causes.
**** If the model doesn't have an event attribute with the specified name, an error message is shown.
**** Analysis column Name should contain Event Attribute Name.
**** Analysis column Value should contain Event Attribute Value and number of occurrences in case: <value> (count).
**** Analysis column Type should have "EventAttributeValue" as its value.
*** '''WeightingExpression''':
**** Expression providing weights for each case.
**** Expression, if defined, must be any of the following types:
***** A string containing the [[SQL Expressions|SqlExpression]] to evaluate.
***** A SqlExpression object, created e.g., using ToSqlExpression-function (see also: [[QPR_ProcessAnalyzer_Expressions#In-memory_expression_blocks_in_SQL_expressions|In-memory expression blocks in SQL expressions]]).
**** A row is filtered out of result if expression result is null.
*** '''MaximumRowCount''':
**** The maximum number of the most and least contributing root causes to return. Thus, the actual number of returned rows can be at most two times this value (if specified).
**** If undefined, 200 is used.
**** If set to 0, all rows are returned.
*** '''MinValueUsage''':
**** The minimum total usage of a value included into the comparison. The number of cases having every returned value should be at least given percentage (a float value between 0.0 and 1.0) of all the compared cases.
***** If not defined or null, all values are included (=default).
*** '''MaxNumUniqueValues''':
**** Maximum number of unique values to include into the comparison for each attribute column. If the amount of unique values for any attribute exceeds this value, only given number of attributes are included that have the highest usage.
**** If not defined or null, all values are included (=default).
*** '''IncludeOthers''':
**** Should the rest of the attribute values not included due to MinValueUsage or MaxNumUniqueValues filtering be included as an aggregated "Others" value?
**** If given any string value, that value is used as the value for all the aggregated values.
**** Default = undefined => others values will not be included into the results.
*** '''ValueIfNull''': Value used to indicate null-values.
**** Default = "(blank)".
**** Must be not null.

=== Return value ===
Returns a [[SqlDataFrame in Expression Language|SqlDataFrame]] object with the following columns:

* Common columns:
** '''Type''':
*** Type of the root cause:
**** "CaseAttributeValue" for case attributes.
**** "EventAttributeValue" for event attributes.
** '''Name''':
*** When type is CaseAttributeValue, case attribute name.
*** When type is EventAttributeValue, event attribute name.
** '''Value''':
*** When type is CaseAttributeValue, case attribute value.
*** When type is EventAttributeValue, event attribute value and number of occurrences in case.
** '''Total''':
*** The total number of cases having the found root cause.
** '''Selected''':
*** The number of cases that have the found root cause and belong to the selected cases.
** '''Compared''':
*** The number of cases that have the found root cause and don't belong to the selected cases.
** Columns when WeightingExpression not have value:
*** '''Contribution''':
**** The number of cases which contribute to the deviation from the average percentage of selected cases among all analyzed cases.
*** '''ContributionPercentage''':
**** The percent of cases which contribute to the deviation from the average percentage.
*** '''DifferencePercentage''':
**** The deviation in percentage between selected cases with the found root cause and the average percentage of selected cases among all analyzed cases.
*** '''SelectedPercentage''':
**** The percent of selected cases that have the found root cause out of all cases with that root cause.
** Columns when WeightingExpression have value:
*** '''Contribution''':
**** The sum of case weights which contribute to the deviation from the average percentage of selected case weights among all analyzed cases.
*** '''ContributionPercentage''':
**** The percent of case weights which contribute to the deviation from the average percentage.
*** '''DifferencePercentage''':
**** The deviation in percentage between selected case weights with the found root cause and the average percentage of selected case weights among all analyzed cases.
*** '''SelectedPercentage''':
**** The percent of selected case weights that have the found root cause out of all case weights with that root cause.
*** '''SelectedWeight''':
**** The sum of weights that have the found root cause and belong to the selected cases.
*** '''ComparedWeight''':
**** The sum of weights that have the found root cause and don't belong to the selected cases.
*** '''TotalWeight''':
**** The sum of weights of all cases with that root cause.

=== Example ===
Calculate root cause analysis for given model using parameters read from the [[Web API: Expression/query|query configuration]].<syntaxhighlight lang="json">
{
"ProcessingMethod": "DataFrame",
"Root":"let m = _; _system.RootCauses.FindForDataFrame(m, _query.Configuration.Parameters.FindRootCausesParameters.Clone().Extend(#{\"Filter\": _query.Configuration.Filter}))",
"Parameters": {
"FindRootCausesParameters": {
"CaseAttributeTypes": ["Account Manager","Customer Group","Product Group","Region"],
"Selection": {"Items":[{"Type":"IncludeCases","Items":[{"Type":"CaseAttributeValue","Attribute":"Product Group","StringifiedValues":["0Hats"]}]}]},
"MaxNumUniqueValues": 2,
"MaximumRowCount": 1000,
"MinValueUsage": 0.20,
"WeightingExpression": "Column(\"Cost\")"
}
},
"Ordering": [
{"Name": "Contribution", "Direction": "Descending"}
]
}

</syntaxhighlight>

== Utils.GetSampledEvents ==
Returns a [[SqlDataFrame in Expression Language|SqlDataFrame]] containing sampled events of given model where given filter is first applied.

=== Parameters ===

* '''sourceModel''':
** ProcessAnalyzer model object of the model whose event data is to be filtered and sampled.
* '''sampledCaseCount''':
** The maximum number of cases to return (or null if all cases should be returned).
* '''filter''':
** JSON filter to be applied on the event data of the source model prior to performing the sampling.

=== Return value ===
[[SqlDataFrame in Expression Language|SqlDataFrame]] containing sampled events of given model where given filter is first applied.

=== Example ===
Get a sample of all the events of at most 1000 cases having "Hats" as "Product Group" case attribute value from model identified by modelId.<syntaxhighlight lang="typescript">
let eventDataSampleSdf = _system.Utils.GetSampledEvents(modelId, 1000, #{
"Items":[#{
"Type":"IncludeCases",
"Items":[#{
"Type":"CaseAttributeValue",
"Attribute":"Product Group",
"StringifiedValues":["0Hats"]
}]
}]
});
</syntaxhighlight>

== Utils.ModifyColumnTypes ==
In-place modifies the column types of given columns in given data table.

=== Parameters ===
* '''dataTable''':
** ProcessAnalyzer model object of the model whose event data is to be filtered and sampled.
* '''columnTypesToSet''':
** Array of column name/type definitions to set.
*** Only columns that are to be changed are required to be listed.
*** Columns that don't exist in the data table will be skipped.

=== Return value ===
Returns the modified data table object.

=== Example ===
Modify column "CaseId" to be of type string.<syntaxhighlight lang="typescript">
_system.Utils.ModifyColumnTypes(
eventsTable,
#{#{"Name": "CaseId", "DataType": "String"}}
);

</syntaxhighlight>

== Utils.RunFunctionWithParallelLogging ==
Runs given function that generates logging information into given data table in a way that all the logging will be included into the generated script run log as well (if run inside a script).

Internally polls the table every 5 seconds for new rows and adds all the newly added rows to script log.

=== Parameters ===

* '''logTable''':
** A DataTable used for logging.
* '''callbackFunc''':
** Function that uses given data table for logging it's current status.

=== Return value ===
The result of the callback function.

=== Example ===
Run given stored procedure that generates new rows to log table identified by logTableId and log the generated rows into the script run log.<syntaxhighlight lang="typescript">
_system.Utils.RunFunctionWithParallelLogging(DataTableById(logTableId), () => {
CreateSnowflakeConnection().CallStoredProcedure("StoredProcedureTest", #{})
});

</syntaxhighlight>

System Library

2024-11-19T13:26:24Z

MarHink:

System library is a collection of Expression Language functions and properties that provide additional tools for scripting Process Analyzer functionalities. System library is referenced in scripts via ''_system''-property, which provides additional properties dedicated for different areas of interests for scripting.

The following hierarchy shows the properties and functions available in System Library:

* ML
** GeneratePredictionModel (documented here: [[Create Predicted Eventlog|GeneratePredictionModel]])
** ApplyTransformations (documented here: [[Create Simulated Eventlog|ApplyTransformations]])
* Parallel
** Run
* RootCauses
** FindRootCausesDataFrame
* Utils
** GetSampledEvents
** ModifyColumnTypes
** RunFunctionWithParallelLogging

== Parallel.Run ==
Runs given functions in parallel.

=== Parameters ===
* '''functions''':
** An array of functions to run in parallel.

=== Return value ===
An array of results returned by the called functions, in the same order as the function generating them in the ''functions''-parameter.

=== Example ===
The following script uses _system.Parallel.Run to run three functions:

* SAP-extraction from VBAK-table in SAP (connection parameters defined in connectionParametersDict-dictionary).
* Transform the extracted data by adding a new column.
* Load the data into data table identified by dataTableId.
<syntaxhighlight lang="typescript">
function ExtractTransformAndLoad(extractFunc, transformFunc, loadFunc)
{
let rawDataFlow = extractFunc();
let transformedDataFlow = ToDataFlow();

_system.Parallel.Run([
() => Catch({
let df;
while (!IsNullTop(df = rawDataFlow.Collect(#{"CollectChunk": true}))) {
transformedDataFlow.Append(transformFunc(df));
WriteLog(`A chunk having ${df.NRows} rows has been transformed.`);
}
if (rawDataFlow.HasError) {
transformedDataFlow.Fail("Error occurred during data extraction.");
}
else {
transformedDataFlow.Complete();
}
}, {
transformedDataFlow.Fail("Error occurred during transformation calculation.");
}),
() => {
loadFunc(transformedDataFlow);
}
]);
}

ExtractTransformAndLoad(
() => ExtractSap(connectionParametersDict.Extend(
[
"FieldNames": "VBELN,ERDAT,ERZET,ERNAM,NETWR,WAERK",
"QueryTable": "VBAK",
"Options": ["VBELN BETWEEN '0000017448'" ,"AND '0060000042'"],
"UseGateway": true
])
),
df => df.SetColumns(["Test": () => `${Column("NETWR")} ${Column("WAERK")}`]),
dataFlow => {
DataTableById(dataTableId).Import(dataFlow, ["Append": 0]);
}
);

DataTableById(dataTableId).SqlDataFrame.OrderByColumns(["VBELN"], [true]).Collect()
</syntaxhighlight>

== RootCauses.FindForDataFrame ==
Finds root causes for a particular process phenomenon by comparing properties of selected cases against those of all cases in given model.

Based on the similar in-memory function: [[FindRootCauses Function|EventLog.FindRootCauses]].

=== Parameters ===

* '''model''':
** Model object of a model whose data tables are used to calculate the root causes.
* '''parameters''':
** A parameter convertible to a StringDictionary object with the following supported key-values:
*** '''Filter''':
**** Filter json (#30921#) that is applied to the event log before calculating root causes.
*** '''Selection''':
**** Selection json (#30927#) that defines selected cases to find root causes for. Is applied on top of the filtered event log (specified by Filter parameter).
**** If nothing is selected, 100% of cases are counted as selected.
*** '''CaseAttributeTypes''':
**** An array of strings with the names of case attributes included into the root causes.
**** Only case attributes of type string, integer or boolean are included.
**** If CaseAttributeTypes is null or empty string or not specified, all case attributes having type "String" are included into the root causes.
**** If the model doesn't have a case attribute with the specified name, an error message is shown.
*** '''EventAttributeTypes''':
**** An array of strings with the names of event attributes included into the root causes.
**** Only event attributes of type string are included.
**** If EventAttributeTypes is null, all event attributes having type "String" are included into the root causes.
**** If EventAttributeTypes is empty array or not specified, event attributes not applied to the root causes.
**** If the model doesn't have an event attribute with the specified name, an error message is shown.
**** Analysis column Name should contain Event Attribute Name.
**** Analysis column Value should contain Event Attribute Value and number of occurrences in case: <value> (count).
**** Analysis column Type should have "EventAttributeValue" as its value.
*** '''WeightingExpression''':
**** Expression providing weights for each case.
**** Expression, if defined, must be any of the following types:
***** A string containing the [[SQL Expressions|SqlExpression]] to evaluate.
***** A SqlExpression object, created e.g., using ToSqlExpression-function (see also: [[QPR_ProcessAnalyzer_Expressions#In-memory_expression_blocks_in_SQL_expressions|In-memory expression blocks in SQL expressions]]).
**** A row is filtered out of result if expression result is null.
*** '''MaximumRowCount''':
**** The maximum number of the most and least contributing root causes to return. Thus, the actual number of returned rows can be at most two times this value (if specified).
**** If undefined, 200 is used.
**** If set to 0, all rows are returned.
*** '''MinValueUsage''':
**** The minimum total usage of a value included into the comparison. The number of cases having every returned value should be at least given percentage (a float value between 0.0 and 1.0) of all the compared cases.
***** If not defined or null, all values are included (=default).
*** '''MaxNumUniqueValues''':
**** Maximum number of unique values to include into the comparison for each attribute column. If the amount of unique values for any attribute exceeds this value, only given number of attributes are included that have the highest usage.
**** If not defined or null, all values are included (=default).
*** '''IncludeOthers''':
**** Should the rest of the attribute values not included due to MinValueUsage or MaxNumUniqueValues filtering be included as an aggregated "Others" value?
**** If given any string value, that value is used as the value for all the aggregated values.
**** Default = undefined => others values will not be included into the results.
*** '''ValueIfNull''': Value used to indicate null-values.
**** Default = "(blank)".
**** Must be not null.

=== Return value ===
Returns a [[SqlDataFrame in Expression Language|SqlDataFrame]] object with the following columns:

* Common columns:
** '''Type''':
*** Type of the root cause:
**** "CaseAttributeValue" for case attributes.
**** "EventAttributeValue" for event attributes.
** '''Name''':
*** When type is CaseAttributeValue, case attribute name.
*** When type is EventAttributeValue, event attribute name.
** '''Value''':
*** When type is CaseAttributeValue, case attribute value.
*** When type is EventAttributeValue, event attribute value and number of occurrences in case.
** '''Total''':
*** The total number of cases having the found root cause.
** '''Selected''':
*** The number of cases that have the found root cause and belong to the selected cases.
** '''Compared''':
*** The number of cases that have the found root cause and don't belong to the selected cases.
** Columns when WeightingExpression not have value:
*** '''Contribution''':
**** The number of cases which contribute to the deviation from the average percentage of selected cases among all analyzed cases.
*** '''ContributionPercentage''':
**** The percent of cases which contribute to the deviation from the average percentage.
*** '''DifferencePercentage''':
**** The deviation in percentage between selected cases with the found root cause and the average percentage of selected cases among all analyzed cases.
*** '''SelectedPercentage''':
**** The percent of selected cases that have the found root cause out of all cases with that root cause.
** Columns when WeightingExpression have value:
*** '''Contribution''':
**** The sum of case weights which contribute to the deviation from the average percentage of selected case weights among all analyzed cases.
*** '''ContributionPercentage''':
**** The percent of case weights which contribute to the deviation from the average percentage.
*** '''DifferencePercentage''':
**** The deviation in percentage between selected case weights with the found root cause and the average percentage of selected case weights among all analyzed cases.
*** '''SelectedPercentage''':
**** The percent of selected case weights that have the found root cause out of all case weights with that root cause.
*** '''SelectedWeight''':
**** The sum of weights that have the found root cause and belong to the selected cases.
*** '''ComparedWeight''':
**** The sum of weights that have the found root cause and don't belong to the selected cases.
*** '''TotalWeight''':
**** The sum of weights of all cases with that root cause.

=== Example ===
Calculate root cause analysis for given model using parameters read from the [[Web API: Expression/query|query configuration]].<syntaxhighlight lang="json">
{
"ProcessingMethod": "DataFrame",
"Root":"let m = _; _system.RootCauses.FindForDataFrame(m, _query.Configuration.Parameters.FindRootCausesParameters.Clone().Extend(#{\"Filter\": _query.Configuration.Filter}))",
"Parameters": {
"FindRootCausesParameters": {
"CaseAttributeTypes": ["Account Manager","Customer Group","Product Group","Region"],
"Selection": {"Items":[{"Type":"IncludeCases","Items":[{"Type":"CaseAttributeValue","Attribute":"Product Group","StringifiedValues":["0Hats"]}]}]},
"MaxNumUniqueValues": 2,
"MaximumRowCount": 1000,
"MinValueUsage": 0.20,
"WeightingExpression": "Column(\"Cost\")"
}
},
"Ordering": [
{"Name": "Contribution", "Direction": "Descending"}
]
}

</syntaxhighlight>

== Utils.GetSampledEvents ==
Returns a [[SqlDataFrame in Expression Language|SqlDataFrame]] containing sampled events of given model where given filter is first applied.

=== Parameters ===

* '''sourceModel''':
** ProcessAnalyzer model object of the model whose event data is to be filtered and sampled.
* '''sampledCaseCount''':
** The maximum number of cases to return (or null if all cases should be returned).
* '''filter''':
** JSON filter to be applied on the event data of the source model prior to performing the sampling.

=== Return value ===
[[SqlDataFrame in Expression Language|SqlDataFrame]] containing sampled events of given model where given filter is first applied.

=== Example ===
Get a sample of all the events of at most 1000 cases having "Hats" as "Product Group" case attribute value from model identified by modelId.<syntaxhighlight lang="typescript">
let eventDataSampleSdf = _system.Utils.GetSampledEvents(modelId, 1000, #{
"Items":[#{
"Type":"IncludeCases",
"Items":[#{
"Type":"CaseAttributeValue",
"Attribute":"Product Group",
"StringifiedValues":["0Hats"]
}]
}]
});
</syntaxhighlight>

== Utils.ModifyColumnTypes ==
In-place modifies the column types of given columns in given data table.

=== Parameters ===
* '''dataTable''':
** ProcessAnalyzer model object of the model whose event data is to be filtered and sampled.
* '''columnTypesToSet''':
** Array of column name/type definitions to set.
*** Only columns that are to be changed are required to be listed.
*** Columns that don't exist in the data table will be skipped.

=== Return value ===
Returns the modified data table object.

=== Example ===
Modify column "CaseId" to be of type string.<syntaxhighlight lang="typescript">
_system.Utils.ModifyColumnTypes(
eventsTable,
#{#{"Name": "CaseId", "DataType": "String"}}
);

</syntaxhighlight>

== Utils.RunFunctionWithParallelLogging ==
Runs given function that generates logging information into given data table in a way that all the logging will be included into the generated script run log as well (if run inside a script).

Internally polls the table every 5 seconds for new rows and adds all the newly added rows to script log.

=== Parameters ===

* '''logTable''':
** A DataTable used for logging.
* '''callbackFunc''':
** Function that uses given data table for logging it's current status.

=== Return value ===
The result of the callback function.

=== Example ===
Run given stored procedure that generates new rows to log table identified by logTableId and log the generated rows into the script run log.<syntaxhighlight lang="typescript">
_system.Utils.RunFunctionWithParallelLogging(DataTableById(logTableId), () => {
CreateSnowflakeConnection().CallStoredProcedure("StoredProcedureTest", #{})
});

</syntaxhighlight>

System Library

2024-11-19T13:24:49Z

MarHink: /* Utils.RunFunctionWithParallelLogging */

System library is a collection of Expression Language functions and properties that provide additional tools for scripting Process Analyzer functionalities. System library is referenced in scripts via ''_system''-property, which provides additional properties dedicated for different areas of interests for scripting.

The following hierarchy shows the properties and functions available in System Library:

* ML
** [[Create Predicted Eventlog|GeneratePredictionModel]]
** [[Create Simulated Eventlog|ApplyTransformations]]
* Parallel
** Run
* RootCauses
** FindRootCausesDataFrame
* Utils
** GetSampledEvents
** ModifyColumnTypes
** RunFunctionWithParallelLogging

== Parallel.Run ==
Runs given functions in parallel.

=== Parameters ===
* '''functions''':
** An array of functions to run in parallel.

=== Return value ===
An array of results returned by the called functions, in the same order as the function generating them in the ''functions''-parameter.

=== Example ===
The following script uses _system.Parallel.Run to run three functions:

* SAP-extraction from VBAK-table in SAP (connection parameters defined in connectionParametersDict-dictionary).
* Transform the extracted data by adding a new column.
* Load the data into data table identified by dataTableId.
<syntaxhighlight lang="typescript">
function ExtractTransformAndLoad(extractFunc, transformFunc, loadFunc)
{
let rawDataFlow = extractFunc();
let transformedDataFlow = ToDataFlow();

_system.Parallel.Run([
() => Catch({
let df;
while (!IsNullTop(df = rawDataFlow.Collect(#{"CollectChunk": true}))) {
transformedDataFlow.Append(transformFunc(df));
WriteLog(`A chunk having ${df.NRows} rows has been transformed.`);
}
if (rawDataFlow.HasError) {
transformedDataFlow.Fail("Error occurred during data extraction.");
}
else {
transformedDataFlow.Complete();
}
}, {
transformedDataFlow.Fail("Error occurred during transformation calculation.");
}),
() => {
loadFunc(transformedDataFlow);
}
]);
}

ExtractTransformAndLoad(
() => ExtractSap(connectionParametersDict.Extend(
[
"FieldNames": "VBELN,ERDAT,ERZET,ERNAM,NETWR,WAERK",
"QueryTable": "VBAK",
"Options": ["VBELN BETWEEN '0000017448'" ,"AND '0060000042'"],
"UseGateway": true
])
),
df => df.SetColumns(["Test": () => `${Column("NETWR")} ${Column("WAERK")}`]),
dataFlow => {
DataTableById(dataTableId).Import(dataFlow, ["Append": 0]);
}
);

DataTableById(dataTableId).SqlDataFrame.OrderByColumns(["VBELN"], [true]).Collect()
</syntaxhighlight>

== RootCauses.FindForDataFrame ==
Finds root causes for a particular process phenomenon by comparing properties of selected cases against those of all cases in given model.

Based on the similar in-memory function: [[FindRootCauses Function|EventLog.FindRootCauses]].

=== Parameters ===

* '''model''':
** Model object of a model whose data tables are used to calculate the root causes.
* '''parameters''':
** A parameter convertible to a StringDictionary object with the following supported key-values:
*** '''Filter''':
**** Filter json (#30921#) that is applied to the event log before calculating root causes.
*** '''Selection''':
**** Selection json (#30927#) that defines selected cases to find root causes for. Is applied on top of the filtered event log (specified by Filter parameter).
**** If nothing is selected, 100% of cases are counted as selected.
*** '''CaseAttributeTypes''':
**** An array of strings with the names of case attributes included into the root causes.
**** Only case attributes of type string, integer or boolean are included.
**** If CaseAttributeTypes is null or empty string or not specified, all case attributes having type "String" are included into the root causes.
**** If the model doesn't have a case attribute with the specified name, an error message is shown.
*** '''EventAttributeTypes''':
**** An array of strings with the names of event attributes included into the root causes.
**** Only event attributes of type string are included.
**** If EventAttributeTypes is null, all event attributes having type "String" are included into the root causes.
**** If EventAttributeTypes is empty array or not specified, event attributes not applied to the root causes.
**** If the model doesn't have an event attribute with the specified name, an error message is shown.
**** Analysis column Name should contain Event Attribute Name.
**** Analysis column Value should contain Event Attribute Value and number of occurrences in case: <value> (count).
**** Analysis column Type should have "EventAttributeValue" as its value.
*** '''WeightingExpression''':
**** Expression providing weights for each case.
**** Expression, if defined, must be any of the following types:
***** A string containing the [[SQL Expressions|SqlExpression]] to evaluate.
***** A SqlExpression object, created e.g., using ToSqlExpression-function (see also: [[QPR_ProcessAnalyzer_Expressions#In-memory_expression_blocks_in_SQL_expressions|In-memory expression blocks in SQL expressions]]).
**** A row is filtered out of result if expression result is null.
*** '''MaximumRowCount''':
**** The maximum number of the most and least contributing root causes to return. Thus, the actual number of returned rows can be at most two times this value (if specified).
**** If undefined, 200 is used.
**** If set to 0, all rows are returned.
*** '''MinValueUsage''':
**** The minimum total usage of a value included into the comparison. The number of cases having every returned value should be at least given percentage (a float value between 0.0 and 1.0) of all the compared cases.
***** If not defined or null, all values are included (=default).
*** '''MaxNumUniqueValues''':
**** Maximum number of unique values to include into the comparison for each attribute column. If the amount of unique values for any attribute exceeds this value, only given number of attributes are included that have the highest usage.
**** If not defined or null, all values are included (=default).
*** '''IncludeOthers''':
**** Should the rest of the attribute values not included due to MinValueUsage or MaxNumUniqueValues filtering be included as an aggregated "Others" value?
**** If given any string value, that value is used as the value for all the aggregated values.
**** Default = undefined => others values will not be included into the results.
*** '''ValueIfNull''': Value used to indicate null-values.
**** Default = "(blank)".
**** Must be not null.

=== Return value ===
Returns a [[SqlDataFrame in Expression Language|SqlDataFrame]] object with the following columns:

* Common columns:
** '''Type''':
*** Type of the root cause:
**** "CaseAttributeValue" for case attributes.
**** "EventAttributeValue" for event attributes.
** '''Name''':
*** When type is CaseAttributeValue, case attribute name.
*** When type is EventAttributeValue, event attribute name.
** '''Value''':
*** When type is CaseAttributeValue, case attribute value.
*** When type is EventAttributeValue, event attribute value and number of occurrences in case.
** '''Total''':
*** The total number of cases having the found root cause.
** '''Selected''':
*** The number of cases that have the found root cause and belong to the selected cases.
** '''Compared''':
*** The number of cases that have the found root cause and don't belong to the selected cases.
** Columns when WeightingExpression not have value:
*** '''Contribution''':
**** The number of cases which contribute to the deviation from the average percentage of selected cases among all analyzed cases.
*** '''ContributionPercentage''':
**** The percent of cases which contribute to the deviation from the average percentage.
*** '''DifferencePercentage''':
**** The deviation in percentage between selected cases with the found root cause and the average percentage of selected cases among all analyzed cases.
*** '''SelectedPercentage''':
**** The percent of selected cases that have the found root cause out of all cases with that root cause.
** Columns when WeightingExpression have value:
*** '''Contribution''':
**** The sum of case weights which contribute to the deviation from the average percentage of selected case weights among all analyzed cases.
*** '''ContributionPercentage''':
**** The percent of case weights which contribute to the deviation from the average percentage.
*** '''DifferencePercentage''':
**** The deviation in percentage between selected case weights with the found root cause and the average percentage of selected case weights among all analyzed cases.
*** '''SelectedPercentage''':
**** The percent of selected case weights that have the found root cause out of all case weights with that root cause.
*** '''SelectedWeight''':
**** The sum of weights that have the found root cause and belong to the selected cases.
*** '''ComparedWeight''':
**** The sum of weights that have the found root cause and don't belong to the selected cases.
*** '''TotalWeight''':
**** The sum of weights of all cases with that root cause.

=== Example ===
Calculate root cause analysis for given model using parameters read from the [[Web API: Expression/query|query configuration]].<syntaxhighlight lang="json">
{
"ProcessingMethod": "DataFrame",
"Root":"let m = _; _system.RootCauses.FindForDataFrame(m, _query.Configuration.Parameters.FindRootCausesParameters.Clone().Extend(#{\"Filter\": _query.Configuration.Filter}))",
"Parameters": {
"FindRootCausesParameters": {
"CaseAttributeTypes": ["Account Manager","Customer Group","Product Group","Region"],
"Selection": {"Items":[{"Type":"IncludeCases","Items":[{"Type":"CaseAttributeValue","Attribute":"Product Group","StringifiedValues":["0Hats"]}]}]},
"MaxNumUniqueValues": 2,
"MaximumRowCount": 1000,
"MinValueUsage": 0.20,
"WeightingExpression": "Column(\"Cost\")"
}
},
"Ordering": [
{"Name": "Contribution", "Direction": "Descending"}
]
}

</syntaxhighlight>

== Utils.GetSampledEvents ==
Returns a [[SqlDataFrame in Expression Language|SqlDataFrame]] containing sampled events of given model where given filter is first applied.

=== Parameters ===

* '''sourceModel''':
** ProcessAnalyzer model object of the model whose event data is to be filtered and sampled.
* '''sampledCaseCount''':
** The maximum number of cases to return (or null if all cases should be returned).
* '''filter''':
** JSON filter to be applied on the event data of the source model prior to performing the sampling.

=== Return value ===
[[SqlDataFrame in Expression Language|SqlDataFrame]] containing sampled events of given model where given filter is first applied.

=== Example ===
Get a sample of all the events of at most 1000 cases having "Hats" as "Product Group" case attribute value from model identified by modelId.<syntaxhighlight lang="typescript">
let eventDataSampleSdf = _system.Utils.GetSampledEvents(modelId, 1000, #{
"Items":[#{
"Type":"IncludeCases",
"Items":[#{
"Type":"CaseAttributeValue",
"Attribute":"Product Group",
"StringifiedValues":["0Hats"]
}]
}]
});
</syntaxhighlight>

== Utils.ModifyColumnTypes ==
In-place modifies the column types of given columns in given data table.

=== Parameters ===
* '''dataTable''':
** ProcessAnalyzer model object of the model whose event data is to be filtered and sampled.
* '''columnTypesToSet''':
** Array of column name/type definitions to set.
*** Only columns that are to be changed are required to be listed.
*** Columns that don't exist in the data table will be skipped.

=== Return value ===
Returns the modified data table object.

=== Example ===
Modify column "CaseId" to be of type string.<syntaxhighlight lang="typescript">
_system.Utils.ModifyColumnTypes(
eventsTable,
#{#{"Name": "CaseId", "DataType": "String"}}
);

</syntaxhighlight>

== Utils.RunFunctionWithParallelLogging ==
Runs given function that generates logging information into given data table in a way that all the logging will be included into the generated script run log as well (if run inside a script).

Internally polls the table every 5 seconds for new rows and adds all the newly added rows to script log.

=== Parameters ===

* '''logTable''':
** A DataTable used for logging.
* '''callbackFunc''':
** Function that uses given data table for logging it's current status.

=== Return value ===
The result of the callback function.

=== Example ===
Run given stored procedure that generates new rows to log table identified by logTableId and log the generated rows into the script run log.<syntaxhighlight lang="typescript">
_system.Utils.RunFunctionWithParallelLogging(DataTableById(logTableId), () => {
CreateSnowflakeConnection().CallStoredProcedure("StoredProcedureTest", #{})
});

</syntaxhighlight>

System Library

2024-11-19T13:24:19Z

MarHink: /* Utils.ModifyColumnTypes */

System library is a collection of Expression Language functions and properties that provide additional tools for scripting Process Analyzer functionalities. System library is referenced in scripts via ''_system''-property, which provides additional properties dedicated for different areas of interests for scripting.

The following hierarchy shows the properties and functions available in System Library:

* ML
** [[Create Predicted Eventlog|GeneratePredictionModel]]
** [[Create Simulated Eventlog|ApplyTransformations]]
* Parallel
** Run
* RootCauses
** FindRootCausesDataFrame
* Utils
** GetSampledEvents
** ModifyColumnTypes
** RunFunctionWithParallelLogging

== Parallel.Run ==
Runs given functions in parallel.

=== Parameters ===
* '''functions''':
** An array of functions to run in parallel.

=== Return value ===
An array of results returned by the called functions, in the same order as the function generating them in the ''functions''-parameter.

=== Example ===
The following script uses _system.Parallel.Run to run three functions:

* SAP-extraction from VBAK-table in SAP (connection parameters defined in connectionParametersDict-dictionary).
* Transform the extracted data by adding a new column.
* Load the data into data table identified by dataTableId.
<syntaxhighlight lang="typescript">
function ExtractTransformAndLoad(extractFunc, transformFunc, loadFunc)
{
let rawDataFlow = extractFunc();
let transformedDataFlow = ToDataFlow();

_system.Parallel.Run([
() => Catch({
let df;
while (!IsNullTop(df = rawDataFlow.Collect(#{"CollectChunk": true}))) {
transformedDataFlow.Append(transformFunc(df));
WriteLog(`A chunk having ${df.NRows} rows has been transformed.`);
}
if (rawDataFlow.HasError) {
transformedDataFlow.Fail("Error occurred during data extraction.");
}
else {
transformedDataFlow.Complete();
}
}, {
transformedDataFlow.Fail("Error occurred during transformation calculation.");
}),
() => {
loadFunc(transformedDataFlow);
}
]);
}

ExtractTransformAndLoad(
() => ExtractSap(connectionParametersDict.Extend(
[
"FieldNames": "VBELN,ERDAT,ERZET,ERNAM,NETWR,WAERK",
"QueryTable": "VBAK",
"Options": ["VBELN BETWEEN '0000017448'" ,"AND '0060000042'"],
"UseGateway": true
])
),
df => df.SetColumns(["Test": () => `${Column("NETWR")} ${Column("WAERK")}`]),
dataFlow => {
DataTableById(dataTableId).Import(dataFlow, ["Append": 0]);
}
);

DataTableById(dataTableId).SqlDataFrame.OrderByColumns(["VBELN"], [true]).Collect()
</syntaxhighlight>

== RootCauses.FindForDataFrame ==
Finds root causes for a particular process phenomenon by comparing properties of selected cases against those of all cases in given model.

Based on the similar in-memory function: [[FindRootCauses Function|EventLog.FindRootCauses]].

=== Parameters ===

* '''model''':
** Model object of a model whose data tables are used to calculate the root causes.
* '''parameters''':
** A parameter convertible to a StringDictionary object with the following supported key-values:
*** '''Filter''':
**** Filter json (#30921#) that is applied to the event log before calculating root causes.
*** '''Selection''':
**** Selection json (#30927#) that defines selected cases to find root causes for. Is applied on top of the filtered event log (specified by Filter parameter).
**** If nothing is selected, 100% of cases are counted as selected.
*** '''CaseAttributeTypes''':
**** An array of strings with the names of case attributes included into the root causes.
**** Only case attributes of type string, integer or boolean are included.
**** If CaseAttributeTypes is null or empty string or not specified, all case attributes having type "String" are included into the root causes.
**** If the model doesn't have a case attribute with the specified name, an error message is shown.
*** '''EventAttributeTypes''':
**** An array of strings with the names of event attributes included into the root causes.
**** Only event attributes of type string are included.
**** If EventAttributeTypes is null, all event attributes having type "String" are included into the root causes.
**** If EventAttributeTypes is empty array or not specified, event attributes not applied to the root causes.
**** If the model doesn't have an event attribute with the specified name, an error message is shown.
**** Analysis column Name should contain Event Attribute Name.
**** Analysis column Value should contain Event Attribute Value and number of occurrences in case: <value> (count).
**** Analysis column Type should have "EventAttributeValue" as its value.
*** '''WeightingExpression''':
**** Expression providing weights for each case.
**** Expression, if defined, must be any of the following types:
***** A string containing the [[SQL Expressions|SqlExpression]] to evaluate.
***** A SqlExpression object, created e.g., using ToSqlExpression-function (see also: [[QPR_ProcessAnalyzer_Expressions#In-memory_expression_blocks_in_SQL_expressions|In-memory expression blocks in SQL expressions]]).
**** A row is filtered out of result if expression result is null.
*** '''MaximumRowCount''':
**** The maximum number of the most and least contributing root causes to return. Thus, the actual number of returned rows can be at most two times this value (if specified).
**** If undefined, 200 is used.
**** If set to 0, all rows are returned.
*** '''MinValueUsage''':
**** The minimum total usage of a value included into the comparison. The number of cases having every returned value should be at least given percentage (a float value between 0.0 and 1.0) of all the compared cases.
***** If not defined or null, all values are included (=default).
*** '''MaxNumUniqueValues''':
**** Maximum number of unique values to include into the comparison for each attribute column. If the amount of unique values for any attribute exceeds this value, only given number of attributes are included that have the highest usage.
**** If not defined or null, all values are included (=default).
*** '''IncludeOthers''':
**** Should the rest of the attribute values not included due to MinValueUsage or MaxNumUniqueValues filtering be included as an aggregated "Others" value?
**** If given any string value, that value is used as the value for all the aggregated values.
**** Default = undefined => others values will not be included into the results.
*** '''ValueIfNull''': Value used to indicate null-values.
**** Default = "(blank)".
**** Must be not null.

=== Return value ===
Returns a [[SqlDataFrame in Expression Language|SqlDataFrame]] object with the following columns:

* Common columns:
** '''Type''':
*** Type of the root cause:
**** "CaseAttributeValue" for case attributes.
**** "EventAttributeValue" for event attributes.
** '''Name''':
*** When type is CaseAttributeValue, case attribute name.
*** When type is EventAttributeValue, event attribute name.
** '''Value''':
*** When type is CaseAttributeValue, case attribute value.
*** When type is EventAttributeValue, event attribute value and number of occurrences in case.
** '''Total''':
*** The total number of cases having the found root cause.
** '''Selected''':
*** The number of cases that have the found root cause and belong to the selected cases.
** '''Compared''':
*** The number of cases that have the found root cause and don't belong to the selected cases.
** Columns when WeightingExpression not have value:
*** '''Contribution''':
**** The number of cases which contribute to the deviation from the average percentage of selected cases among all analyzed cases.
*** '''ContributionPercentage''':
**** The percent of cases which contribute to the deviation from the average percentage.
*** '''DifferencePercentage''':
**** The deviation in percentage between selected cases with the found root cause and the average percentage of selected cases among all analyzed cases.
*** '''SelectedPercentage''':
**** The percent of selected cases that have the found root cause out of all cases with that root cause.
** Columns when WeightingExpression have value:
*** '''Contribution''':
**** The sum of case weights which contribute to the deviation from the average percentage of selected case weights among all analyzed cases.
*** '''ContributionPercentage''':
**** The percent of case weights which contribute to the deviation from the average percentage.
*** '''DifferencePercentage''':
**** The deviation in percentage between selected case weights with the found root cause and the average percentage of selected case weights among all analyzed cases.
*** '''SelectedPercentage''':
**** The percent of selected case weights that have the found root cause out of all case weights with that root cause.
*** '''SelectedWeight''':
**** The sum of weights that have the found root cause and belong to the selected cases.
*** '''ComparedWeight''':
**** The sum of weights that have the found root cause and don't belong to the selected cases.
*** '''TotalWeight''':
**** The sum of weights of all cases with that root cause.

=== Example ===
Calculate root cause analysis for given model using parameters read from the [[Web API: Expression/query|query configuration]].<syntaxhighlight lang="json">
{
"ProcessingMethod": "DataFrame",
"Root":"let m = _; _system.RootCauses.FindForDataFrame(m, _query.Configuration.Parameters.FindRootCausesParameters.Clone().Extend(#{\"Filter\": _query.Configuration.Filter}))",
"Parameters": {
"FindRootCausesParameters": {
"CaseAttributeTypes": ["Account Manager","Customer Group","Product Group","Region"],
"Selection": {"Items":[{"Type":"IncludeCases","Items":[{"Type":"CaseAttributeValue","Attribute":"Product Group","StringifiedValues":["0Hats"]}]}]},
"MaxNumUniqueValues": 2,
"MaximumRowCount": 1000,
"MinValueUsage": 0.20,
"WeightingExpression": "Column(\"Cost\")"
}
},
"Ordering": [
{"Name": "Contribution", "Direction": "Descending"}
]
}

</syntaxhighlight>

== Utils.GetSampledEvents ==
Returns a [[SqlDataFrame in Expression Language|SqlDataFrame]] containing sampled events of given model where given filter is first applied.

=== Parameters ===

* '''sourceModel''':
** ProcessAnalyzer model object of the model whose event data is to be filtered and sampled.
* '''sampledCaseCount''':
** The maximum number of cases to return (or null if all cases should be returned).
* '''filter''':
** JSON filter to be applied on the event data of the source model prior to performing the sampling.

=== Return value ===
[[SqlDataFrame in Expression Language|SqlDataFrame]] containing sampled events of given model where given filter is first applied.

=== Example ===
Get a sample of all the events of at most 1000 cases having "Hats" as "Product Group" case attribute value from model identified by modelId.<syntaxhighlight lang="typescript">
let eventDataSampleSdf = _system.Utils.GetSampledEvents(modelId, 1000, #{
"Items":[#{
"Type":"IncludeCases",
"Items":[#{
"Type":"CaseAttributeValue",
"Attribute":"Product Group",
"StringifiedValues":["0Hats"]
}]
}]
});
</syntaxhighlight>

== Utils.ModifyColumnTypes ==
In-place modifies the column types of given columns in given data table.

=== Parameters ===
* '''dataTable''':
** ProcessAnalyzer model object of the model whose event data is to be filtered and sampled.
* '''columnTypesToSet''':
** Array of column name/type definitions to set.
*** Only columns that are to be changed are required to be listed.
*** Columns that don't exist in the data table will be skipped.

=== Return value ===
Returns the modified data table object.

=== Example ===
Modify column "CaseId" to be of type string.<syntaxhighlight lang="typescript">
_system.Utils.ModifyColumnTypes(
eventsTable,
#{#{"Name": "CaseId", "DataType": "String"}}
);

</syntaxhighlight>

== Utils.RunFunctionWithParallelLogging ==
Runs given function that generates logging information into given data table in a way that all the logging will be included into the generated script run log as well (if run inside a script).

Internally polls the table every 5 seconds for new rows and adds all the newly added rows to script log.

=== Parameters ===

* logTable:
** A DataTable used for logging.
* callbackFunc:
** Function that uses given data table for logging it's current status.

=== Return value ===
The result of the callback function.

=== Example ===
Run given stored procedure that generates new rows to log table identified by logTableId and log the generated rows into the script run log.<syntaxhighlight lang="typescript">
_system.Utils.RunFunctionWithParallelLogging(DataTableById(logTableId), () => {
CreateSnowflakeConnection().CallStoredProcedure("StoredProcedureTest", #{})
});

</syntaxhighlight>

System Library

2024-11-19T13:05:26Z

MarHink: /* Utils.ModifyColumnTypes */

System library is a collection of Expression Language functions and properties that provide additional tools for scripting Process Analyzer functionalities. System library is referenced in scripts via ''_system''-property, which provides additional properties dedicated for different areas of interests for scripting.

The following hierarchy shows the properties and functions available in System Library:

* ML
** [[Create Predicted Eventlog|GeneratePredictionModel]]
** [[Create Simulated Eventlog|ApplyTransformations]]
* Parallel
** Run
* RootCauses
** FindRootCausesDataFrame
* Utils
** GetSampledEvents
** ModifyColumnTypes
** RunFunctionWithParallelLogging

== Parallel.Run ==
Runs given functions in parallel.

=== Parameters ===
* '''functions''':
** An array of functions to run in parallel.

=== Return value ===
An array of results returned by the called functions, in the same order as the function generating them in the ''functions''-parameter.

=== Example ===
The following script uses _system.Parallel.Run to run three functions:

* SAP-extraction from VBAK-table in SAP (connection parameters defined in connectionParametersDict-dictionary).
* Transform the extracted data by adding a new column.
* Load the data into data table identified by dataTableId.
<syntaxhighlight lang="typescript">
function ExtractTransformAndLoad(extractFunc, transformFunc, loadFunc)
{
let rawDataFlow = extractFunc();
let transformedDataFlow = ToDataFlow();

_system.Parallel.Run([
() => Catch({
let df;
while (!IsNullTop(df = rawDataFlow.Collect(#{"CollectChunk": true}))) {
transformedDataFlow.Append(transformFunc(df));
WriteLog(`A chunk having ${df.NRows} rows has been transformed.`);
}
if (rawDataFlow.HasError) {
transformedDataFlow.Fail("Error occurred during data extraction.");
}
else {
transformedDataFlow.Complete();
}
}, {
transformedDataFlow.Fail("Error occurred during transformation calculation.");
}),
() => {
loadFunc(transformedDataFlow);
}
]);
}

ExtractTransformAndLoad(
() => ExtractSap(connectionParametersDict.Extend(
[
"FieldNames": "VBELN,ERDAT,ERZET,ERNAM,NETWR,WAERK",
"QueryTable": "VBAK",
"Options": ["VBELN BETWEEN '0000017448'" ,"AND '0060000042'"],
"UseGateway": true
])
),
df => df.SetColumns(["Test": () => `${Column("NETWR")} ${Column("WAERK")}`]),
dataFlow => {
DataTableById(dataTableId).Import(dataFlow, ["Append": 0]);
}
);

DataTableById(dataTableId).SqlDataFrame.OrderByColumns(["VBELN"], [true]).Collect()
</syntaxhighlight>

== RootCauses.FindForDataFrame ==
Finds root causes for a particular process phenomenon by comparing properties of selected cases against those of all cases in given model.

Based on the similar in-memory function: [[FindRootCauses Function|EventLog.FindRootCauses]].

=== Parameters ===

* '''model''':
** Model object of a model whose data tables are used to calculate the root causes.
* '''parameters''':
** A parameter convertible to a StringDictionary object with the following supported key-values:
*** '''Filter''':
**** Filter json (#30921#) that is applied to the event log before calculating root causes.
*** '''Selection''':
**** Selection json (#30927#) that defines selected cases to find root causes for. Is applied on top of the filtered event log (specified by Filter parameter).
**** If nothing is selected, 100% of cases are counted as selected.
*** '''CaseAttributeTypes''':
**** An array of strings with the names of case attributes included into the root causes.
**** Only case attributes of type string, integer or boolean are included.
**** If CaseAttributeTypes is null or empty string or not specified, all case attributes having type "String" are included into the root causes.
**** If the model doesn't have a case attribute with the specified name, an error message is shown.
*** '''EventAttributeTypes''':
**** An array of strings with the names of event attributes included into the root causes.
**** Only event attributes of type string are included.
**** If EventAttributeTypes is null, all event attributes having type "String" are included into the root causes.
**** If EventAttributeTypes is empty array or not specified, event attributes not applied to the root causes.
**** If the model doesn't have an event attribute with the specified name, an error message is shown.
**** Analysis column Name should contain Event Attribute Name.
**** Analysis column Value should contain Event Attribute Value and number of occurrences in case: <value> (count).
**** Analysis column Type should have "EventAttributeValue" as its value.
*** '''WeightingExpression''':
**** Expression providing weights for each case.
**** Expression, if defined, must be any of the following types:
***** A string containing the [[SQL Expressions|SqlExpression]] to evaluate.
***** A SqlExpression object, created e.g., using ToSqlExpression-function (see also: [[QPR_ProcessAnalyzer_Expressions#In-memory_expression_blocks_in_SQL_expressions|In-memory expression blocks in SQL expressions]]).
**** A row is filtered out of result if expression result is null.
*** '''MaximumRowCount''':
**** The maximum number of the most and least contributing root causes to return. Thus, the actual number of returned rows can be at most two times this value (if specified).
**** If undefined, 200 is used.
**** If set to 0, all rows are returned.
*** '''MinValueUsage''':
**** The minimum total usage of a value included into the comparison. The number of cases having every returned value should be at least given percentage (a float value between 0.0 and 1.0) of all the compared cases.
***** If not defined or null, all values are included (=default).
*** '''MaxNumUniqueValues''':
**** Maximum number of unique values to include into the comparison for each attribute column. If the amount of unique values for any attribute exceeds this value, only given number of attributes are included that have the highest usage.
**** If not defined or null, all values are included (=default).
*** '''IncludeOthers''':
**** Should the rest of the attribute values not included due to MinValueUsage or MaxNumUniqueValues filtering be included as an aggregated "Others" value?
**** If given any string value, that value is used as the value for all the aggregated values.
**** Default = undefined => others values will not be included into the results.
*** '''ValueIfNull''': Value used to indicate null-values.
**** Default = "(blank)".
**** Must be not null.

=== Return value ===
Returns a [[SqlDataFrame in Expression Language|SqlDataFrame]] object with the following columns:

* Common columns:
** '''Type''':
*** Type of the root cause:
**** "CaseAttributeValue" for case attributes.
**** "EventAttributeValue" for event attributes.
** '''Name''':
*** When type is CaseAttributeValue, case attribute name.
*** When type is EventAttributeValue, event attribute name.
** '''Value''':
*** When type is CaseAttributeValue, case attribute value.
*** When type is EventAttributeValue, event attribute value and number of occurrences in case.
** '''Total''':
*** The total number of cases having the found root cause.
** '''Selected''':
*** The number of cases that have the found root cause and belong to the selected cases.
** '''Compared''':
*** The number of cases that have the found root cause and don't belong to the selected cases.
** Columns when WeightingExpression not have value:
*** '''Contribution''':
**** The number of cases which contribute to the deviation from the average percentage of selected cases among all analyzed cases.
*** '''ContributionPercentage''':
**** The percent of cases which contribute to the deviation from the average percentage.
*** '''DifferencePercentage''':
**** The deviation in percentage between selected cases with the found root cause and the average percentage of selected cases among all analyzed cases.
*** '''SelectedPercentage''':
**** The percent of selected cases that have the found root cause out of all cases with that root cause.
** Columns when WeightingExpression have value:
*** '''Contribution''':
**** The sum of case weights which contribute to the deviation from the average percentage of selected case weights among all analyzed cases.
*** '''ContributionPercentage''':
**** The percent of case weights which contribute to the deviation from the average percentage.
*** '''DifferencePercentage''':
**** The deviation in percentage between selected case weights with the found root cause and the average percentage of selected case weights among all analyzed cases.
*** '''SelectedPercentage''':
**** The percent of selected case weights that have the found root cause out of all case weights with that root cause.
*** '''SelectedWeight''':
**** The sum of weights that have the found root cause and belong to the selected cases.
*** '''ComparedWeight''':
**** The sum of weights that have the found root cause and don't belong to the selected cases.
*** '''TotalWeight''':
**** The sum of weights of all cases with that root cause.

=== Example ===
Calculate root cause analysis for given model using parameters read from the [[Web API: Expression/query|query configuration]].<syntaxhighlight lang="json">
{
"ProcessingMethod": "DataFrame",
"Root":"let m = _; _system.RootCauses.FindForDataFrame(m, _query.Configuration.Parameters.FindRootCausesParameters.Clone().Extend(#{\"Filter\": _query.Configuration.Filter}))",
"Parameters": {
"FindRootCausesParameters": {
"CaseAttributeTypes": ["Account Manager","Customer Group","Product Group","Region"],
"Selection": {"Items":[{"Type":"IncludeCases","Items":[{"Type":"CaseAttributeValue","Attribute":"Product Group","StringifiedValues":["0Hats"]}]}]},
"MaxNumUniqueValues": 2,
"MaximumRowCount": 1000,
"MinValueUsage": 0.20,
"WeightingExpression": "Column(\"Cost\")"
}
},
"Ordering": [
{"Name": "Contribution", "Direction": "Descending"}
]
}

</syntaxhighlight>

== Utils.GetSampledEvents ==
Returns a [[SqlDataFrame in Expression Language|SqlDataFrame]] containing sampled events of given model where given filter is first applied.

=== Parameters ===

* '''sourceModel''':
** ProcessAnalyzer model object of the model whose event data is to be filtered and sampled.
* '''sampledCaseCount''':
** The maximum number of cases to return (or null if all cases should be returned).
* '''filter''':
** JSON filter to be applied on the event data of the source model prior to performing the sampling.

=== Return value ===
[[SqlDataFrame in Expression Language|SqlDataFrame]] containing sampled events of given model where given filter is first applied.

=== Example ===
Get a sample of all the events of at most 1000 cases having "Hats" as "Product Group" case attribute value from model identified by modelId.<syntaxhighlight lang="typescript">
let eventDataSampleSdf = _system.Utils.GetSampledEvents(modelId, 1000, #{
"Items":[#{
"Type":"IncludeCases",
"Items":[#{
"Type":"CaseAttributeValue",
"Attribute":"Product Group",
"StringifiedValues":["0Hats"]
}]
}]
});
</syntaxhighlight>

== Utils.ModifyColumnTypes ==
In-place modifies the column types of given columns in given data table.

=== Parameters ===
* '''dataTable''':
** ProcessAnalyzer model object of the model whose event data is to be filtered and sampled.
* '''columnTypesToSet''':
** Array of column name/type definitions to set (not all columns in the data table need to be listed).

=== Return value ===
Returns the modified data table object.

=== Example ===
Modify column "CaseId" to be of type string.<syntaxhighlight lang="typescript">
_system.Utils.ModifyColumnTypes(
eventsTable,
#{#{"Name": "CaseId", "DataType": "String"}}
);

</syntaxhighlight>

== Utils.RunFunctionWithParallelLogging ==
foo

System Library

2024-11-19T12:55:58Z

MarHink: /* Utils.GetSampledEvents */

System library is a collection of Expression Language functions and properties that provide additional tools for scripting Process Analyzer functionalities. System library is referenced in scripts via ''_system''-property, which provides additional properties dedicated for different areas of interests for scripting.

The following hierarchy shows the properties and functions available in System Library:

* ML
** [[Create Predicted Eventlog|GeneratePredictionModel]]
** [[Create Simulated Eventlog|ApplyTransformations]]
* Parallel
** Run
* RootCauses
** FindRootCausesDataFrame
* Utils
** GetSampledEvents
** ModifyColumnTypes
** RunFunctionWithParallelLogging

== Parallel.Run ==
Runs given functions in parallel.

=== Parameters ===
* '''functions''':
** An array of functions to run in parallel.

=== Return value ===
An array of results returned by the called functions, in the same order as the function generating them in the ''functions''-parameter.

=== Example ===
The following script uses _system.Parallel.Run to run three functions:

* SAP-extraction from VBAK-table in SAP (connection parameters defined in connectionParametersDict-dictionary).
* Transform the extracted data by adding a new column.
* Load the data into data table identified by dataTableId.
<syntaxhighlight lang="typescript">
function ExtractTransformAndLoad(extractFunc, transformFunc, loadFunc)
{
let rawDataFlow = extractFunc();
let transformedDataFlow = ToDataFlow();

_system.Parallel.Run([
() => Catch({
let df;
while (!IsNullTop(df = rawDataFlow.Collect(#{"CollectChunk": true}))) {
transformedDataFlow.Append(transformFunc(df));
WriteLog(`A chunk having ${df.NRows} rows has been transformed.`);
}
if (rawDataFlow.HasError) {
transformedDataFlow.Fail("Error occurred during data extraction.");
}
else {
transformedDataFlow.Complete();
}
}, {
transformedDataFlow.Fail("Error occurred during transformation calculation.");
}),
() => {
loadFunc(transformedDataFlow);
}
]);
}

ExtractTransformAndLoad(
() => ExtractSap(connectionParametersDict.Extend(
[
"FieldNames": "VBELN,ERDAT,ERZET,ERNAM,NETWR,WAERK",
"QueryTable": "VBAK",
"Options": ["VBELN BETWEEN '0000017448'" ,"AND '0060000042'"],
"UseGateway": true
])
),
df => df.SetColumns(["Test": () => `${Column("NETWR")} ${Column("WAERK")}`]),
dataFlow => {
DataTableById(dataTableId).Import(dataFlow, ["Append": 0]);
}
);

DataTableById(dataTableId).SqlDataFrame.OrderByColumns(["VBELN"], [true]).Collect()
</syntaxhighlight>

== RootCauses.FindForDataFrame ==
Finds root causes for a particular process phenomenon by comparing properties of selected cases against those of all cases in given model.

Based on the similar in-memory function: [[FindRootCauses Function|EventLog.FindRootCauses]].

=== Parameters ===

* '''model''':
** Model object of a model whose data tables are used to calculate the root causes.
* '''parameters''':
** A parameter convertible to a StringDictionary object with the following supported key-values:
*** '''Filter''':
**** Filter json (#30921#) that is applied to the event log before calculating root causes.
*** '''Selection''':
**** Selection json (#30927#) that defines selected cases to find root causes for. Is applied on top of the filtered event log (specified by Filter parameter).
**** If nothing is selected, 100% of cases are counted as selected.
*** '''CaseAttributeTypes''':
**** An array of strings with the names of case attributes included into the root causes.
**** Only case attributes of type string, integer or boolean are included.
**** If CaseAttributeTypes is null or empty string or not specified, all case attributes having type "String" are included into the root causes.
**** If the model doesn't have a case attribute with the specified name, an error message is shown.
*** '''EventAttributeTypes''':
**** An array of strings with the names of event attributes included into the root causes.
**** Only event attributes of type string are included.
**** If EventAttributeTypes is null, all event attributes having type "String" are included into the root causes.
**** If EventAttributeTypes is empty array or not specified, event attributes not applied to the root causes.
**** If the model doesn't have an event attribute with the specified name, an error message is shown.
**** Analysis column Name should contain Event Attribute Name.
**** Analysis column Value should contain Event Attribute Value and number of occurrences in case: <value> (count).
**** Analysis column Type should have "EventAttributeValue" as its value.
*** '''WeightingExpression''':
**** Expression providing weights for each case.
**** Expression, if defined, must be any of the following types:
***** A string containing the [[SQL Expressions|SqlExpression]] to evaluate.
***** A SqlExpression object, created e.g., using ToSqlExpression-function (see also: [[QPR_ProcessAnalyzer_Expressions#In-memory_expression_blocks_in_SQL_expressions|In-memory expression blocks in SQL expressions]]).
**** A row is filtered out of result if expression result is null.
*** '''MaximumRowCount''':
**** The maximum number of the most and least contributing root causes to return. Thus, the actual number of returned rows can be at most two times this value (if specified).
**** If undefined, 200 is used.
**** If set to 0, all rows are returned.
*** '''MinValueUsage''':
**** The minimum total usage of a value included into the comparison. The number of cases having every returned value should be at least given percentage (a float value between 0.0 and 1.0) of all the compared cases.
***** If not defined or null, all values are included (=default).
*** '''MaxNumUniqueValues''':
**** Maximum number of unique values to include into the comparison for each attribute column. If the amount of unique values for any attribute exceeds this value, only given number of attributes are included that have the highest usage.
**** If not defined or null, all values are included (=default).
*** '''IncludeOthers''':
**** Should the rest of the attribute values not included due to MinValueUsage or MaxNumUniqueValues filtering be included as an aggregated "Others" value?
**** If given any string value, that value is used as the value for all the aggregated values.
**** Default = undefined => others values will not be included into the results.
*** '''ValueIfNull''': Value used to indicate null-values.
**** Default = "(blank)".
**** Must be not null.

=== Return value ===
Returns a [[SqlDataFrame in Expression Language|SqlDataFrame]] object with the following columns:

* Common columns:
** '''Type''':
*** Type of the root cause:
**** "CaseAttributeValue" for case attributes.
**** "EventAttributeValue" for event attributes.
** '''Name''':
*** When type is CaseAttributeValue, case attribute name.
*** When type is EventAttributeValue, event attribute name.
** '''Value''':
*** When type is CaseAttributeValue, case attribute value.
*** When type is EventAttributeValue, event attribute value and number of occurrences in case.
** '''Total''':
*** The total number of cases having the found root cause.
** '''Selected''':
*** The number of cases that have the found root cause and belong to the selected cases.
** '''Compared''':
*** The number of cases that have the found root cause and don't belong to the selected cases.
** Columns when WeightingExpression not have value:
*** '''Contribution''':
**** The number of cases which contribute to the deviation from the average percentage of selected cases among all analyzed cases.
*** '''ContributionPercentage''':
**** The percent of cases which contribute to the deviation from the average percentage.
*** '''DifferencePercentage''':
**** The deviation in percentage between selected cases with the found root cause and the average percentage of selected cases among all analyzed cases.
*** '''SelectedPercentage''':
**** The percent of selected cases that have the found root cause out of all cases with that root cause.
** Columns when WeightingExpression have value:
*** '''Contribution''':
**** The sum of case weights which contribute to the deviation from the average percentage of selected case weights among all analyzed cases.
*** '''ContributionPercentage''':
**** The percent of case weights which contribute to the deviation from the average percentage.
*** '''DifferencePercentage''':
**** The deviation in percentage between selected case weights with the found root cause and the average percentage of selected case weights among all analyzed cases.
*** '''SelectedPercentage''':
**** The percent of selected case weights that have the found root cause out of all case weights with that root cause.
*** '''SelectedWeight''':
**** The sum of weights that have the found root cause and belong to the selected cases.
*** '''ComparedWeight''':
**** The sum of weights that have the found root cause and don't belong to the selected cases.
*** '''TotalWeight''':
**** The sum of weights of all cases with that root cause.

=== Example ===
Calculate root cause analysis for given model using parameters read from the [[Web API: Expression/query|query configuration]].<syntaxhighlight lang="json">
{
"ProcessingMethod": "DataFrame",
"Root":"let m = _; _system.RootCauses.FindForDataFrame(m, _query.Configuration.Parameters.FindRootCausesParameters.Clone().Extend(#{\"Filter\": _query.Configuration.Filter}))",
"Parameters": {
"FindRootCausesParameters": {
"CaseAttributeTypes": ["Account Manager","Customer Group","Product Group","Region"],
"Selection": {"Items":[{"Type":"IncludeCases","Items":[{"Type":"CaseAttributeValue","Attribute":"Product Group","StringifiedValues":["0Hats"]}]}]},
"MaxNumUniqueValues": 2,
"MaximumRowCount": 1000,
"MinValueUsage": 0.20,
"WeightingExpression": "Column(\"Cost\")"
}
},
"Ordering": [
{"Name": "Contribution", "Direction": "Descending"}
]
}

</syntaxhighlight>

== Utils.GetSampledEvents ==
Returns a [[SqlDataFrame in Expression Language|SqlDataFrame]] containing sampled events of given model where given filter is first applied.

=== Parameters ===

* '''sourceModel''':
** ProcessAnalyzer model object of the model whose event data is to be filtered and sampled.
* '''sampledCaseCount''':
** The maximum number of cases to return (or null if all cases should be returned).
* '''filter''':
** JSON filter to be applied on the event data of the source model prior to performing the sampling.

=== Return value ===
[[SqlDataFrame in Expression Language|SqlDataFrame]] containing sampled events of given model where given filter is first applied.

=== Example ===
Get a sample of all the events of at most 1000 cases having "Hats" as "Product Group" case attribute value from model identified by modelId.<syntaxhighlight lang="typescript">
let eventDataSampleSdf = _system.Utils.GetSampledEvents(modelId, 1000, #{
"Items":[#{
"Type":"IncludeCases",
"Items":[#{
"Type":"CaseAttributeValue",
"Attribute":"Product Group",
"StringifiedValues":["0Hats"]
}]
}]
});
</syntaxhighlight>

== Utils.ModifyColumnTypes ==

== Utils.RunFunctionWithParallelLogging ==
foo

System Library

2024-11-19T12:48:02Z

MarHink:

System library is a collection of Expression Language functions and properties that provide additional tools for scripting Process Analyzer functionalities. System library is referenced in scripts via ''_system''-property, which provides additional properties dedicated for different areas of interests for scripting.

The following hierarchy shows the properties and functions available in System Library:

* ML
** [[Create Predicted Eventlog|GeneratePredictionModel]]
** [[Create Simulated Eventlog|ApplyTransformations]]
* Parallel
** Run
* RootCauses
** FindRootCausesDataFrame
* Utils
** GetSampledEvents
** ModifyColumnTypes
** RunFunctionWithParallelLogging

== Parallel.Run ==
Runs given functions in parallel.

=== Parameters ===
* '''functions''':
** An array of functions to run in parallel.

=== Return value ===
An array of results returned by the called functions, in the same order as the function generating them in the ''functions''-parameter.

=== Example ===
The following script uses _system.Parallel.Run to run three functions:

* SAP-extraction from VBAK-table in SAP (connection parameters defined in connectionParametersDict-dictionary).
* Transform the extracted data by adding a new column.
* Load the data into data table identified by dataTableId.
<syntaxhighlight lang="typescript">
function ExtractTransformAndLoad(extractFunc, transformFunc, loadFunc)
{
let rawDataFlow = extractFunc();
let transformedDataFlow = ToDataFlow();

_system.Parallel.Run([
() => Catch({
let df;
while (!IsNullTop(df = rawDataFlow.Collect(#{"CollectChunk": true}))) {
transformedDataFlow.Append(transformFunc(df));
WriteLog(`A chunk having ${df.NRows} rows has been transformed.`);
}
if (rawDataFlow.HasError) {
transformedDataFlow.Fail("Error occurred during data extraction.");
}
else {
transformedDataFlow.Complete();
}
}, {
transformedDataFlow.Fail("Error occurred during transformation calculation.");
}),
() => {
loadFunc(transformedDataFlow);
}
]);
}

ExtractTransformAndLoad(
() => ExtractSap(connectionParametersDict.Extend(
[
"FieldNames": "VBELN,ERDAT,ERZET,ERNAM,NETWR,WAERK",
"QueryTable": "VBAK",
"Options": ["VBELN BETWEEN '0000017448'" ,"AND '0060000042'"],
"UseGateway": true
])
),
df => df.SetColumns(["Test": () => `${Column("NETWR")} ${Column("WAERK")}`]),
dataFlow => {
DataTableById(dataTableId).Import(dataFlow, ["Append": 0]);
}
);

DataTableById(dataTableId).SqlDataFrame.OrderByColumns(["VBELN"], [true]).Collect()
</syntaxhighlight>

== RootCauses.FindForDataFrame ==
Finds root causes for a particular process phenomenon by comparing properties of selected cases against those of all cases in given model.

Based on the similar in-memory function: [[FindRootCauses Function|EventLog.FindRootCauses]].

=== Parameters ===

* '''model''':
** Model object of a model whose data tables are used to calculate the root causes.
* '''parameters''':
** A parameter convertible to a StringDictionary object with the following supported key-values:
*** '''Filter''':
**** Filter json (#30921#) that is applied to the event log before calculating root causes.
*** '''Selection''':
**** Selection json (#30927#) that defines selected cases to find root causes for. Is applied on top of the filtered event log (specified by Filter parameter).
**** If nothing is selected, 100% of cases are counted as selected.
*** '''CaseAttributeTypes''':
**** An array of strings with the names of case attributes included into the root causes.
**** Only case attributes of type string, integer or boolean are included.
**** If CaseAttributeTypes is null or empty string or not specified, all case attributes having type "String" are included into the root causes.
**** If the model doesn't have a case attribute with the specified name, an error message is shown.
*** '''EventAttributeTypes''':
**** An array of strings with the names of event attributes included into the root causes.
**** Only event attributes of type string are included.
**** If EventAttributeTypes is null, all event attributes having type "String" are included into the root causes.
**** If EventAttributeTypes is empty array or not specified, event attributes not applied to the root causes.
**** If the model doesn't have an event attribute with the specified name, an error message is shown.
**** Analysis column Name should contain Event Attribute Name.
**** Analysis column Value should contain Event Attribute Value and number of occurrences in case: <value> (count).
**** Analysis column Type should have "EventAttributeValue" as its value.
*** '''WeightingExpression''':
**** Expression providing weights for each case.
**** Expression, if defined, must be any of the following types:
***** A string containing the [[SQL Expressions|SqlExpression]] to evaluate.
***** A SqlExpression object, created e.g., using ToSqlExpression-function (see also: [[QPR_ProcessAnalyzer_Expressions#In-memory_expression_blocks_in_SQL_expressions|In-memory expression blocks in SQL expressions]]).
**** A row is filtered out of result if expression result is null.
*** '''MaximumRowCount''':
**** The maximum number of the most and least contributing root causes to return. Thus, the actual number of returned rows can be at most two times this value (if specified).
**** If undefined, 200 is used.
**** If set to 0, all rows are returned.
*** '''MinValueUsage''':
**** The minimum total usage of a value included into the comparison. The number of cases having every returned value should be at least given percentage (a float value between 0.0 and 1.0) of all the compared cases.
***** If not defined or null, all values are included (=default).
*** '''MaxNumUniqueValues''':
**** Maximum number of unique values to include into the comparison for each attribute column. If the amount of unique values for any attribute exceeds this value, only given number of attributes are included that have the highest usage.
**** If not defined or null, all values are included (=default).
*** '''IncludeOthers''':
**** Should the rest of the attribute values not included due to MinValueUsage or MaxNumUniqueValues filtering be included as an aggregated "Others" value?
**** If given any string value, that value is used as the value for all the aggregated values.
**** Default = undefined => others values will not be included into the results.
*** '''ValueIfNull''': Value used to indicate null-values.
**** Default = "(blank)".
**** Must be not null.

=== Return value ===
Returns a [[SqlDataFrame in Expression Language|SqlDataFrame]] object with the following columns:

* Common columns:
** '''Type''':
*** Type of the root cause:
**** "CaseAttributeValue" for case attributes.
**** "EventAttributeValue" for event attributes.
** '''Name''':
*** When type is CaseAttributeValue, case attribute name.
*** When type is EventAttributeValue, event attribute name.
** '''Value''':
*** When type is CaseAttributeValue, case attribute value.
*** When type is EventAttributeValue, event attribute value and number of occurrences in case.
** '''Total''':
*** The total number of cases having the found root cause.
** '''Selected''':
*** The number of cases that have the found root cause and belong to the selected cases.
** '''Compared''':
*** The number of cases that have the found root cause and don't belong to the selected cases.
** Columns when WeightingExpression not have value:
*** '''Contribution''':
**** The number of cases which contribute to the deviation from the average percentage of selected cases among all analyzed cases.
*** '''ContributionPercentage''':
**** The percent of cases which contribute to the deviation from the average percentage.
*** '''DifferencePercentage''':
**** The deviation in percentage between selected cases with the found root cause and the average percentage of selected cases among all analyzed cases.
*** '''SelectedPercentage''':
**** The percent of selected cases that have the found root cause out of all cases with that root cause.
** Columns when WeightingExpression have value:
*** '''Contribution''':
**** The sum of case weights which contribute to the deviation from the average percentage of selected case weights among all analyzed cases.
*** '''ContributionPercentage''':
**** The percent of case weights which contribute to the deviation from the average percentage.
*** '''DifferencePercentage''':
**** The deviation in percentage between selected case weights with the found root cause and the average percentage of selected case weights among all analyzed cases.
*** '''SelectedPercentage''':
**** The percent of selected case weights that have the found root cause out of all case weights with that root cause.
*** '''SelectedWeight''':
**** The sum of weights that have the found root cause and belong to the selected cases.
*** '''ComparedWeight''':
**** The sum of weights that have the found root cause and don't belong to the selected cases.
*** '''TotalWeight''':
**** The sum of weights of all cases with that root cause.

=== Example ===
Calculate root cause analysis for given model using parameters read from the [[Web API: Expression/query|query configuration]].<syntaxhighlight lang="json">
{
"ProcessingMethod": "DataFrame",
"Root":"let m = _; _system.RootCauses.FindForDataFrame(m, _query.Configuration.Parameters.FindRootCausesParameters.Clone().Extend(#{\"Filter\": _query.Configuration.Filter}))",
"Parameters": {
"FindRootCausesParameters": {
"CaseAttributeTypes": ["Account Manager","Customer Group","Product Group","Region"],
"Selection": {"Items":[{"Type":"IncludeCases","Items":[{"Type":"CaseAttributeValue","Attribute":"Product Group","StringifiedValues":["0Hats"]}]}]},
"MaxNumUniqueValues": 2,
"MaximumRowCount": 1000,
"MinValueUsage": 0.20,
"WeightingExpression": "Column(\"Cost\")"
}
},
"Ordering": [
{"Name": "Contribution", "Direction": "Descending"}
]
}

</syntaxhighlight>

== Utils.GetSampledEvents ==
Returns a [[SqlDataFrame in Expression Language|SqlDataFrame]] containing sampled events of given model where given filter is first applied.

=== Parameters ===

* '''sourceModel''':
** ProcessAnalyzer model object of the model whose event data is to be filtered and sampled.
* '''sampledCaseCount''':
** The maximum number of cases to return (or null if all cases should be returned).
* '''filter''':
** JSON filter to be applied on the event data of the source model prior to performing the sampling.

=== Return value ===
[[SqlDataFrame in Expression Language|SqlDataFrame]] containing sampled events of given model where given filter is first applied.

== Utils.RunFunctionWithParallelLogging ==
foo

System Library

2024-11-19T12:41:03Z

MarHink: /* Parameters */

System library is a collection of Expression Language functions and properties that provide additional tools for scripting Process Analyzer functionalities. System library is referenced in scripts via ''_system''-property, which provides additional properties dedicated for different areas of interests for scripting.

The following hierarchy shows the properties and functions available in System Library:

* ML
** [[Create Predicted Eventlog|GeneratePredictionModel]]
** [[Create Simulated Eventlog|ApplyTransformations]]
* Parallel
** Run
* RootCauses
** FindRootCausesDataFrame
* Utils
** GetSampledEvents
** ModifyColumnTypes
** RunFunctionWithParallelLogging

== Parallel.Run ==
Runs given functions in parallel.

=== Parameters ===
* '''functions''':
** An array of functions to run in parallel.

=== Return value ===
An array of results returned by the called functions, in the same order as the function generating them in the ''functions''-parameter.

=== Example ===
The following script uses _system.Parallel.Run to run three functions:

* SAP-extraction from VBAK-table in SAP (connection parameters defined in connectionParametersDict-dictionary).
* Transform the extracted data by adding a new column.
* Load the data into data table identified by dataTableId.
<syntaxhighlight lang="typescript">
function ExtractTransformAndLoad(extractFunc, transformFunc, loadFunc)
{
let rawDataFlow = extractFunc();
let transformedDataFlow = ToDataFlow();

_system.Parallel.Run([
() => Catch({
let df;
while (!IsNullTop(df = rawDataFlow.Collect(#{"CollectChunk": true}))) {
transformedDataFlow.Append(transformFunc(df));
WriteLog(`A chunk having ${df.NRows} rows has been transformed.`);
}
if (rawDataFlow.HasError) {
transformedDataFlow.Fail("Error occurred during data extraction.");
}
else {
transformedDataFlow.Complete();
}
}, {
transformedDataFlow.Fail("Error occurred during transformation calculation.");
}),
() => {
loadFunc(transformedDataFlow);
}
]);
}

ExtractTransformAndLoad(
() => ExtractSap(connectionParametersDict.Extend(
[
"FieldNames": "VBELN,ERDAT,ERZET,ERNAM,NETWR,WAERK",
"QueryTable": "VBAK",
"Options": ["VBELN BETWEEN '0000017448'" ,"AND '0060000042'"],
"UseGateway": true
])
),
df => df.SetColumns(["Test": () => `${Column("NETWR")} ${Column("WAERK")}`]),
dataFlow => {
DataTableById(dataTableId).Import(dataFlow, ["Append": 0]);
}
);

DataTableById(dataTableId).SqlDataFrame.OrderByColumns(["VBELN"], [true]).Collect()
</syntaxhighlight>

== RootCauses.FindForDataFrame ==
Finds root causes for a particular process phenomenon by comparing properties of selected cases against those of all cases in given model.

Based on the similar in-memory function: [[FindRootCauses Function|EventLog.FindRootCauses]].

=== Parameters ===

* '''model''':
** Model object of a model whose data tables are used to calculate the root causes.
* '''parameters''':
** A parameter convertible to a StringDictionary object with the following supported key-values:
*** '''Filter''':
**** Filter json (#30921#) that is applied to the event log before calculating root causes.
*** '''Selection''':
**** Selection json (#30927#) that defines selected cases to find root causes for. Is applied on top of the filtered event log (specified by Filter parameter).
**** If nothing is selected, 100% of cases are counted as selected.
*** '''CaseAttributeTypes''':
**** An array of strings with the names of case attributes included into the root causes.
**** Only case attributes of type string, integer or boolean are included.
**** If CaseAttributeTypes is null or empty string or not specified, all case attributes having type "String" are included into the root causes.
**** If the model doesn't have a case attribute with the specified name, an error message is shown.
*** '''EventAttributeTypes''':
**** An array of strings with the names of event attributes included into the root causes.
**** Only event attributes of type string are included.
**** If EventAttributeTypes is null, all event attributes having type "String" are included into the root causes.
**** If EventAttributeTypes is empty array or not specified, event attributes not applied to the root causes.
**** If the model doesn't have an event attribute with the specified name, an error message is shown.
**** Analysis column Name should contain Event Attribute Name.
**** Analysis column Value should contain Event Attribute Value and number of occurrences in case: <value> (count).
**** Analysis column Type should have "EventAttributeValue" as its value.
*** '''WeightingExpression''':
**** Expression providing weights for each case.
**** Expression, if defined, must be any of the following types:
***** A string containing the [[SQL Expressions|SqlExpression]] to evaluate.
***** A SqlExpression object, created e.g., using ToSqlExpression-function (see also: [[QPR_ProcessAnalyzer_Expressions#In-memory_expression_blocks_in_SQL_expressions|In-memory expression blocks in SQL expressions]]).
**** A row is filtered out of result if expression result is null.
*** '''MaximumRowCount''':
**** The maximum number of the most and least contributing root causes to return. Thus, the actual number of returned rows can be at most two times this value (if specified).
**** If undefined, 200 is used.
**** If set to 0, all rows are returned.
*** '''MinValueUsage''':
**** The minimum total usage of a value included into the comparison. The number of cases having every returned value should be at least given percentage (a float value between 0.0 and 1.0) of all the compared cases.
***** If not defined or null, all values are included (=default).
*** '''MaxNumUniqueValues''':
**** Maximum number of unique values to include into the comparison for each attribute column. If the amount of unique values for any attribute exceeds this value, only given number of attributes are included that have the highest usage.
**** If not defined or null, all values are included (=default).
*** '''IncludeOthers''':
**** Should the rest of the attribute values not included due to MinValueUsage or MaxNumUniqueValues filtering be included as an aggregated "Others" value?
**** If given any string value, that value is used as the value for all the aggregated values.
**** Default = undefined => others values will not be included into the results.
*** '''ValueIfNull''': Value used to indicate null-values.
**** Default = "(blank)".
**** Must be not null.

=== Return value ===
Returns a SqlDataFrame object with the following columns:

* Common columns:
** '''Type''':
*** Type of the root cause:
**** "CaseAttributeValue" for case attributes.
**** "EventAttributeValue" for event attributes.
** '''Name''':
*** When type is CaseAttributeValue, case attribute name.
*** When type is EventAttributeValue, event attribute name.
** '''Value''':
*** When type is CaseAttributeValue, case attribute value.
*** When type is EventAttributeValue, event attribute value and number of occurrences in case.
** '''Total''':
*** The total number of cases having the found root cause.
** '''Selected''':
*** The number of cases that have the found root cause and belong to the selected cases.
** '''Compared''':
*** The number of cases that have the found root cause and don't belong to the selected cases.
** Columns when WeightingExpression not have value:
*** '''Contribution''':
**** The number of cases which contribute to the deviation from the average percentage of selected cases among all analyzed cases.
*** '''ContributionPercentage''':
**** The percent of cases which contribute to the deviation from the average percentage.
*** '''DifferencePercentage''':
**** The deviation in percentage between selected cases with the found root cause and the average percentage of selected cases among all analyzed cases.
*** '''SelectedPercentage''':
**** The percent of selected cases that have the found root cause out of all cases with that root cause.
** Columns when WeightingExpression have value:
*** '''Contribution''':
**** The sum of case weights which contribute to the deviation from the average percentage of selected case weights among all analyzed cases.
*** '''ContributionPercentage''':
**** The percent of case weights which contribute to the deviation from the average percentage.
*** '''DifferencePercentage''':
**** The deviation in percentage between selected case weights with the found root cause and the average percentage of selected case weights among all analyzed cases.
*** '''SelectedPercentage''':
**** The percent of selected case weights that have the found root cause out of all case weights with that root cause.
*** '''SelectedWeight''':
**** The sum of weights that have the found root cause and belong to the selected cases.
*** '''ComparedWeight''':
**** The sum of weights that have the found root cause and don't belong to the selected cases.
*** '''TotalWeight''':
**** The sum of weights of all cases with that root cause.

=== Example ===
Calculate root cause analysis for given model using parameters read from the [[Web API: Expression/query|query configuration]].<syntaxhighlight lang="json">
{
"ProcessingMethod": "DataFrame",
"Root":"let m = _; _system.RootCauses.FindForDataFrame(m, _query.Configuration.Parameters.FindRootCausesParameters.Clone().Extend(#{\"Filter\": _query.Configuration.Filter}))",
"Parameters": {
"FindRootCausesParameters": {
"CaseAttributeTypes": ["Account Manager","Customer Group","Product Group","Region"],
"Selection": {"Items":[{"Type":"IncludeCases","Items":[{"Type":"CaseAttributeValue","Attribute":"Product Group","StringifiedValues":["0Hats"]}]}]},
"MaxNumUniqueValues": 2,
"MaximumRowCount": 1000,
"MinValueUsage": 0.20,
"WeightingExpression": "Column(\"Cost\")"
}
},
"Ordering": [
{"Name": "Contribution", "Direction": "Descending"}
]
}

</syntaxhighlight>

== Utils.GetSampledEvents ==
foo

== Utils.RunFunctionWithParallelLogging ==
foo

System Library

2024-11-19T12:38:42Z

MarHink: /* RootCauses.FindRootCausesDataFrame */

System library is a collection of Expression Language functions and properties that provide additional tools for scripting Process Analyzer functionalities. System library is referenced in scripts via ''_system''-property, which provides additional properties dedicated for different areas of interests for scripting.

The following hierarchy shows the properties and functions available in System Library:

* ML
** [[Create Predicted Eventlog|GeneratePredictionModel]]
** [[Create Simulated Eventlog|ApplyTransformations]]
* Parallel
** Run
* RootCauses
** FindRootCausesDataFrame
* Utils
** GetSampledEvents
** ModifyColumnTypes
** RunFunctionWithParallelLogging

== Parallel.Run ==
Runs given functions in parallel.

=== Parameters ===
* '''functions''':
** An array of functions to run in parallel.

=== Return value ===
An array of results returned by the called functions, in the same order as the function generating them in the ''functions''-parameter.

=== Example ===
The following script uses _system.Parallel.Run to run three functions:

* SAP-extraction from VBAK-table in SAP (connection parameters defined in connectionParametersDict-dictionary).
* Transform the extracted data by adding a new column.
* Load the data into data table identified by dataTableId.
<syntaxhighlight lang="typescript">
function ExtractTransformAndLoad(extractFunc, transformFunc, loadFunc)
{
let rawDataFlow = extractFunc();
let transformedDataFlow = ToDataFlow();

_system.Parallel.Run([
() => Catch({
let df;
while (!IsNullTop(df = rawDataFlow.Collect(#{"CollectChunk": true}))) {
transformedDataFlow.Append(transformFunc(df));
WriteLog(`A chunk having ${df.NRows} rows has been transformed.`);
}
if (rawDataFlow.HasError) {
transformedDataFlow.Fail("Error occurred during data extraction.");
}
else {
transformedDataFlow.Complete();
}
}, {
transformedDataFlow.Fail("Error occurred during transformation calculation.");
}),
() => {
loadFunc(transformedDataFlow);
}
]);
}

ExtractTransformAndLoad(
() => ExtractSap(connectionParametersDict.Extend(
[
"FieldNames": "VBELN,ERDAT,ERZET,ERNAM,NETWR,WAERK",
"QueryTable": "VBAK",
"Options": ["VBELN BETWEEN '0000017448'" ,"AND '0060000042'"],
"UseGateway": true
])
),
df => df.SetColumns(["Test": () => `${Column("NETWR")} ${Column("WAERK")}`]),
dataFlow => {
DataTableById(dataTableId).Import(dataFlow, ["Append": 0]);
}
);

DataTableById(dataTableId).SqlDataFrame.OrderByColumns(["VBELN"], [true]).Collect()
</syntaxhighlight>

== RootCauses.FindForDataFrame ==
Finds root causes for a particular process phenomenon by comparing properties of selected cases against those of all cases in given model (#27619#).

Based on the similar in-memory function: [[FindRootCauses Function|EventLog.FindRootCauses]].

=== Parameters ===

* '''model''':
** Model object of a model whose data tables are used to calculate the root causes.
* '''parameters''':
** A parameter convertible to a StringDictionary object with the following supported key-values:
*** '''Filter''':
**** Filter json (#30921#) that is applied to the event log before calculating root causes.
*** '''Selection''':
**** Selection json (#30927#) that defines selected cases to find root causes for. Is applied on top of the filtered event log (specified by Filter parameter).
**** If nothing is selected, 100% of cases are counted as selected.
*** '''CaseAttributeTypes''':
**** An array of strings with the names of case attributes included into the root causes.
**** Only case attributes of type string, integer or boolean are included.
**** If CaseAttributeTypes is null or empty string or not specified, all case attributes having type "String" are included into the root causes.
**** If the model doesn't have a case attribute with the specified name, an error message is shown.
*** '''EventAttributeTypes''':
**** An array of strings with the names of event attributes included into the root causes.
**** Only event attributes of type string are included.
**** If EventAttributeTypes is null, all event attributes having type "String" are included into the root causes.
**** If EventAttributeTypes is empty array or not specified, event attributes not applied to the root causes.
**** If the model doesn't have an event attribute with the specified name, an error message is shown.
**** Analysis column Name should contain Event Attribute Name.
**** Analysis column Value should contain Event Attribute Value and number of occurrences in case: <value> (count).
**** Analysis column Type should have "EventAttributeValue" as its value.
*** '''WeightingExpression''':
**** Expression providing weights for each case.
**** Expression, if defined, must be any of the following types:
***** A string containing the [[SQL Expressions|SqlExpression]] to evaluate.
***** A SqlExpression object, created e.g., using ToSqlExpression-function (see also: [[QPR_ProcessAnalyzer_Expressions#In-memory_expression_blocks_in_SQL_expressions|In-memory expression blocks in SQL expressions]]).
**** A row is filtered out of result if expression result is null.
*** '''MaximumRowCount''':
**** The maximum number of the most and least contributing root causes to return. Thus, the actual number of returned rows can be at most two times this value (if specified).
**** If undefined, 200 is used.
**** If set to 0, all rows are returned.
*** '''MinValueUsage''':
**** The minimum total usage of a value included into the comparison. The number of cases having every returned value should be at least given percentage (a float value between 0.0 and 1.0) of all the compared cases.
***** If not defined or null, all values are included (=default).
*** '''MaxNumUniqueValues''':
**** Maximum number of unique values to include into the comparison for each attribute column. If the amount of unique values for any attribute exceeds this value, only given number of attributes are included that have the highest usage.
**** If not defined or null, all values are included (=default).
*** '''IncludeOthers''':
**** Should the rest of the attribute values not included due to MinValueUsage or MaxNumUniqueValues filtering be included as an aggregated "Others" value?
**** If given any string value, that value is used as the value for all the aggregated values.
**** Default = undefined => others values will not be included into the results.
*** '''ValueIfNull''': Value used to indicate null-values.
**** Default = "(blank)".
**** Must be not null.

=== Return value ===
Returns a SqlDataFrame object with the following columns:

* Common columns:
** '''Type''':
*** Type of the root cause:
**** "CaseAttributeValue" for case attributes.
**** "EventAttributeValue" for event attributes.
** '''Name''':
*** When type is CaseAttributeValue, case attribute name.
*** When type is EventAttributeValue, event attribute name.
** '''Value''':
*** When type is CaseAttributeValue, case attribute value.
*** When type is EventAttributeValue, event attribute value and number of occurrences in case.
** '''Total''':
*** The total number of cases having the found root cause.
** '''Selected''':
*** The number of cases that have the found root cause and belong to the selected cases.
** '''Compared''':
*** The number of cases that have the found root cause and don't belong to the selected cases.
** Columns when WeightingExpression not have value:
*** '''Contribution''':
**** The number of cases which contribute to the deviation from the average percentage of selected cases among all analyzed cases.
*** '''ContributionPercentage''':
**** The percent of cases which contribute to the deviation from the average percentage.
*** '''DifferencePercentage''':
**** The deviation in percentage between selected cases with the found root cause and the average percentage of selected cases among all analyzed cases.
*** '''SelectedPercentage''':
**** The percent of selected cases that have the found root cause out of all cases with that root cause.
** Columns when WeightingExpression have value:
*** '''Contribution''':
**** The sum of case weights which contribute to the deviation from the average percentage of selected case weights among all analyzed cases.
*** '''ContributionPercentage''':
**** The percent of case weights which contribute to the deviation from the average percentage.
*** '''DifferencePercentage''':
**** The deviation in percentage between selected case weights with the found root cause and the average percentage of selected case weights among all analyzed cases.
*** '''SelectedPercentage''':
**** The percent of selected case weights that have the found root cause out of all case weights with that root cause.
*** '''SelectedWeight''':
**** The sum of weights that have the found root cause and belong to the selected cases.
*** '''ComparedWeight''':
**** The sum of weights that have the found root cause and don't belong to the selected cases.
*** '''TotalWeight''':
**** The sum of weights of all cases with that root cause.

=== Example ===
Calculate root cause analysis for given model using parameters read from the query configuration.<syntaxhighlight lang="json">
{
"ProcessingMethod": "DataFrame",
"Root":"let m = _; _system.RootCauses.FindForDataFrame(m, _query.Configuration.Parameters.FindRootCausesParameters.Clone().Extend(#{\"Filter\": _query.Configuration.Filter}))",
"Parameters": {
"FindRootCausesParameters": {
"CaseAttributeTypes": ["Account Manager","Customer Group","Product Group","Region"],
"Selection": {"Items":[{"Type":"IncludeCases","Items":[{"Type":"CaseAttributeValue","Attribute":"Product Group","StringifiedValues":["0Hats"]}]}]},
"MaxNumUniqueValues": 2,
"MaximumRowCount": 1000,
"MinValueUsage": 0.20,
"WeightingExpression": "Column(\"Cost\")"
}
},
"Ordering": [
{"Name": "Contribution", "Direction": "Descending"}
]
}

</syntaxhighlight>

== Utils.GetSampledEvents ==
foo

== Utils.RunFunctionWithParallelLogging ==
foo

System Library

2024-11-19T12:36:56Z

MarHink: /* Return value */

System library is a collection of Expression Language functions and properties that provide additional tools for scripting Process Analyzer functionalities. System library is referenced in scripts via ''_system''-property, which provides additional properties dedicated for different areas of interests for scripting.

The following hierarchy shows the properties and functions available in System Library:

* ML
** [[Create Predicted Eventlog|GeneratePredictionModel]]
** [[Create Simulated Eventlog|ApplyTransformations]]
* Parallel
** Run
* RootCauses
** FindRootCausesDataFrame
* Utils
** GetSampledEvents
** ModifyColumnTypes
** RunFunctionWithParallelLogging

== Parallel.Run ==
Runs given functions in parallel.

=== Parameters ===
* '''functions''':
** An array of functions to run in parallel.

=== Return value ===
An array of results returned by the called functions, in the same order as the function generating them in the ''functions''-parameter.

=== Example ===
The following script uses _system.Parallel.Run to run three functions:

* SAP-extraction from VBAK-table in SAP (connection parameters defined in connectionParametersDict-dictionary).
* Transform the extracted data by adding a new column.
* Load the data into data table identified by dataTableId.
<syntaxhighlight lang="typescript">
function ExtractTransformAndLoad(extractFunc, transformFunc, loadFunc)
{
let rawDataFlow = extractFunc();
let transformedDataFlow = ToDataFlow();

_system.Parallel.Run([
() => Catch({
let df;
while (!IsNullTop(df = rawDataFlow.Collect(#{"CollectChunk": true}))) {
transformedDataFlow.Append(transformFunc(df));
WriteLog(`A chunk having ${df.NRows} rows has been transformed.`);
}
if (rawDataFlow.HasError) {
transformedDataFlow.Fail("Error occurred during data extraction.");
}
else {
transformedDataFlow.Complete();
}
}, {
transformedDataFlow.Fail("Error occurred during transformation calculation.");
}),
() => {
loadFunc(transformedDataFlow);
}
]);
}

ExtractTransformAndLoad(
() => ExtractSap(connectionParametersDict.Extend(
[
"FieldNames": "VBELN,ERDAT,ERZET,ERNAM,NETWR,WAERK",
"QueryTable": "VBAK",
"Options": ["VBELN BETWEEN '0000017448'" ,"AND '0060000042'"],
"UseGateway": true
])
),
df => df.SetColumns(["Test": () => `${Column("NETWR")} ${Column("WAERK")}`]),
dataFlow => {
DataTableById(dataTableId).Import(dataFlow, ["Append": 0]);
}
);

DataTableById(dataTableId).SqlDataFrame.OrderByColumns(["VBELN"], [true]).Collect()
</syntaxhighlight>

== RootCauses.FindRootCausesDataFrame ==
Finds root causes for a particular process phenomenon by comparing properties of selected cases against those of all cases in given model (#27619#).

Based on the similar in-memory function: [[FindRootCauses Function|EventLog.FindRootCauses]].

=== Parameters ===

* model:
** Model object of a model whose data tables are used to calculate the root causes.
* parameters:
** A parameter convertible to a StringDictionary object with the following supported key-values:
*** Filter:
**** Filter json (#30921#) that is applied to the event log before calculating root causes.
*** Selection:
**** Selection json (#30927#) that defines selected cases to find root causes for. Is applied on top of the filtered event log (specified by Filter parameter).
**** If nothing is selected, 100% of cases are counted as selected.
*** CaseAttributeTypes:
**** An array of strings with the names of case attributes included into the root causes.
**** Only case attributes of type string, integer or boolean are included.
**** If CaseAttributeTypes is null or empty string or not specified, all case attributes having type "String" are included into the root causes.
**** If the model doesn't have a case attribute with the specified name, an error message is shown.
*** EventAttributeTypes:
**** An array of strings with the names of event attributes included into the root causes.
**** Only event attributes of type string are included.
**** If EventAttributeTypes is null, all event attributes having type "String" are included into the root causes.
**** If EventAttributeTypes is empty array or not specified, event attributes not applied to the root causes.
**** If the model doesn't have an event attribute with the specified name, an error message is shown.
**** Analysis column Name should contain Event Attribute Name.
**** Analysis column Value should contain Event Attribute Value and number of occurrences in case: <value> (count).
**** Analysis column Type should have "EventAttributeValue" as its value.
*** WeightingExpression:
**** Expression providing weights for each case.
**** Expression, if defined, must be any of the following types:
***** A string containing the [[SQL Expressions|SqlExpression]] to evaluate.
***** A SqlExpression object, created e.g., using ToSqlExpression-function (see also: [[QPR_ProcessAnalyzer_Expressions#In-memory_expression_blocks_in_SQL_expressions|In-memory expression blocks in SQL expressions]]).
**** A row is filtered out of result if expression result is null.
*** MaximumRowCount:
**** The maximum number of the most and least contributing root causes to return. Thus, the actual number of returned rows can be at most two times this value (if specified).
**** If undefined, 200 is used.
**** If set to 0, all rows are returned.
*** MinValueUsage:
**** The minimum total usage of a value included into the comparison. The number of cases having every returned value should be at least given percentage (a float value between 0.0 and 1.0) of all the compared cases.
***** If not defined or null, all values are included (=default).
*** MaxNumUniqueValues:
**** Maximum number of unique values to include into the comparison for each attribute column. If the amount of unique values for any attribute exceeds this value, only given number of attributes are included that have the highest usage.
**** If not defined or null, all values are included (=default).
*** IncludeOthers:
**** Should the rest of the attribute values not included due to MinValueUsage or MaxNumUniqueValues filtering be included as an aggregated "Others" value?
**** If given any string value, that value is used as the value for all the aggregated values.
**** Default = undefined => others values will not be included into the results.
*** ValueIfNull: Value used to indicate null-values.
**** Default = "(blank)".
**** Must be not null.

=== Return value ===
Returns a SqlDataFrame object with the following columns:

* Common columns:
** Type:
*** Type of the root cause:
**** "CaseAttributeValue" for case attributes.
**** "EventAttributeValue" for event attributes.
** Name:
*** When type is CaseAttributeValue, case attribute name.
*** When type is EventAttributeValue, event attribute name.
** Value:
*** When type is CaseAttributeValue, case attribute value.
*** When type is EventAttributeValue, event attribute value and number of occurrences in case.
** Total:
*** The total number of cases having the found root cause.
** Selected:
*** The number of cases that have the found root cause and belong to the selected cases.
** Compared:
*** The number of cases that have the found root cause and don't belong to the selected cases.
** Columns when WeightingExpression not have value:
*** Contribution:
**** The number of cases which contribute to the deviation from the average percentage of selected cases among all analyzed cases.
*** ContributionPercentage:
**** The percent of cases which contribute to the deviation from the average percentage.
*** DifferencePercentage:
**** The deviation in percentage between selected cases with the found root cause and the average percentage of selected cases among all analyzed cases.
*** SelectedPercentage:
**** The percent of selected cases that have the found root cause out of all cases with that root cause.
** Columns when WeightingExpression have value:
*** Contribution:
**** The sum of case weights which contribute to the deviation from the average percentage of selected case weights among all analyzed cases.
*** ContributionPercentage:
**** The percent of case weights which contribute to the deviation from the average percentage.
*** DifferencePercentage:
**** The deviation in percentage between selected case weights with the found root cause and the average percentage of selected case weights among all analyzed cases.
*** SelectedPercentage:
**** The percent of selected case weights that have the found root cause out of all case weights with that root cause.
*** SelectedWeight:
**** The sum of weights that have the found root cause and belong to the selected cases.
*** ComparedWeight:
**** The sum of weights that have the found root cause and don't belong to the selected cases.
*** TotalWeight:
**** The sum of weights of all cases with that root cause.

=== Example ===
Calculate root cause analysis for given model using parameters read from the query configuration.<syntaxhighlight lang="json">
{
"ProcessingMethod": "DataFrame",
"Root":"let m = _; _system.RootCauses.FindForDataFrame(m, _query.Configuration.Parameters.FindRootCausesParameters.Clone().Extend(#{\"Filter\": _query.Configuration.Filter}))",
"Parameters": {
"FindRootCausesParameters": {
"CaseAttributeTypes": ["Account Manager","Customer Group","Product Group","Region"],
"Selection": {"Items":[{"Type":"IncludeCases","Items":[{"Type":"CaseAttributeValue","Attribute":"Product Group","StringifiedValues":["0Hats"]}]}]},
"MaxNumUniqueValues": 2,
"MaximumRowCount": 1000,
"MinValueUsage": 0.20,
"WeightingExpression": "Column(\"Cost\")"
}
},
"Ordering": [
{"Name": "Contribution", "Direction": "Descending"}
]
}

</syntaxhighlight>

== Utils.GetSampledEvents ==
foo

== Utils.RunFunctionWithParallelLogging ==
foo

System Library

2024-11-19T12:34:28Z

MarHink: /* Return value */

System library is a collection of Expression Language functions and properties that provide additional tools for scripting Process Analyzer functionalities. System library is referenced in scripts via ''_system''-property, which provides additional properties dedicated for different areas of interests for scripting.

The following hierarchy shows the properties and functions available in System Library:

* ML
** [[Create Predicted Eventlog|GeneratePredictionModel]]
** [[Create Simulated Eventlog|ApplyTransformations]]
* Parallel
** Run
* RootCauses
** FindRootCausesDataFrame
* Utils
** GetSampledEvents
** ModifyColumnTypes
** RunFunctionWithParallelLogging

== Parallel.Run ==
Runs given functions in parallel.

=== Parameters ===
* '''functions''':
** An array of functions to run in parallel.

=== Return value ===
An array of results returned by the called functions, in the same order as the function generating them in the ''functions''-parameter.

=== Example ===
The following script uses _system.Parallel.Run to run three functions:

* SAP-extraction from VBAK-table in SAP (connection parameters defined in connectionParametersDict-dictionary).
* Transform the extracted data by adding a new column.
* Load the data into data table identified by dataTableId.
<syntaxhighlight lang="typescript">
function ExtractTransformAndLoad(extractFunc, transformFunc, loadFunc)
{
let rawDataFlow = extractFunc();
let transformedDataFlow = ToDataFlow();

_system.Parallel.Run([
() => Catch({
let df;
while (!IsNullTop(df = rawDataFlow.Collect(#{"CollectChunk": true}))) {
transformedDataFlow.Append(transformFunc(df));
WriteLog(`A chunk having ${df.NRows} rows has been transformed.`);
}
if (rawDataFlow.HasError) {
transformedDataFlow.Fail("Error occurred during data extraction.");
}
else {
transformedDataFlow.Complete();
}
}, {
transformedDataFlow.Fail("Error occurred during transformation calculation.");
}),
() => {
loadFunc(transformedDataFlow);
}
]);
}

ExtractTransformAndLoad(
() => ExtractSap(connectionParametersDict.Extend(
[
"FieldNames": "VBELN,ERDAT,ERZET,ERNAM,NETWR,WAERK",
"QueryTable": "VBAK",
"Options": ["VBELN BETWEEN '0000017448'" ,"AND '0060000042'"],
"UseGateway": true
])
),
df => df.SetColumns(["Test": () => `${Column("NETWR")} ${Column("WAERK")}`]),
dataFlow => {
DataTableById(dataTableId).Import(dataFlow, ["Append": 0]);
}
);

DataTableById(dataTableId).SqlDataFrame.OrderByColumns(["VBELN"], [true]).Collect()
</syntaxhighlight>

== RootCauses.FindRootCausesDataFrame ==
Finds root causes for a particular process phenomenon by comparing properties of selected cases against those of all cases in given model (#27619#).

Based on the similar in-memory function: [[FindRootCauses Function|EventLog.FindRootCauses]].

=== Parameters ===

* model:
** Model object of a model whose data tables are used to calculate the root causes.
* parameters:
** A parameter convertible to a StringDictionary object with the following supported key-values:
*** Filter:
**** Filter json (#30921#) that is applied to the event log before calculating root causes.
*** Selection:
**** Selection json (#30927#) that defines selected cases to find root causes for. Is applied on top of the filtered event log (specified by Filter parameter).
**** If nothing is selected, 100% of cases are counted as selected.
*** CaseAttributeTypes:
**** An array of strings with the names of case attributes included into the root causes.
**** Only case attributes of type string, integer or boolean are included.
**** If CaseAttributeTypes is null or empty string or not specified, all case attributes having type "String" are included into the root causes.
**** If the model doesn't have a case attribute with the specified name, an error message is shown.
*** EventAttributeTypes:
**** An array of strings with the names of event attributes included into the root causes.
**** Only event attributes of type string are included.
**** If EventAttributeTypes is null, all event attributes having type "String" are included into the root causes.
**** If EventAttributeTypes is empty array or not specified, event attributes not applied to the root causes.
**** If the model doesn't have an event attribute with the specified name, an error message is shown.
**** Analysis column Name should contain Event Attribute Name.
**** Analysis column Value should contain Event Attribute Value and number of occurrences in case: <value> (count).
**** Analysis column Type should have "EventAttributeValue" as its value.
*** WeightingExpression:
**** Expression providing weights for each case.
**** Expression, if defined, must be any of the following types:
***** A string containing the [[SQL Expressions|SqlExpression]] to evaluate.
***** A SqlExpression object, created e.g., using ToSqlExpression-function (see also: [[QPR_ProcessAnalyzer_Expressions#In-memory_expression_blocks_in_SQL_expressions|In-memory expression blocks in SQL expressions]]).
**** A row is filtered out of result if expression result is null.
*** MaximumRowCount:
**** The maximum number of the most and least contributing root causes to return. Thus, the actual number of returned rows can be at most two times this value (if specified).
**** If undefined, 200 is used.
**** If set to 0, all rows are returned.
*** MinValueUsage:
**** The minimum total usage of a value included into the comparison. The number of cases having every returned value should be at least given percentage (a float value between 0.0 and 1.0) of all the compared cases.
***** If not defined or null, all values are included (=default).
*** MaxNumUniqueValues:
**** Maximum number of unique values to include into the comparison for each attribute column. If the amount of unique values for any attribute exceeds this value, only given number of attributes are included that have the highest usage.
**** If not defined or null, all values are included (=default).
*** IncludeOthers:
**** Should the rest of the attribute values not included due to MinValueUsage or MaxNumUniqueValues filtering be included as an aggregated "Others" value?
**** If given any string value, that value is used as the value for all the aggregated values.
**** Default = undefined => others values will not be included into the results.
*** ValueIfNull: Value used to indicate null-values.
**** Default = "(blank)".
**** Must be not null.

=== Return value ===
Returns a SqlDataFrame object with the following columns:

* Common columns:
** Type:
*** Type of the root cause:
**** "CaseAttributeValue" for case attributes.
**** "EventAttributeValue" for event attributes.
** Name:
*** When type is CaseAttributeValue, case attribute name.
*** When type is EventAttributeValue, event attribute name.
** Value:
*** When type is CaseAttributeValue, case attribute value.
*** When type is EventAttributeValue, event attribute value and number of occurrences in case.
** Total:
*** The total number of cases having the found root cause.
** Selected:
*** The number of cases that have the found root cause and belong to the selected cases.
** Compared:
*** The number of cases that have the found root cause and don't belong to the selected cases.
** Columns when WeightingExpression not have value:
*** Contribution:
**** The number of cases which contribute to the deviation from the average percentage of selected cases among all analyzed cases.
*** ContributionPercentage:
**** The percent of cases which contribute to the deviation from the average percentage.
*** DifferencePercentage:
**** The deviation in percentage between selected cases with the found root cause and the average percentage of selected cases among all analyzed cases.
*** SelectedPercentage:
**** The percent of selected cases that have the found root cause out of all cases with that root cause.
** Columns when WeightingExpression have value:
*** Contribution:
**** The sum of case weights which contribute to the deviation from the average percentage of selected case weights among all analyzed cases.
*** ContributionPercentage:
**** The percent of case weights which contribute to the deviation from the average percentage.
*** DifferencePercentage:
**** The deviation in percentage between selected case weights with the found root cause and the average percentage of selected case weights among all analyzed cases.
*** SelectedPercentage:
**** The percent of selected case weights that have the found root cause out of all case weights with that root cause.
*** SelectedWeight:
**** The sum of weights that have the found root cause and belong to the selected cases.
*** ComparedWeight:
**** The sum of weights that have the found root cause and don't belong to the selected cases.
*** TotalWeight:
**** The sum of weights of all cases with that root cause.

Examples:

{

"ProcessingMethod": "DataFrame",

"Root":"let m = _; _system.RootCauses.'''FindForDataFrame'''(m, _query.Configuration.Parameters.FindRootCausesParameters.Clone().Extend(#{\"Filter\": _query.Configuration.Filter}))",

"Parameters": {

   "FindRootCausesParameters": {

   "CaseAttributeTypes": ["Account Manager","Customer Group","Product Group","Region"],

"Selection": {"Items":[{"Type":"IncludeCases","Items":[{"Type":"CaseAttributeValue","Attribute":"Product Group","StringifiedValues":["0Hats"]}]}]},

   "MaxNumUniqueValues": 2,

   "MaximumRowCount": 1000,

   "MinValueUsage": 0.20,

   "WeightingExpression": "Column(\"Cost\")"

   }

},

"Ordering": [

   {"Name": "Contribution", "Direction": "Descending"}

]

}

Calculates root cause analysis for given model using parameters read from the query configuration.

== Utils.GetSampledEvents ==
foo

== Utils.RunFunctionWithParallelLogging ==
foo

System Library

2024-11-19T12:28:55Z

MarHink: /* Parameters */

System library is a collection of Expression Language functions and properties that provide additional tools for scripting Process Analyzer functionalities. System library is referenced in scripts via ''_system''-property, which provides additional properties dedicated for different areas of interests for scripting.

The following hierarchy shows the properties and functions available in System Library:

* ML
** [[Create Predicted Eventlog|GeneratePredictionModel]]
** [[Create Simulated Eventlog|ApplyTransformations]]
* Parallel
** Run
* RootCauses
** FindRootCausesDataFrame
* Utils
** GetSampledEvents
** ModifyColumnTypes
** RunFunctionWithParallelLogging

== Parallel.Run ==
Runs given functions in parallel.

=== Parameters ===
* '''functions''':
** An array of functions to run in parallel.

=== Return value ===
An array of results returned by the called functions, in the same order as the function generating them in the ''functions''-parameter.

=== Example ===
The following script uses _system.Parallel.Run to run three functions:

* SAP-extraction from VBAK-table in SAP (connection parameters defined in connectionParametersDict-dictionary).
* Transform the extracted data by adding a new column.
* Load the data into data table identified by dataTableId.
<syntaxhighlight lang="typescript">
function ExtractTransformAndLoad(extractFunc, transformFunc, loadFunc)
{
let rawDataFlow = extractFunc();
let transformedDataFlow = ToDataFlow();

_system.Parallel.Run([
() => Catch({
let df;
while (!IsNullTop(df = rawDataFlow.Collect(#{"CollectChunk": true}))) {
transformedDataFlow.Append(transformFunc(df));
WriteLog(`A chunk having ${df.NRows} rows has been transformed.`);
}
if (rawDataFlow.HasError) {
transformedDataFlow.Fail("Error occurred during data extraction.");
}
else {
transformedDataFlow.Complete();
}
}, {
transformedDataFlow.Fail("Error occurred during transformation calculation.");
}),
() => {
loadFunc(transformedDataFlow);
}
]);
}

ExtractTransformAndLoad(
() => ExtractSap(connectionParametersDict.Extend(
[
"FieldNames": "VBELN,ERDAT,ERZET,ERNAM,NETWR,WAERK",
"QueryTable": "VBAK",
"Options": ["VBELN BETWEEN '0000017448'" ,"AND '0060000042'"],
"UseGateway": true
])
),
df => df.SetColumns(["Test": () => `${Column("NETWR")} ${Column("WAERK")}`]),
dataFlow => {
DataTableById(dataTableId).Import(dataFlow, ["Append": 0]);
}
);

DataTableById(dataTableId).SqlDataFrame.OrderByColumns(["VBELN"], [true]).Collect()
</syntaxhighlight>

== RootCauses.FindRootCausesDataFrame ==
Finds root causes for a particular process phenomenon by comparing properties of selected cases against those of all cases in given model (#27619#).

Based on the similar in-memory function: [[FindRootCauses Function|EventLog.FindRootCauses]].

=== Parameters ===

* model:
** Model object of a model whose data tables are used to calculate the root causes.
* parameters:
** A parameter convertible to a StringDictionary object with the following supported key-values:
*** Filter:
**** Filter json (#30921#) that is applied to the event log before calculating root causes.
*** Selection:
**** Selection json (#30927#) that defines selected cases to find root causes for. Is applied on top of the filtered event log (specified by Filter parameter).
**** If nothing is selected, 100% of cases are counted as selected.
*** CaseAttributeTypes:
**** An array of strings with the names of case attributes included into the root causes.
**** Only case attributes of type string, integer or boolean are included.
**** If CaseAttributeTypes is null or empty string or not specified, all case attributes having type "String" are included into the root causes.
**** If the model doesn't have a case attribute with the specified name, an error message is shown.
*** EventAttributeTypes:
**** An array of strings with the names of event attributes included into the root causes.
**** Only event attributes of type string are included.
**** If EventAttributeTypes is null, all event attributes having type "String" are included into the root causes.
**** If EventAttributeTypes is empty array or not specified, event attributes not applied to the root causes.
**** If the model doesn't have an event attribute with the specified name, an error message is shown.
**** Analysis column Name should contain Event Attribute Name.
**** Analysis column Value should contain Event Attribute Value and number of occurrences in case: <value> (count).
**** Analysis column Type should have "EventAttributeValue" as its value.
*** WeightingExpression:
**** Expression providing weights for each case.
**** Expression, if defined, must be any of the following types:
***** A string containing the [[SQL Expressions|SqlExpression]] to evaluate.
***** A SqlExpression object, created e.g., using ToSqlExpression-function (see also: [[QPR_ProcessAnalyzer_Expressions#In-memory_expression_blocks_in_SQL_expressions|In-memory expression blocks in SQL expressions]]).
**** A row is filtered out of result if expression result is null.
*** MaximumRowCount:
**** The maximum number of the most and least contributing root causes to return. Thus, the actual number of returned rows can be at most two times this value (if specified).
**** If undefined, 200 is used.
**** If set to 0, all rows are returned.
*** MinValueUsage:
**** The minimum total usage of a value included into the comparison. The number of cases having every returned value should be at least given percentage (a float value between 0.0 and 1.0) of all the compared cases.
***** If not defined or null, all values are included (=default).
*** MaxNumUniqueValues:
**** Maximum number of unique values to include into the comparison for each attribute column. If the amount of unique values for any attribute exceeds this value, only given number of attributes are included that have the highest usage.
**** If not defined or null, all values are included (=default).
*** IncludeOthers:
**** Should the rest of the attribute values not included due to MinValueUsage or MaxNumUniqueValues filtering be included as an aggregated "Others" value?
**** If given any string value, that value is used as the value for all the aggregated values.
**** Default = undefined => others values will not be included into the results.
*** ValueIfNull: Value used to indicate null-values.
**** Default = "(blank)".
**** Must be not null.

=== Return value ===
Returns a SqlDataFrame object with the following columns:

* Common columns:
** Type:
*** Type of the root cause:
**** "CaseAttributeValue" for case attributes
**** "EventAttributeValue" for event attributes

3.1.2 Name:

3.1.2.1 When type is CaseAttributeValue, case attribute name

3.1.2.2 When type is EventAttributeValue, event attribute name

3.1.3 Value:

3.1.3.1 When type is CaseAttributeValue, case attribute value

3.1.3.2 When type is EventAttributeValue, event attribute value and number of occurrences in case

3.1.4 Total: total number of cases having the found root cause

3.1.5 Selected: Number of cases that have the found root cause and belong to the selected cases

3.1.6 Compared: number of cases that have the found root cause and don't belong to the selected cases

3.2 Columns when WeightingExpression not have value

3.2.1 Contribution: the number of cases which contribute to the deviation from the average percentage of selected cases among all analyzed cases

3.2.2 ContributionPercentage: the percent of cases which contribute to the deviation from the average percentage

3.2.3 DifferencePercentage: deviation in percentage between selected cases with the found root cause and the average percentage of selected cases among all analyzed cases

3.1.4 SelectedPercentage: percent of selected cases that have the found root cause out of all cases with that root cause

3.3 Columns when WeightingExpression have value

3.3.1 Contribution: Sum of case weights which contribute to the deviation from the average percentage of selected case weights among all analyzed cases

3.3.2 ContributionPercentage: the percent of case weights which contribute to the deviation from the average percentage

3.3.3 DifferencePercentage: deviation in percentage between selected case weights with the found root cause and the average percentage of selected case weights among all analyzed cases

3.3.4 SelectedPercentage: percent of selected case weights that have the found root cause out of all case weights with that root cause

3.3.5 SelectedWeight: Sum of weights that have the found root cause and belong to the selected cases

3.3.6 ComparedWeight: Sum of weights that have the found root cause and don't belong to the selected cases

3.3.7 TotalWeight: Sum of weights of all cases with that root cause

4. Notable differences between this function and FindRootCauses (#71058#) used for in-memory event logs:

4.1. In FindRootCauses, Zero-weighted contibution rows are removed from the results.

4.2. Non-numeric case cost in FindForDataFrame causes and exception to be thrown. Not in FindRootCauses, where is behaves as if null was used.

4.3. All the case-data table columns can be used as CaseAttributeTypes in FindForDataFrame, even the case id column, which can't be used in FindRootCauses.

4.4. WeightingExpression is given as SqlExpression, not expression language expression as in FindRootCauses.

4.5. FindForDataFrame only supports querying one type of columns at a time. Querying, e.g., columns having both string and integer values will throw an exception.

4.6. If an empty array is given for CaseAttributeTypes-parameter, FindForDataFrame throws an exception, whereas FindRootCauses returns a result without any rows.

4.7. FindForDataFrame converts attribute values to string, because internally it uses UNPIVOT instruction for calculating Value column (requires same data type for unpivoted columns). So even if numeric case attribute is used, Value column will contain strings and sorting by Value will use string comparison.

Examples:

{

"ProcessingMethod": "DataFrame",

"Root":"let m = _; _system.RootCauses.'''FindForDataFrame'''(m, _query.Configuration.Parameters.FindRootCausesParameters.Clone().Extend(#{\"Filter\": _query.Configuration.Filter}))",

"Parameters": {

   "FindRootCausesParameters": {

   "CaseAttributeTypes": ["Account Manager","Customer Group","Product Group","Region"],

"Selection": {"Items":[{"Type":"IncludeCases","Items":[{"Type":"CaseAttributeValue","Attribute":"Product Group","StringifiedValues":["0Hats"]}]}]},

   "MaxNumUniqueValues": 2,

   "MaximumRowCount": 1000,

   "MinValueUsage": 0.20,

   "WeightingExpression": "Column(\"Cost\")"

   }

},

"Ordering": [

   {"Name": "Contribution", "Direction": "Descending"}

]

}

Calculates root cause analysis for given model using parameters read from the query configuration.

== Utils.GetSampledEvents ==
foo

== Utils.RunFunctionWithParallelLogging ==
foo

System Library

2024-11-19T12:27:32Z

MarHink: /* Parameters */

System library is a collection of Expression Language functions and properties that provide additional tools for scripting Process Analyzer functionalities. System library is referenced in scripts via ''_system''-property, which provides additional properties dedicated for different areas of interests for scripting.

The following hierarchy shows the properties and functions available in System Library:

* ML
** [[Create Predicted Eventlog|GeneratePredictionModel]]
** [[Create Simulated Eventlog|ApplyTransformations]]
* Parallel
** Run
* RootCauses
** FindRootCausesDataFrame
* Utils
** GetSampledEvents
** ModifyColumnTypes
** RunFunctionWithParallelLogging

== Parallel.Run ==
Runs given functions in parallel.

=== Parameters ===
* '''functions''':
** An array of functions to run in parallel.

=== Return value ===
An array of results returned by the called functions, in the same order as the function generating them in the ''functions''-parameter.

=== Example ===
The following script uses _system.Parallel.Run to run three functions:

* SAP-extraction from VBAK-table in SAP (connection parameters defined in connectionParametersDict-dictionary).
* Transform the extracted data by adding a new column.
* Load the data into data table identified by dataTableId.
<syntaxhighlight lang="typescript">
function ExtractTransformAndLoad(extractFunc, transformFunc, loadFunc)
{
let rawDataFlow = extractFunc();
let transformedDataFlow = ToDataFlow();

_system.Parallel.Run([
() => Catch({
let df;
while (!IsNullTop(df = rawDataFlow.Collect(#{"CollectChunk": true}))) {
transformedDataFlow.Append(transformFunc(df));
WriteLog(`A chunk having ${df.NRows} rows has been transformed.`);
}
if (rawDataFlow.HasError) {
transformedDataFlow.Fail("Error occurred during data extraction.");
}
else {
transformedDataFlow.Complete();
}
}, {
transformedDataFlow.Fail("Error occurred during transformation calculation.");
}),
() => {
loadFunc(transformedDataFlow);
}
]);
}

ExtractTransformAndLoad(
() => ExtractSap(connectionParametersDict.Extend(
[
"FieldNames": "VBELN,ERDAT,ERZET,ERNAM,NETWR,WAERK",
"QueryTable": "VBAK",
"Options": ["VBELN BETWEEN '0000017448'" ,"AND '0060000042'"],
"UseGateway": true
])
),
df => df.SetColumns(["Test": () => `${Column("NETWR")} ${Column("WAERK")}`]),
dataFlow => {
DataTableById(dataTableId).Import(dataFlow, ["Append": 0]);
}
);

DataTableById(dataTableId).SqlDataFrame.OrderByColumns(["VBELN"], [true]).Collect()
</syntaxhighlight>

== RootCauses.FindRootCausesDataFrame ==
Finds root causes for a particular process phenomenon by comparing properties of selected cases against those of all cases in given model (#27619#).

Based on the similar in-memory function: [[FindRootCauses Function|EventLog.FindRootCauses]].

=== Parameters ===

* model:
** Model object of a model whose data tables are used to calculate the root causes.
* parameters:
** A parameter convertible to a StringDictionary object with the following supported key-values:
*** Filter:
**** Filter json (#30921#) that is applied to the event log before calculating root causes.
*** Selection:
**** Selection json (#30927#) that defines selected cases to find root causes for. Is applied on top of the filtered event log (specified by Filter parameter).
**** If nothing is selected, 100% of cases are counted as selected.
*** CaseAttributeTypes:
**** An array of strings with the names of case attributes included into the root causes.
**** Only case attributes of type string, integer or boolean are included.
**** If CaseAttributeTypes is null or empty string or not specified, all case attributes having type "String" are included into the root causes.
**** If the model doesn't have a case attribute with the specified name, an error message is shown.
*** EventAttributeTypes:
**** An array of strings with the names of event attributes included into the root causes.
**** Only event attributes of type string are included.
**** If EventAttributeTypes is null, all event attributes having type "String" are included into the root causes.
**** If EventAttributeTypes is empty array or not specified, event attributes not applied to the root causes.
**** If the model doesn't have an event attribute with the specified name, an error message is shown.
**** Analysis column Name should contain Event Attribute Name.
**** Analysis column Value should contain Event Attribute Value and number of occurrences in case: <value> (count).
**** Analysis column Type should have "EventAttributeValue" as its value.
*** WeightingExpression:
**** Expression providing weights for each case.
**** Expression, if defined, must be any of the following types:
***** A string containing the [[SQL Expressions|SqlExpression]] to evaluate.
***** A SqlExpression object, created e.g., using ToSqlExpression-function (see also: [[QPR_ProcessAnalyzer_Expressions#In-memory_expression_blocks_in_SQL_expressions|In-memory expression blocks in SQL expressions]]).
**** A row is filtered out of result if expression result is null.
*** MaximumRowCount:
**** The maximum number of the most and least contributing root causes to return. Thus, the actual number of returned rows can be at most two times this value (if specified).
**** If undefined, 200 is used.
**** If set to 0, all rows are returned.
*** MinValueUsage:
**** The minimum total usage of a value included into the comparison. The number of cases having every returned value should be at least given percentage (a float value between 0.0 and 1.0) of all the compared cases.
***** If not defined or null, all values are included (=default).
*** MaxNumUniqueValues:
**** Maximum number of unique values to include into the comparison for each attribute column. If the amount of unique values for any attribute exceeds this value, only given number of attributes are included that have the highest usage.
**** If not defined or null, all values are included (=default).
*** IncludeOthers:
**** Should the rest of the attribute values not included due to MinValueUsage or MaxNumUniqueValues filtering be included as an aggregated "Others" value?
**** If given any string value, that value is used as the value for all the aggregated values.
**** Default = undefined => others values will not be included into the results.
*** ValueIfNull: Value used to indicate null-values.
**** Default = "(blank)".
**** Must be not null.

3 Returns a SqlDataFrame object (#70611#) with the following columns:

3.1 Common columns

3.1.1 Type: type of the root cause

3.1.1.1 "CaseAttributeValue" for case attributes

3.1.1.2 "EventAttributeValue" for event attributes

3.1.2 Name:

3.1.2.1 When type is CaseAttributeValue, case attribute name

3.1.2.2 When type is EventAttributeValue, event attribute name

3.1.3 Value:

3.1.3.1 When type is CaseAttributeValue, case attribute value

3.1.3.2 When type is EventAttributeValue, event attribute value and number of occurrences in case

3.1.4 Total: total number of cases having the found root cause

3.1.5 Selected: Number of cases that have the found root cause and belong to the selected cases

3.1.6 Compared: number of cases that have the found root cause and don't belong to the selected cases

3.2 Columns when WeightingExpression not have value

3.2.1 Contribution: the number of cases which contribute to the deviation from the average percentage of selected cases among all analyzed cases

3.2.2 ContributionPercentage: the percent of cases which contribute to the deviation from the average percentage

3.2.3 DifferencePercentage: deviation in percentage between selected cases with the found root cause and the average percentage of selected cases among all analyzed cases

3.1.4 SelectedPercentage: percent of selected cases that have the found root cause out of all cases with that root cause

3.3 Columns when WeightingExpression have value

3.3.1 Contribution: Sum of case weights which contribute to the deviation from the average percentage of selected case weights among all analyzed cases

3.3.2 ContributionPercentage: the percent of case weights which contribute to the deviation from the average percentage

3.3.3 DifferencePercentage: deviation in percentage between selected case weights with the found root cause and the average percentage of selected case weights among all analyzed cases

3.3.4 SelectedPercentage: percent of selected case weights that have the found root cause out of all case weights with that root cause

3.3.5 SelectedWeight: Sum of weights that have the found root cause and belong to the selected cases

3.3.6 ComparedWeight: Sum of weights that have the found root cause and don't belong to the selected cases

3.3.7 TotalWeight: Sum of weights of all cases with that root cause

4. Notable differences between this function and FindRootCauses (#71058#) used for in-memory event logs:

4.1. In FindRootCauses, Zero-weighted contibution rows are removed from the results.

4.2. Non-numeric case cost in FindForDataFrame causes and exception to be thrown. Not in FindRootCauses, where is behaves as if null was used.

4.3. All the case-data table columns can be used as CaseAttributeTypes in FindForDataFrame, even the case id column, which can't be used in FindRootCauses.

4.4. WeightingExpression is given as SqlExpression, not expression language expression as in FindRootCauses.

4.5. FindForDataFrame only supports querying one type of columns at a time. Querying, e.g., columns having both string and integer values will throw an exception.

4.6. If an empty array is given for CaseAttributeTypes-parameter, FindForDataFrame throws an exception, whereas FindRootCauses returns a result without any rows.

4.7. FindForDataFrame converts attribute values to string, because internally it uses UNPIVOT instruction for calculating Value column (requires same data type for unpivoted columns). So even if numeric case attribute is used, Value column will contain strings and sorting by Value will use string comparison.

Examples:

{

"ProcessingMethod": "DataFrame",

"Root":"let m = _; _system.RootCauses.'''FindForDataFrame'''(m, _query.Configuration.Parameters.FindRootCausesParameters.Clone().Extend(#{\"Filter\": _query.Configuration.Filter}))",

"Parameters": {

   "FindRootCausesParameters": {

   "CaseAttributeTypes": ["Account Manager","Customer Group","Product Group","Region"],

"Selection": {"Items":[{"Type":"IncludeCases","Items":[{"Type":"CaseAttributeValue","Attribute":"Product Group","StringifiedValues":["0Hats"]}]}]},

   "MaxNumUniqueValues": 2,

   "MaximumRowCount": 1000,

   "MinValueUsage": 0.20,

   "WeightingExpression": "Column(\"Cost\")"

   }

},

"Ordering": [

   {"Name": "Contribution", "Direction": "Descending"}

]

}

Calculates root cause analysis for given model using parameters read from the query configuration.

== Utils.GetSampledEvents ==
foo

== Utils.RunFunctionWithParallelLogging ==
foo

System Library

2024-11-19T12:23:36Z

MarHink: /* Parameters */

System library is a collection of Expression Language functions and properties that provide additional tools for scripting Process Analyzer functionalities. System library is referenced in scripts via ''_system''-property, which provides additional properties dedicated for different areas of interests for scripting.

The following hierarchy shows the properties and functions available in System Library:

* ML
** [[Create Predicted Eventlog|GeneratePredictionModel]]
** [[Create Simulated Eventlog|ApplyTransformations]]
* Parallel
** Run
* RootCauses
** FindRootCausesDataFrame
* Utils
** GetSampledEvents
** ModifyColumnTypes
** RunFunctionWithParallelLogging

== Parallel.Run ==
Runs given functions in parallel.

=== Parameters ===
* '''functions''':
** An array of functions to run in parallel.

=== Return value ===
An array of results returned by the called functions, in the same order as the function generating them in the ''functions''-parameter.

=== Example ===
The following script uses _system.Parallel.Run to run three functions:

* SAP-extraction from VBAK-table in SAP (connection parameters defined in connectionParametersDict-dictionary).
* Transform the extracted data by adding a new column.
* Load the data into data table identified by dataTableId.
<syntaxhighlight lang="typescript">
function ExtractTransformAndLoad(extractFunc, transformFunc, loadFunc)
{
let rawDataFlow = extractFunc();
let transformedDataFlow = ToDataFlow();

_system.Parallel.Run([
() => Catch({
let df;
while (!IsNullTop(df = rawDataFlow.Collect(#{"CollectChunk": true}))) {
transformedDataFlow.Append(transformFunc(df));
WriteLog(`A chunk having ${df.NRows} rows has been transformed.`);
}
if (rawDataFlow.HasError) {
transformedDataFlow.Fail("Error occurred during data extraction.");
}
else {
transformedDataFlow.Complete();
}
}, {
transformedDataFlow.Fail("Error occurred during transformation calculation.");
}),
() => {
loadFunc(transformedDataFlow);
}
]);
}

ExtractTransformAndLoad(
() => ExtractSap(connectionParametersDict.Extend(
[
"FieldNames": "VBELN,ERDAT,ERZET,ERNAM,NETWR,WAERK",
"QueryTable": "VBAK",
"Options": ["VBELN BETWEEN '0000017448'" ,"AND '0060000042'"],
"UseGateway": true
])
),
df => df.SetColumns(["Test": () => `${Column("NETWR")} ${Column("WAERK")}`]),
dataFlow => {
DataTableById(dataTableId).Import(dataFlow, ["Append": 0]);
}
);

DataTableById(dataTableId).SqlDataFrame.OrderByColumns(["VBELN"], [true]).Collect()
</syntaxhighlight>

== RootCauses.FindRootCausesDataFrame ==
Finds root causes for a particular process phenomenon by comparing properties of selected cases against those of all cases in given model (#27619#).

Based on the similar in-memory function: [[FindRootCauses Function|EventLog.FindRootCauses]].

=== Parameters ===

* model:
** Model object of a model whose data tables are used to calculate the root causes.
* parameters:
** A parameter convertible to a StringDictionary object with the following supported key-values:
*** Filter:
**** Filter json (#30921#) that is applied to the event log before calculating root causes.
*** Selection:
**** Selection json (#30927#) that defines selected cases to find root causes for. Is applied on top of the filtered event log (specified by Filter parameter).
**** If nothing is selected, 100% of cases are counted as selected.
*** CaseAttributeTypes:
**** An array of strings with the names of case attributes included into the root causes.
**** Only case attributes of type string, integer or boolean are included.
**** If CaseAttributeTypes is null or empty string or not specified, all case attributes having type "String" are included into the root causes.
**** If the model doesn't have a case attribute with the specified name, an error message is shown.
*** EventAttributeTypes:
**** An array of strings with the names of event attributes included into the root causes.
**** Only event attributes of type string are included.
**** If EventAttributeTypes is null, all event attributes having type "String" are included into the root causes.
**** If EventAttributeTypes is empty array or not specified, event attributes not applied to the root causes.
**** If the model doesn't have an event attribute with the specified name, an error message is shown.
**** Analysis column Name should contain Event Attribute Name.
**** Analysis column Value should contain Event Attribute Value and number of occurrences in case: <value> (count).
**** Analysis column Type should have "EventAttributeValue" as its value.
*** WeightingExpression:
**** Expression providing weights for each case.
**** Expression, if defined, must be any of the following types:
***** A string containing the [[SQL Expressions|SqlExpression]] to evaluate.
***** A SqlExpression object, created e.g., using ToSqlExpression-function (see also: [[QPR_ProcessAnalyzer_Expressions#In-memory_expression_blocks_in_SQL_expressions|In-memory expression blocks in SQL expressions]]).
**** A row is filtered out of result if expression result is null.

2.2.1.6 MaximumRowCount: maximum number of the most and least contributing root causes to return. Thus, the actual number of returned rows can be at most two times this value (if specified).

2.2.1.6.1. If undefined, 200 is used.

2.2.1.6.2. If set to 0, all rows are returned.

2.2.1.7. MinValueUsage: Minimum total usage of a value included into the comparison. The number of cases having every returned value should be at least given percentage (a float value between 0.0 and 1.0) of all the compared cases.

2.2.1.7.1. If not defined or null, all values are included (=default).

2.2.1.8. MaxNumUniqueValues: Maximum number of unique values to include into the comparison for each attribute column. If the amount of unique values for any attribute exceeds this value, only given number of attributes are included that have the highest usage.

2.2.1.8.1. If not defined or null, all values are included (=default).

2.2.1.9. IncludeOthers: Should the rest of the attribute values not included due to MinValueUsage or MaxNumUniqueValues filtering be included as an aggregated "Others" value?

2.2.1.9.1. If given any string value, that value is used as the value for all the aggregated values.

2.2.1.9.2. Default = undefined => others values will not be included into the results.

2.2.1.10. ValueIfNull: Value used to indicate null-values.

2.2.1.10.1. Default = "(blank)".

2.2.1.10.2. Must be not null.

3 Returns a SqlDataFrame object (#70611#) with the following columns:

3.1 Common columns

3.1.1 Type: type of the root cause

3.1.1.1 "CaseAttributeValue" for case attributes

3.1.1.2 "EventAttributeValue" for event attributes

3.1.2 Name:

3.1.2.1 When type is CaseAttributeValue, case attribute name

3.1.2.2 When type is EventAttributeValue, event attribute name

3.1.3 Value:

3.1.3.1 When type is CaseAttributeValue, case attribute value

3.1.3.2 When type is EventAttributeValue, event attribute value and number of occurrences in case

3.1.4 Total: total number of cases having the found root cause

3.1.5 Selected: Number of cases that have the found root cause and belong to the selected cases

3.1.6 Compared: number of cases that have the found root cause and don't belong to the selected cases

3.2 Columns when WeightingExpression not have value

3.2.1 Contribution: the number of cases which contribute to the deviation from the average percentage of selected cases among all analyzed cases

3.2.2 ContributionPercentage: the percent of cases which contribute to the deviation from the average percentage

3.2.3 DifferencePercentage: deviation in percentage between selected cases with the found root cause and the average percentage of selected cases among all analyzed cases

3.1.4 SelectedPercentage: percent of selected cases that have the found root cause out of all cases with that root cause

3.3 Columns when WeightingExpression have value

3.3.1 Contribution: Sum of case weights which contribute to the deviation from the average percentage of selected case weights among all analyzed cases

3.3.2 ContributionPercentage: the percent of case weights which contribute to the deviation from the average percentage

3.3.3 DifferencePercentage: deviation in percentage between selected case weights with the found root cause and the average percentage of selected case weights among all analyzed cases

3.3.4 SelectedPercentage: percent of selected case weights that have the found root cause out of all case weights with that root cause

3.3.5 SelectedWeight: Sum of weights that have the found root cause and belong to the selected cases

3.3.6 ComparedWeight: Sum of weights that have the found root cause and don't belong to the selected cases

3.3.7 TotalWeight: Sum of weights of all cases with that root cause

4. Notable differences between this function and FindRootCauses (#71058#) used for in-memory event logs:

4.1. In FindRootCauses, Zero-weighted contibution rows are removed from the results.

4.2. Non-numeric case cost in FindForDataFrame causes and exception to be thrown. Not in FindRootCauses, where is behaves as if null was used.

4.3. All the case-data table columns can be used as CaseAttributeTypes in FindForDataFrame, even the case id column, which can't be used in FindRootCauses.

4.4. WeightingExpression is given as SqlExpression, not expression language expression as in FindRootCauses.

4.5. FindForDataFrame only supports querying one type of columns at a time. Querying, e.g., columns having both string and integer values will throw an exception.

4.6. If an empty array is given for CaseAttributeTypes-parameter, FindForDataFrame throws an exception, whereas FindRootCauses returns a result without any rows.

4.7. FindForDataFrame converts attribute values to string, because internally it uses UNPIVOT instruction for calculating Value column (requires same data type for unpivoted columns). So even if numeric case attribute is used, Value column will contain strings and sorting by Value will use string comparison.

Examples:

{

"ProcessingMethod": "DataFrame",

"Root":"let m = _; _system.RootCauses.'''FindForDataFrame'''(m, _query.Configuration.Parameters.FindRootCausesParameters.Clone().Extend(#{\"Filter\": _query.Configuration.Filter}))",

"Parameters": {

   "FindRootCausesParameters": {

   "CaseAttributeTypes": ["Account Manager","Customer Group","Product Group","Region"],

"Selection": {"Items":[{"Type":"IncludeCases","Items":[{"Type":"CaseAttributeValue","Attribute":"Product Group","StringifiedValues":["0Hats"]}]}]},

   "MaxNumUniqueValues": 2,

   "MaximumRowCount": 1000,

   "MinValueUsage": 0.20,

   "WeightingExpression": "Column(\"Cost\")"

   }

},

"Ordering": [

   {"Name": "Contribution", "Direction": "Descending"}

]

}

Calculates root cause analysis for given model using parameters read from the query configuration.

== Utils.GetSampledEvents ==
foo

== Utils.RunFunctionWithParallelLogging ==
foo

System Library

2024-11-19T12:22:45Z

MarHink: /* Returns */

System library is a collection of Expression Language functions and properties that provide additional tools for scripting Process Analyzer functionalities. System library is referenced in scripts via ''_system''-property, which provides additional properties dedicated for different areas of interests for scripting.

The following hierarchy shows the properties and functions available in System Library:

* ML
** [[Create Predicted Eventlog|GeneratePredictionModel]]
** [[Create Simulated Eventlog|ApplyTransformations]]
* Parallel
** Run
* RootCauses
** FindRootCausesDataFrame
* Utils
** GetSampledEvents
** ModifyColumnTypes
** RunFunctionWithParallelLogging

== Parallel.Run ==
Runs given functions in parallel.

=== Parameters ===
* '''functions''':
** An array of functions to run in parallel.

=== Return value ===
An array of results returned by the called functions, in the same order as the function generating them in the ''functions''-parameter.

=== Example ===
The following script uses _system.Parallel.Run to run three functions:

* SAP-extraction from VBAK-table in SAP (connection parameters defined in connectionParametersDict-dictionary).
* Transform the extracted data by adding a new column.
* Load the data into data table identified by dataTableId.
<syntaxhighlight lang="typescript">
function ExtractTransformAndLoad(extractFunc, transformFunc, loadFunc)
{
let rawDataFlow = extractFunc();
let transformedDataFlow = ToDataFlow();

_system.Parallel.Run([
() => Catch({
let df;
while (!IsNullTop(df = rawDataFlow.Collect(#{"CollectChunk": true}))) {
transformedDataFlow.Append(transformFunc(df));
WriteLog(`A chunk having ${df.NRows} rows has been transformed.`);
}
if (rawDataFlow.HasError) {
transformedDataFlow.Fail("Error occurred during data extraction.");
}
else {
transformedDataFlow.Complete();
}
}, {
transformedDataFlow.Fail("Error occurred during transformation calculation.");
}),
() => {
loadFunc(transformedDataFlow);
}
]);
}

ExtractTransformAndLoad(
() => ExtractSap(connectionParametersDict.Extend(
[
"FieldNames": "VBELN,ERDAT,ERZET,ERNAM,NETWR,WAERK",
"QueryTable": "VBAK",
"Options": ["VBELN BETWEEN '0000017448'" ,"AND '0060000042'"],
"UseGateway": true
])
),
df => df.SetColumns(["Test": () => `${Column("NETWR")} ${Column("WAERK")}`]),
dataFlow => {
DataTableById(dataTableId).Import(dataFlow, ["Append": 0]);
}
);

DataTableById(dataTableId).SqlDataFrame.OrderByColumns(["VBELN"], [true]).Collect()
</syntaxhighlight>

== RootCauses.FindRootCausesDataFrame ==
Finds root causes for a particular process phenomenon by comparing properties of selected cases against those of all cases in given model (#27619#).

Based on the similar in-memory function: [[FindRootCauses Function|EventLog.FindRootCauses]].

=== Parameters ===

* model:
** Model object of a model whose data tables are used to calculate the root causes.
* parameters:
** A parameter convertible to a StringDictionary object with the following supported key-values:
*** Filter:
**** Filter json (#30921#) that is applied to the event log before calculating root causes.
*** Selection:
**** Selection json (#30927#) that defines selected cases to find root causes for. Is applied on top of the filtered event log (specified by Filter parameter).
**** If nothing is selected, 100% of cases are counted as selected.
*** CaseAttributeTypes:
**** An array of strings with the names of case attributes included into the root causes.
**** Only case attributes of type string, integer or boolean are included.
**** If CaseAttributeTypes is null or empty string or not specified, all case attributes having type "String" are included into the root causes.
**** If the model doesn't have a case attribute with the specified name, an error message is shown.
*** EventAttributeTypes:
**** An array of strings with the names of event attributes included into the root causes.
**** Only event attributes of type string are included.
**** If EventAttributeTypes is null, all event attributes having type "String" are included into the root causes.
**** If EventAttributeTypes is empty array or not specified, event attributes not applied to the root causes.
**** If the model doesn't have an event attribute with the specified name, an error message is shown.
**** Analysis column Name should contain Event Attribute Name.
**** Analysis column Value should contain Event Attribute Value and number of occurrences in case: <value> (count).
**** Analysis column Type should have "EventAttributeValue" as its value.
*** WeightingExpression:
**** Expression providing weights for each case.
**** Expression, if defined, must be any of the following types:
***** A string containing the [[SQL Expressions|SqlExpression]] to evaluate.
***** A SqlExpression object, created e.g., using ToSqlExpression-function (see also: [[QPR ProcessAnalyzer Expressions#In-memory expression blocks in SQL expressions|In-memory expression blocks in SQL expressions]]).
**** A row is filtered out of result if expression result is null.

2.2.1.6 MaximumRowCount: maximum number of the most and least contributing root causes to return. Thus, the actual number of returned rows can be at most two times this value (if specified).

2.2.1.6.1. If undefined, 200 is used.

2.2.1.6.2. If set to 0, all rows are returned.

2.2.1.7. MinValueUsage: Minimum total usage of a value included into the comparison. The number of cases having every returned value should be at least given percentage (a float value between 0.0 and 1.0) of all the compared cases.

2.2.1.7.1. If not defined or null, all values are included (=default).

2.2.1.8. MaxNumUniqueValues: Maximum number of unique values to include into the comparison for each attribute column. If the amount of unique values for any attribute exceeds this value, only given number of attributes are included that have the highest usage.

2.2.1.8.1. If not defined or null, all values are included (=default).

2.2.1.9. IncludeOthers: Should the rest of the attribute values not included due to MinValueUsage or MaxNumUniqueValues filtering be included as an aggregated "Others" value?

2.2.1.9.1. If given any string value, that value is used as the value for all the aggregated values.

2.2.1.9.2. Default = undefined => others values will not be included into the results.

2.2.1.10. ValueIfNull: Value used to indicate null-values.

2.2.1.10.1. Default = "(blank)".

2.2.1.10.2. Must be not null.

3 Returns a SqlDataFrame object (#70611#) with the following columns:

3.1 Common columns

3.1.1 Type: type of the root cause

3.1.1.1 "CaseAttributeValue" for case attributes

3.1.1.2 "EventAttributeValue" for event attributes

3.1.2 Name:

3.1.2.1 When type is CaseAttributeValue, case attribute name

3.1.2.2 When type is EventAttributeValue, event attribute name

3.1.3 Value:

3.1.3.1 When type is CaseAttributeValue, case attribute value

3.1.3.2 When type is EventAttributeValue, event attribute value and number of occurrences in case

3.1.4 Total: total number of cases having the found root cause

3.1.5 Selected: Number of cases that have the found root cause and belong to the selected cases

3.1.6 Compared: number of cases that have the found root cause and don't belong to the selected cases

3.2 Columns when WeightingExpression not have value

3.2.1 Contribution: the number of cases which contribute to the deviation from the average percentage of selected cases among all analyzed cases

3.2.2 ContributionPercentage: the percent of cases which contribute to the deviation from the average percentage

3.2.3 DifferencePercentage: deviation in percentage between selected cases with the found root cause and the average percentage of selected cases among all analyzed cases

3.1.4 SelectedPercentage: percent of selected cases that have the found root cause out of all cases with that root cause

3.3 Columns when WeightingExpression have value

3.3.1 Contribution: Sum of case weights which contribute to the deviation from the average percentage of selected case weights among all analyzed cases

3.3.2 ContributionPercentage: the percent of case weights which contribute to the deviation from the average percentage

3.3.3 DifferencePercentage: deviation in percentage between selected case weights with the found root cause and the average percentage of selected case weights among all analyzed cases

3.3.4 SelectedPercentage: percent of selected case weights that have the found root cause out of all case weights with that root cause

3.3.5 SelectedWeight: Sum of weights that have the found root cause and belong to the selected cases

3.3.6 ComparedWeight: Sum of weights that have the found root cause and don't belong to the selected cases

3.3.7 TotalWeight: Sum of weights of all cases with that root cause

4. Notable differences between this function and FindRootCauses (#71058#) used for in-memory event logs:

4.1. In FindRootCauses, Zero-weighted contibution rows are removed from the results.

4.2. Non-numeric case cost in FindForDataFrame causes and exception to be thrown. Not in FindRootCauses, where is behaves as if null was used.

4.3. All the case-data table columns can be used as CaseAttributeTypes in FindForDataFrame, even the case id column, which can't be used in FindRootCauses.

4.4. WeightingExpression is given as SqlExpression, not expression language expression as in FindRootCauses.

4.5. FindForDataFrame only supports querying one type of columns at a time. Querying, e.g., columns having both string and integer values will throw an exception.

4.6. If an empty array is given for CaseAttributeTypes-parameter, FindForDataFrame throws an exception, whereas FindRootCauses returns a result without any rows.

4.7. FindForDataFrame converts attribute values to string, because internally it uses UNPIVOT instruction for calculating Value column (requires same data type for unpivoted columns). So even if numeric case attribute is used, Value column will contain strings and sorting by Value will use string comparison.

Examples:

{

"ProcessingMethod": "DataFrame",

"Root":"let m = _; _system.RootCauses.'''FindForDataFrame'''(m, _query.Configuration.Parameters.FindRootCausesParameters.Clone().Extend(#{\"Filter\": _query.Configuration.Filter}))",

"Parameters": {

   "FindRootCausesParameters": {

   "CaseAttributeTypes": ["Account Manager","Customer Group","Product Group","Region"],

"Selection": {"Items":[{"Type":"IncludeCases","Items":[{"Type":"CaseAttributeValue","Attribute":"Product Group","StringifiedValues":["0Hats"]}]}]},

   "MaxNumUniqueValues": 2,

   "MaximumRowCount": 1000,

   "MinValueUsage": 0.20,

   "WeightingExpression": "Column(\"Cost\")"

   }

},

"Ordering": [

   {"Name": "Contribution", "Direction": "Descending"}

]

}

Calculates root cause analysis for given model using parameters read from the query configuration.

== Utils.GetSampledEvents ==
foo

== Utils.RunFunctionWithParallelLogging ==
foo

System Library

2024-11-19T10:29:58Z

MarHink: /* RootCauses.FindRootCausesDataFrame */

System library is a collection of Expression Language functions and properties that provide additional tools for scripting Process Analyzer functionalities. System library is referenced in scripts via ''_system''-property, which provides additional properties dedicated for different areas of interests for scripting.

The following hierarchy shows the properties and functions available in System Library:

* ML
** [[Create Predicted Eventlog|GeneratePredictionModel]]
** [[Create Simulated Eventlog|ApplyTransformations]]
* Parallel
** Run
* RootCauses
** FindRootCausesDataFrame
* Utils
** GetSampledEvents
** ModifyColumnTypes
** RunFunctionWithParallelLogging

== Parallel.Run ==
Runs given functions in parallel.

=== Parameters ===
* '''functions''':
** An array of functions to run in parallel.

=== Returns ===
An array of results returned by the called functions, in the same order as the function generating them in the ''functions''-parameter.

=== Example ===
The following script uses _system.Parallel.Run to run three functions:

* SAP-extraction from VBAK-table in SAP (connection parameters defined in connectionParametersDict-dictionary).
* Transform the extracted data by adding a new column.
* Load the data into data table identified by dataTableId.
<syntaxhighlight lang="typescript">
function ExtractTransformAndLoad(extractFunc, transformFunc, loadFunc)
{
let rawDataFlow = extractFunc();
let transformedDataFlow = ToDataFlow();

_system.Parallel.Run([
() => Catch({
let df;
while (!IsNullTop(df = rawDataFlow.Collect(#{"CollectChunk": true}))) {
transformedDataFlow.Append(transformFunc(df));
WriteLog(`A chunk having ${df.NRows} rows has been transformed.`);
}
if (rawDataFlow.HasError) {
transformedDataFlow.Fail("Error occurred during data extraction.");
}
else {
transformedDataFlow.Complete();
}
}, {
transformedDataFlow.Fail("Error occurred during transformation calculation.");
}),
() => {
loadFunc(transformedDataFlow);
}
]);
}

ExtractTransformAndLoad(
() => ExtractSap(connectionParametersDict.Extend(
[
"FieldNames": "VBELN,ERDAT,ERZET,ERNAM,NETWR,WAERK",
"QueryTable": "VBAK",
"Options": ["VBELN BETWEEN '0000017448'" ,"AND '0060000042'"],
"UseGateway": true
])
),
df => df.SetColumns(["Test": () => `${Column("NETWR")} ${Column("WAERK")}`]),
dataFlow => {
DataTableById(dataTableId).Import(dataFlow, ["Append": 0]);
}
);

DataTableById(dataTableId).SqlDataFrame.OrderByColumns(["VBELN"], [true]).Collect()
</syntaxhighlight>

== RootCauses.FindRootCausesDataFrame ==
1 Finds root causes for a particular process phenomenon by comparing properties of selected cases against those of all cases in given model (#27619#).

1.1. Based on the similar in-memory function: EventLog.FindRootCauses (#71058#).

1.2. Based on IDataFrame (#70613#)-based calculations, which, in case of SqlDataFrame (#70611#), offload the acctual work to the data sources (#70864#, e.g., Snowflake).

2. Parameters:

2.1. model:

2.1.1. Model object of a model whose data tables are used to calculate the root causes.

2.2. parameters

2.2.1. A parameter convertible to a StringDictionary (#48323#) object with the following supported key-values:

2.2.1.1 Filter: filter json (#30921#) that is applied to the event log before calculating root causes

2.2.1.2 Selection: selection json (#30927#) that defines selected cases to find root causes for

2.2.1.2.1 Selection is applied on top of the filtered event log (specified by Filter parameter)

2.2.1.2.2 If nothing is selected, 100% of cases are counted as selected.

2.2.1.3 CaseAttributeTypes: array of strings with the names of case attributes included into the root causes

2.2.1.3.1 Only case attributes of type string, integer or boolean are included

2.2.1.3.2 If CaseAttributeTypes is null or empty string or not specified, all case attributes having type "String" are included into the root causes

2.2.1.3.3 If the model doesn't have a case attribute with the specified name, an error message is shown

2.2.1.4 EventAttributeTypes: array of strings with the names of event attributes included into the root causes

2.2.1.4.1 Only event attributes of type string are included

2.2.1.4.2 If EventAttributeTypes is null, all event attributes having type "String" are included into the root causes

2.2.1.4.3 If EventAttributeTypes is empty array or not specified, event attributes not applied to the root causes

2.2.1.4.4 If the model doesn't have an event attribute with the specified name, an error message is shown

2.2.1.4.5 Analysis column Name should contain Event Attribute Name

2.2.1.4.6 Analysis column Value should contain Event Attribute Value and number of occurrences in case: <value> (count)

2.2.1.4.7 Analysis column Type should have "EventAttributeValue" as its value

2.2.1.5 WeightingExpression: Expression provide weights for each case.

2.2.1.5.1 Expression, if defined, must be any of the following types:

2.2.1.5.1.1. A string containing the SqlExpression (#70558#).

2.2.1.5.1.1.1. Supports extended functionalities specified here #70773#.

2.2.1.5.1.1. A SqlExpression object, created e.g., using ToSqlExpression-function (#71302#).

2.2.1.5.2 A row is filtered out of result if expression result is null.

2.2.1.6 MaximumRowCount: maximum number of the most and least contributing root causes to return. Thus, the actual number of returned rows can be at most two times this value (if specified).

2.2.1.6.1. If undefined, 200 is used.

2.2.1.6.2. If set to 0, all rows are returned.

2.2.1.7. MinValueUsage: Minimum total usage of a value included into the comparison. The number of cases having every returned value should be at least given percentage (a float value between 0.0 and 1.0) of all the compared cases.

2.2.1.7.1. If not defined or null, all values are included (=default).

2.2.1.8. MaxNumUniqueValues: Maximum number of unique values to include into the comparison for each attribute column. If the amount of unique values for any attribute exceeds this value, only given number of attributes are included that have the highest usage.

2.2.1.8.1. If not defined or null, all values are included (=default).

2.2.1.9. IncludeOthers: Should the rest of the attribute values not included due to MinValueUsage or MaxNumUniqueValues filtering be included as an aggregated "Others" value?

2.2.1.9.1. If given any string value, that value is used as the value for all the aggregated values.

2.2.1.9.2. Default = undefined => others values will not be included into the results.

2.2.1.10. ValueIfNull: Value used to indicate null-values.

2.2.1.10.1. Default = "(blank)".

2.2.1.10.2. Must be not null.

3 Returns a SqlDataFrame object (#70611#) with the following columns:

3.1 Common columns

3.1.1 Type: type of the root cause

3.1.1.1 "CaseAttributeValue" for case attributes

3.1.1.2 "EventAttributeValue" for event attributes

3.1.2 Name:

3.1.2.1 When type is CaseAttributeValue, case attribute name

3.1.2.2 When type is EventAttributeValue, event attribute name

3.1.3 Value:

3.1.3.1 When type is CaseAttributeValue, case attribute value

3.1.3.2 When type is EventAttributeValue, event attribute value and number of occurrences in case

3.1.4 Total: total number of cases having the found root cause

3.1.5 Selected: Number of cases that have the found root cause and belong to the selected cases

3.1.6 Compared: number of cases that have the found root cause and don't belong to the selected cases

3.2 Columns when WeightingExpression not have value

3.2.1 Contribution: the number of cases which contribute to the deviation from the average percentage of selected cases among all analyzed cases

3.2.2 ContributionPercentage: the percent of cases which contribute to the deviation from the average percentage

3.2.3 DifferencePercentage: deviation in percentage between selected cases with the found root cause and the average percentage of selected cases among all analyzed cases

3.1.4 SelectedPercentage: percent of selected cases that have the found root cause out of all cases with that root cause

3.3 Columns when WeightingExpression have value

3.3.1 Contribution: Sum of case weights which contribute to the deviation from the average percentage of selected case weights among all analyzed cases

3.3.2 ContributionPercentage: the percent of case weights which contribute to the deviation from the average percentage

3.3.3 DifferencePercentage: deviation in percentage between selected case weights with the found root cause and the average percentage of selected case weights among all analyzed cases

3.3.4 SelectedPercentage: percent of selected case weights that have the found root cause out of all case weights with that root cause

3.3.5 SelectedWeight: Sum of weights that have the found root cause and belong to the selected cases

3.3.6 ComparedWeight: Sum of weights that have the found root cause and don't belong to the selected cases

3.3.7 TotalWeight: Sum of weights of all cases with that root cause

4. Notable differences between this function and FindRootCauses (#71058#) used for in-memory event logs:

4.1. In FindRootCauses, Zero-weighted contibution rows are removed from the results.

4.2. Non-numeric case cost in FindForDataFrame causes and exception to be thrown. Not in FindRootCauses, where is behaves as if null was used.

4.3. All the case-data table columns can be used as CaseAttributeTypes in FindForDataFrame, even the case id column, which can't be used in FindRootCauses.

4.4. WeightingExpression is given as SqlExpression, not expression language expression as in FindRootCauses.

4.5. FindForDataFrame only supports querying one type of columns at a time. Querying, e.g., columns having both string and integer values will throw an exception.

4.6. If an empty array is given for CaseAttributeTypes-parameter, FindForDataFrame throws an exception, whereas FindRootCauses returns a result without any rows.

4.7. FindForDataFrame converts attribute values to string, because internally it uses UNPIVOT instruction for calculating Value column (requires same data type for unpivoted columns). So even if numeric case attribute is used, Value column will contain strings and sorting by Value will use string comparison.

Examples:

{

"ProcessingMethod": "DataFrame",

"Root":"let m = _; _system.RootCauses.'''FindForDataFrame'''(m, _query.Configuration.Parameters.FindRootCausesParameters.Clone().Extend(#{\"Filter\": _query.Configuration.Filter}))",

"Parameters": {

   "FindRootCausesParameters": {

   "CaseAttributeTypes": ["Account Manager","Customer Group","Product Group","Region"],

"Selection": {"Items":[{"Type":"IncludeCases","Items":[{"Type":"CaseAttributeValue","Attribute":"Product Group","StringifiedValues":["0Hats"]}]}]},

   "MaxNumUniqueValues": 2,

   "MaximumRowCount": 1000,

   "MinValueUsage": 0.20,

   "WeightingExpression": "Column(\"Cost\")"

   }

},

"Ordering": [

   {"Name": "Contribution", "Direction": "Descending"}

]

}

Calculates root cause analysis for given model using parameters read from the query configuration.

== Utils.GetSampledEvents ==
foo

== Utils.RunFunctionWithParallelLogging ==
foo

System Library

2024-11-19T10:26:59Z

MarHink: /* Parameters */