|
|
| (80 intermediate revisions by 4 users not shown) |
| Line 1: |
Line 1: |
| Each QPR ProcessAnalyzer model has a '''[[Project_Workspace_in_QPR_ProcessAnalyzer_Excel_Client#Model_Properties|Configuration]]''' field containing model related settings in a JSON format. Those settings are documented in this page. When the model JSON configuration is changed, the model is dropped from the memory.
| | In addition to [[Roles_and_Permissions|project level permissions]], it's possible to set '''case level permissions''' for models, where visibility can be defined for each case separately based on rules. Usually the rules are based on case attribute values. Case level permissions are effective for all users, but users who have the '''GenericWrite''' permission, can change the case permissions setting for a model and thus change the case permissions. |
|
| |
|
| == Memory Usage Settings == | | == Setting up== |
| The memory usage settings are used to manage, how long objects are kept in the memory, which affect both the memory usage and performance. The longer the object storing durations, the more memory is consumed, but on the other hand users perceive better performance as analyses are more likely to be found in the memory already calculated. Memory usage settings can be defined in the [[Web.config_file_in_QPR_ProcessAnalyzer#Qpr.ProcessAnalyzer.Common.Properties.Settings_Section|web.config]] file and for each model separately in the model settings. | | Case level permissions are configured in the model properties (can be opened in the models list in the header) in the '''Case Level Permissions''' tab. The following settings are available: |
| | * '''Initialization expression''': [[QPR_ProcessAnalyzer_Expressions|Expression]] that is run only once and can be used as initializing needed objects. This expression can be used to improve performance when part of the Case or EventLogKey expressions are common and thus they don't need to be calculated again for every Case separately. |
| | * '''Permissions expression''': Expression determining which cases the accessing user can see. The expression is evaluated for each case (in the case context). If the evaluation results ''true'', the case is visible for the user; otherwise not. The evaluation is done when no existing eventlog if found based on the ''Eventlog key'' and a new eventlog needs to be generated. This expression is mandatory for the case level permissions to work |
| | * '''Eventlog key expression''': Expression used to generate a uniquely identifying string for the eventlogs when the case level permissions are in use. If a cached eventLog with the same key already exists in the system, that eventlog is used instead of creating a new. |
|
| |
|
| {| class="wikitable"
| | == Configuration examples == |
| !'''Property'''
| | Visibility of cases is limited in a way that only those users can see cases belonging to a group which name is same as the ''Region'' (case attribute). |
| ! '''Description'''
| | * Initialization expression: '''let groupNames = OrderByValue(CurrentUser.GroupNames)''' |
| |-
| | * Permissions expression: '''Region.In(groupNames)''' |
| ||
| | * Eventlog key: '''StringJoin("_", groupNames)''' |
| CacheUsage/
| |
| DropUnusedModelsAfter
| |
| ||
| |
| Duration after which the unused model, is dropped from the memory. Defined in format ''HH:mm:ss'' or ''d.HH:mm:ss'', for example 01:00:00 (one hour), 00:30:00 (30 minutes) or 1.00:00:00 (24 hours). When an analysis is requested for a model, the model's last used time is updated (also filter's last used time is updated).
| |
| | |
| If this setting is not defined, the server level [[Web.config_file_in_QPR_ProcessAnalyzer#Qpr.ProcessAnalyzer.Common.Properties.Settings_Section|default setting]] is used.
| |
| |-
| |
| ||
| |
| CacheUsage/
| |
| DropUnusedFiltersAfter
| |
| ||
| |
| Duration after which the model's unused filters, are dropped from the memory. Defined in format ''HH:mm:ss'' or ''d.HH:mm:ss'', for example 01:00:00 (one hour), 00:30:00 (30 minutes) or 1.00:00:00 (24 hours). When an analysis is requested for a filter , the filter's last used time is updated (also model's last used time is updated).
| |
| | |
| If this setting is not defined, the server level [[Web.config_file_in_QPR_ProcessAnalyzer#Qpr.ProcessAnalyzer.Common.Properties.Settings_Section|default setting]] is used.
| |
| |}
| |
| | |
| ===Examples=== | |
| The following configuration keeps the model in memory for 1 hour and filters for 30 minutes.
| |
| <pre>
| |
| {
| |
| "CacheUsage": {
| |
| "DropUnusedModelsAfter": "1:00:00",
| |
| "DropUnusedFiltersAfter": "00:30:00"
| |
| }
| |
| }
| |
| </pre>
| |
| | |
| The following configuration keeps the model in memory for 15 minutes and filters for 5 minutes.
| |
| <pre>
| |
| {
| |
| "CacheUsage": {
| |
| "DropUnusedModelsAfter": "0:15:00",
| |
| "DropUnusedFiltersAfter": "00:05:00"
| |
| }
| |
| }
| |
| </pre>
| |
| | |
| The following configuration keeps the model in memory for 100 days (practically it's never dropped) and filters for 2 hours.
| |
| <pre>
| |
| {
| |
| "CacheUsage": {
| |
| "DropUnusedModelsAfter": "100.00:00:00",
| |
| "DropUnusedFiltersAfter": "02:00:00"
| |
| }
| |
| }
| |
| </pre>
| |
| | |
| Data handled by QPR ProcessAnalyzer memory is stored in following types of objects:
| |
| {| class="wikitable"
| |
| !'''Stored Objects'''
| |
| ! '''Contents'''
| |
| ! '''Unused objects dropped after'''
| |
| ! '''Time to reload/calculate'''
| |
| ! '''Memory consumption'''
| |
| |-
| |
| ||Models
| |
| ||Models objects contain all data in a QPR ProcessAnalyzer model, such as
| |
| * cases
| |
| * events
| |
| * variations
| |
| * event types
| |
| * flows
| |
| * flow occurrences
| |
| ||Primarily defined by a model setting ''CacheUsage/DropUnusedModelsAfter''. If the model setting is not defined, the server level setting in the web.config file is used. If the server setting is not defined either, a default value of 1 hour is used. When the is a memory shortage in the server, models are not dropped from the memory.
| |
| ||'''Slow'''
| |
| | |
| Models are loaded from the database requiring to transfer considerable amount of data, which takes much more time than e.g. calculating filters and analyses.
| |
| ||'''High'''
| |
| |-
| |
| ||Filters
| |
| ||Filters contain all filtered data in QPR ProcessAnalyzer model, which includes similar data and structures as the models.
| |
| ||Primarily defined by a model setting. If the model setting is not defined, the server setting is used. If the server setting is not defined, a default value of 30 minutes is used. When the is a memory shortage in the server, filters may be dropped from the memory to free memory.
| |
| ||'''Fast'''
| |
| | |
| Filters are calculated from the model data that already exists in the memory. Practically, filters are subsets of models.
| |
| ||'''Medium'''
| |
| |-
| |
| ||Analysis results
| |
| ||Results of the analysis (in tabular form).
| |
| ||Unused analysis results are kept maximum of 15 minutes in the memory. When the is a memory shortage in the server, analysis results may be dropped from the memory to free memory.
| |
| ||'''Fast'''
| |
| | |
| Analysis results are calculated from filters, which already exist in memory.
| |
| ||'''Low'''
| |
| |}
| |
| | |
| If the QPR ProcessAnalyzer server doesn't have enough memory to store more objects, already stored objects are dropped from the memory starting from the one having the longest time from the last usage. Thus, when there is a short of memory, objects may be dropped earlier than their settings define. Only filters and analysis results are dropped during the memory shortage, i.e. model objects are not dropped prematurely. This is because recalculating filters and analysis results is usually faster than loading models from the database. That's why, when trying to load more models than there is available memory in the server, an out of memory error situation may occur.
| |
| | |
| == Model Automatic Loading on Server Startup ==
| |
| QPR ProcessAnalyzer models can be loaded automatically, when QPR ProcessAnalyzer Server starts. When the model loading takes long time, it's useful to load it automatically beforehand. In the model JSON settings, when the '''LoadOnStartup''' property is set to ''true'', the model is loaded automatically during QPR ProcessAnalyzer Server startup. The JSON configuration is as follows:
| |
| <pre>
| |
| {
| |
| "LoadOnStartup": true,
| |
| "CacheUsage": {
| |
| "DropUnusedModelsAfter": "100.00:00:00",
| |
| "DropUnusedFiltersAfter": "00:30:00"
| |
| }
| |
| }
| |
| </pre>
| |
| | |
| Note: For models that are set to load automatically on server startup, you also need to set the ''CacheUsage/DropUnusedModelsAfter'' setting is large value (as shown by the previous example), so that the model is not dropped from memory even if it's not used.
| |
| | |
| Note that ''LoadOnStartup'' setting to work, [[Installing_QPR_ProcessAnalyzer_Server#IIS_Configuration|IIS configurations]] related to QPR ProcessAnalyzer Server installation needs to be in place.
| |
| | |
| == Loading Process Mining Data from ODBC Datasource == | |
| The '''DataSource''' section is used to define where the QPR ProcessAnalyzer model data is loaded. See below examples, how to construct the full JSON.
| |
| {| class="wikitable"
| |
| !'''Property'''
| |
| ! '''Description'''
| |
| |-
| |
| ||DataSource/Cases/DataSourceType
| |
| ||Datasource type to use, when fetching the Cases data. Currently the only supported value is '''odbc'''.
| |
| |-
| |
| ||DataSource/Cases/OdbcConnectionString
| |
| ||ODBC connection string to use to fetch the Cases data. Connection strings can be found in https://www.connectionstrings.com.
| |
| |-
| |
| ||DataSource/Cases/OdbcQuery
| |
| ||ODBC query to use to fetch the Cases data. Query syntax depends on the source system where the data is fetched.
| |
| |-
| |
| ||DataSource/Cases/Columns
| |
| ||Column name mappings for QPR ProcessAnalyzer data model. Only supported mapping is '''CaseId''' defining the case id (case name). Example:
| |
| <pre>
| |
| {
| |
| "CaseId": "SalesOrderHeaderId"
| |
| }
| |
| </pre>
| |
| All unmapped columns are taken to the model as case attributes. If the fetched data doesn't contain a mapped column, the model loading fails and an error message is given.
| |
| |-
| |
| ||DataSource/Events/DataSourceType
| |
| ||Datasource type to use, when fetching the Events data. Currently the only supported value is '''odbc'''.
| |
| |-
| |
| ||DataSource/Events/OdbcConnectionString
| |
| ||ODBC connection string to use to fetch the Events data. Connection strings can be found in https://www.connectionstrings.com.
| |
| |-
| |
| ||DataSource/Events/OdbcQuery
| |
| ||ODBC query to use to fetch the Events data. Query syntax depends on the source system where the data is fetched.
| |
| |-
| |
| ||DataSource/Events/Columns
| |
| ||
| |
| Column name mappings for QPR ProcessAnalyzer data model. Supported mappings for Events are '''CaseId''', '''EventType''' and '''Timestamp'''. Example:
| |
| <pre>
| |
| {
| |
| "CaseId": "SalesOrderHeaderId",
| |
| "EventType": "EventType",
| |
| "Timestamp": "CreatedDate"
| |
| }
| |
| </pre>
| |
| All unmapped columns are taken to the model as event attributes. If the fetched data doesn't contain a mapped column, the model loading fails and an error message is given.
| |
| |}
| |
| | |
| === Configurations examples ===
| |
| | |
| ==== ODBC Driver Installation ====
| |
| '''Microsoft Access Database Engine 2016 Redistributable''' is software package that contains ODBC drivers for
| |
| * Microsoft SQL Server
| |
| * Microsoft Access (*.mdb and *.accdb) files
| |
| * Microsoft Excel (*.xls, *.xlsx, and *.xlsb) files
| |
| * CSV text files
| |
| | |
| The package needs to be installed in the same computer where the QPR ProcessAnalyzer Server is running. Installation instructions:
| |
| # Go to https://www.microsoft.com/en-us/download/details.aspx?id=54920 and click Download.
| |
| # Select whether to use the 32bit or 64bit (x64) version (usually it's the 64bit version).
| |
| # Double-click the executable file on your hard disk to start the setup program.
| |
| # Follow the instructions on the screen to complete the installation.
| |
| | |
| ==== Read data from CSV file ====
| |
| In this example, data is loaded from CSV files located in the file system using ''Microsoft Access Text Driver (*.txt, *.csv)'' driver. Loaded files are C:\ProcessMiningData\ModelCaseAttributes.csv and C:\ProcessMiningData\ModelEventData.csv.
| |
| <pre>
| |
| {
| |
| "DataSource": {
| |
| "Cases": {
| |
| "DataSourceType": "odbc",
| |
| "OdbcConnectionString": "Driver={Microsoft Access Text Driver (*.txt, *.csv)};DefaultDir=C:\\ProcessMiningData\\;Extensions=asc,csv,tab,txt",
| |
| "OdbcQuery": "SELECT * FROM [ModelCaseAttributes.csv]",
| |
| "Columns": {
| |
| "CaseId": "Name"
| |
| }
| |
| },
| |
| "Events": {
| |
| "DataSourceType": "odbc",
| |
| "OdbcConnectionString": "Driver={Microsoft Access Text Driver (*.txt, *.csv)};DefaultDir=C:\\ProcessMiningData\\;Extensions=asc,csv,tab,txt",
| |
| "OdbcQuery": "SELECT * FROM [ModelEventData.csv]",
| |
| "Columns": {
| |
| "CaseId": "Case",
| |
| "EventType": "Event Type",
| |
| "Timestamp": "Start Time"
| |
| }
| |
| }
| |
| }
| |
| }
| |
| </pre>
| |
|
| |
|
| When reading from CSV files, you may need to set the CSV file format for the ODBC driver using the '''Schema.ini''' file (more information: https://docs.microsoft.com/en-us/sql/odbc/microsoft/schema-ini-file-text-file-driver?view=sql-server-2017).
| | Cases are only visible for users whose user name is same as the ''Account Manager'' (case attribute): |
| | * Initialization expression: '''let userName = CurrentUser.Name''' |
| | * Permissions expression: '''Attribute("Account Manager") == userName''' |
| | * Eventlog key: '''CurrentUser.Id''' |
|
| |
|
| ==== Read data from Excel file ==== | | Cases where ''Region'' case attribute is ''Dallas'' will only be visible for users belonging to group ''GroupA'' (and ''New York'' for group ''GroupB''): |
| In this example, data is loaded from a Excel file that is accessible in the file system. In this example, the loaded file is C:\ProcessMiningData\ModelData.xlsx and cases are in sheet ''MyCases'' and events in sheet ''MyEvents''.
| | * Initialization expression: '''let groupNames = CurrentUser.GroupNames''' |
| | * Permissions expression: '''(Region == "Dallas" && "GroupA".In(groupNames)) || (Region == "New York" && "GroupB".In(groupNames))''' |
| | * Eventlog key: '''If("GroupA".In(groupNames), "_A", "_") + If("GroupB".In(groupNames), "_B", "_")''' |
|
| |
|
| <pre>
| | User ''qpr'' can only see cases where ''Region'' is ''Dallas'', and all other users can see all cases: |
| {
| | * Initialization expression: '''let userName = CurrentUser.Name''' |
| "DataSource": {
| | * Permissions expression: '''(userName == "qpr" && Attribute("Region") == "Dallas") || userName != "qpr"''' |
| "Cases": {
| | * Eventlog key: '''CurrentUser.Id''' |
| "DataSourceType": "odbc",
| |
| "OdbcConnectionString": "Driver={Microsoft Excel Driver (*.xls, *.xlsx, *.xlsm, *.xlsb)};DBQ=C:\\ProcessMiningData\\ModelData.xlsx",
| |
| "OdbcQuery": "SELECT * FROM [MyCases$]",
| |
| "Columns": {
| |
| "CaseId": "Case ID"
| |
| }
| |
| },
| |
| "Events": {
| |
| "DataSourceType": "odbc",
| |
| "OdbcConnectionString": "Driver={Microsoft Excel Driver (*.xls, *.xlsx, *.xlsm, *.xlsb)};DBQ=C:\\ProcessMiningData\\ModelData.xlsx",
| |
| "OdbcQuery": "SELECT * FROM [MyEvents$]",
| |
| "Columns": {
| |
| "CaseId": "Case ID",
| |
| "EventType": "Activity",
| |
| "Timestamp": "Start Time"
| |
| }
| |
| }
| |
| }
| |
| }
| |
| </pre>
| |
|
| |
|
| ==== Read data from SQL Server table ==== | | == Use case == |
| In this example, data is loaded from an SQL Server table. In this example, the SQL Server hostname is ''MySQLServer'' and database name is ''MyDatabase''.
| | Case level permissions can be implemented with the principle illustrated in the image below. ''Users'' already belong to certain ''groups'' in the user management, and ''cases'' have certain ''case attribute values'' which is part of the loaded process mining data. Additionally, the linkage between case attribute values (of a certain case attribute) and groups needs to be defined when this security feature is configured. The image below illustrates the chain between ''users'' and ''cases'', how certain users are able to see certain cases when viewing analyses from a QPR ProcessAnalyzer model. |
| | |
| <pre>
| |
| {
| |
| "DataSource": {
| |
| "Cases": {
| |
| "DataSourceType": "odbc",
| |
| "OdbcConnectionString": "Driver={SQL Server};Server=MySQLServer;DataBase=MyDatabase;Trusted_Connection=True;",
| |
| "OdbcQuery": "SELECT CASENAME, COST, COUNTRY, PRODUCT FROM CASES_TABLE",
| |
| "Columns": {
| |
| "CaseId": "CASENAME"
| |
| }
| |
| },
| |
| "Events": {
| |
| "DataSourceType": "odbc",
| |
| "OdbcConnectionString": "Driver={SQL Server};Server=MySQLServer;DataBase=MyDatabase;Trusted_Connection=True;",
| |
| "OdbcQuery": "SELECT CASENAME, CREATED_DATE, CREATED_BY FROM EVENT_TABLE",
| |
| "Columns": {
| |
| "CaseId": "CASENAME",
| |
| "Timestamp": "CREATED_DATE",
| |
| "EventType": "CREATED_BY"
| |
| }
| |
| }
| |
| }
| |
| }
| |
| </pre>
| |
| | |
| === Troubleshooting ===
| |
| | |
| The following error message may be encountered: ''System.Data.Odbc.OdbcException (0x80131937): ERROR [IM002] [Microsoft][ODBC Driver Manager] Data source name not found and no default driver specified''. Probable reason is that the ODBC driver is missing or driver name is not correct. To solve the issue:
| |
| * Check ODBC drivers is installed by running '''C:\Windows\System32\odbcad32.exe'''.
| |
| * Check the connection string.
| |
| | |
| The following error message may be encountered: '''ERROR [HY024] [Microsoft][ODBC Text Driver] '(unknown)' is not a valid path. Make sure that the path name is spelled correctly and that you are connected to the server on which the file resides.'''.
| |
| Example error when specified directory is missing:
| |
| | |
| Error when CSV file is missing: '''System.Data.Odbc.OdbcException (0x80131937): ERROR [42S02] [Microsoft][ODBC Text Driver] The Microsoft Access database engine could not find the object 'CaseAttributes.csv'. Make sure the object exists and that you spell its name and the path name correctly. If 'CaseAttributes.csv' is not a local object, check your network connection or contact the server administrator.'''
| |
| | |
| Error codes for troubleshooting: [https://docs.microsoft.com/en-us/sql/odbc/reference/appendixes/appendix-a-odbc-error-codes?view=sql-server-2017| ODBC Error Codes].
| |
| | |
| If the error message mention about Case or Event attributes named as ''F<column number>'' (e.g. F10), the data might be inconsistent, as there might be different amount of columns in different rows.
| |
| | |
| == Case Permissions ==
| |
| The '''Permissions''' section specifies data security restrictions for objects within the QPR ProcessAnalyzer model (i.e. limit visibility). If the Permissions section hasn't been defined, all the model data is visible to all users having '''GenericRead''' permission for the project in which the model resides ([[User Roles and Permissions in QPR ProcessAnalyzer|more information about roles and permissions]]). Permissions defined in this section, are only available when using the [[Installing QPR ProcessAnalyzer Server#Configure_Web_Service_to_use_In-Memory_or_In-Database_processing|In-Memory core]].
| |
| | |
| {| class="wikitable"
| |
| !'''Property'''
| |
| ! '''Description'''
| |
| |-
| |
| |Permissions/Initialization
| |
| |[[QPR_ProcessAnalyzer_Expressions|Expression language]] expression used to make an initial calculation for all the other expressions within this same permissions context. This expression can be used to improve performance when part of the Case or EventLogKey expressions are common and thus they don't need to be calculated again for every Case separately. See the examples below of using the Initialization expression.
| |
| |-
| |
| ||Permissions/Case
| |
| ||[[QPR_ProcessAnalyzer_Expressions|Expression language]] expression determining which users can see each Cases. The expression is evaluated within the context of each Case. If the evaluation results '''true''', the Case is visible for the user. Otherwise the Case, its Events and case and event attributes are not visible. This setting implements case level security restrictions.
| |
| |-
| |
| ||Permissions/EventLogKey
| |
| ||[[QPR_ProcessAnalyzer_Expressions|Expression language]] expression used to uniquely identify all the unique event logs created by case permission filters. If a cached EventLog with the same key is already in the system, that EventLog is used instead of creating a new. The new EventLog is created by applying the Case expression to filter the Cases users have rights to.
| |
| |}
| |
| | |
| === Example usecase for case permissions ===
| |
| Case level permissions (security control) can be implemented with the principle illustrated in the image below. ''Users'' already belong to certain ''groups'' in the user management, and ''cases'' have certain ''case attribute values'' which is part of the loaded process mining data. Additionally, the linkage between case attribute values (of a certain case attribute) and groups needs to be defined when this security feature is configured. The image below illustrates the chain between ''users'' and ''cases'', how certain users are able to see certain cases when viewing analyses from a QPR ProcessAnalyzer model. | |
|
| |
|
| [[File:CasePermissions.png|800px]] | | [[File:CasePermissions.png|800px]] |
| Line 340: |
Line 77: |
| There is no way for a user to be aware of the existence of cases that the user doesn't have rights to. | | There is no way for a user to be aware of the existence of cases that the user doesn't have rights to. |
|
| |
|
| === Configuration examples for case permissions ===
| | [[Category: QPR ProcessAnalyzer]] |
| In this example, visibility of cases is limited in a way that only those users can see the cases belonging to a user group which name is same as the Region (case attribute).
| |
| <pre>
| |
| {
| |
| "Permissions": {
| |
| "Initialization": "Let(\"groupNames\", OrderByValue(CurrentUser.GroupNames))",
| |
| "Case": "Region.In(groupNames)",
| |
| "EventLogKey": "StringJoin(\"_\", groupNames)"
| |
| }
| |
| }
| |
| </pre>
| |
| | |
| In this example, cases are only visible for users whose user name is same as the Account Manager (case attribute).
| |
| <pre>
| |
| {
| |
| "Permissions": {
| |
| "Initialization": "Let(\"userName\", CurrentUser.Name)",
| |
| "Case": "(Attribute(\"Account Manager\") == userName)",
| |
| "EventLogKey": "CurrentUser.Id"
| |
| }
| |
| }
| |
| </pre>
| |
| | |
| In this example, cases having "Region" case attribute of "Dallas" will only be visible for users belonging to user group "GroupA" (and "New York" for group "GroupB").
| |
| <pre>
| |
| {
| |
| "Permissions": {
| |
| "Initialization": "Let(\"groupNames\", CurrentUser.GroupNames)",
| |
| "Case": "(Region == \"Dallas\" && \"GroupA\".In(groupNames)) || (Region == \"New York\" && \"GroupB\".In(groupNames))",
| |
| "EventLogKey": "If(\"GroupA\".In(groupNames), \"_A\", \"_\") + If(\"GroupB\".In(groupNames), \"_B\", \"_\")"
| |
| }
| |
| }
| |
| </pre>
| |
