In-memory Models Management: Difference between revisions

From QPR ProcessAnalyzer Wiki
Jump to navigation Jump to search
No edit summary
No edit summary
Line 1: Line 1:
Memory management settings determine, how long models and calculation results are kept in the memory, balancing between the performance and memory consumption. The longer the items are in the memory, the more memory is reserved, but on the other hand users perceive better performance as models and calculation results are more often available in the memory cache (i.e., higher hit rate). There are memory settings that can be defined for the system level in the [[Web.config_file#Qpr.ProcessAnalyzer.Common.Properties.Settings_Section|web.config]] file and for each model separately.
Memory management settings determine, how long models and calculation results are kept in the memory, balancing between the performance and memory consumption. The longer the items are in the memory, the more memory is reserved, but on the other hand users perceive better performance as models and calculation results are more often available in the memory cache (i.e., higher hit rate). There are memory settings that can be defined for the system level in the [[Web.config_file#Qpr.ProcessAnalyzer.Common.Properties.Settings_Section|web.config]] file and for each model separately.


For models, there is the '''Drop Unused Filters After''' setting available in the [[QPR_ProcessAnalyzer_Project_Workspace#Editing_Model_Settings|Model properties]] dialog. The setting determines the duration after which the unused filters in the model are dropped from the memory. Defined in format ''HH:mm:ss'' or ''d.HH:mm:ss'', for example 01:00:00 (one hour), 00:30:00 (30 minutes) or 1.00:00:00 (24 hours). When any calculation is performed using a filter, the filter's last used time is updated (also model's last used time is updated). If this setting is not defined, the server level [[Web.config_file#Qpr.ProcessAnalyzer.Common.Properties.Settings_Section|default setting]] is used. If unspecified, the server level setting is used (which is by default 1 hour).
For models, there is the '''Drop Unused Filters After''' setting available in the [[QPR_ProcessAnalyzer_Project_Workspace#Editing_Model_Settings|Model properties]] dialog. The setting determines the duration after which the unused filters in the model are dropped from the memory. The Drop Unused Filters After setting is defined in format ''HH:mm:ss'' or ''d.HH:mm:ss'', for example 01:00:00 (one hour), 00:30:00 (30 minutes) or 1.00:00:00 (24 hours). If this setting is not defined, the server level [[Web.config_file#Qpr.ProcessAnalyzer.Common.Properties.Settings_Section|default setting]] is used. When any calculation is performed using a filter, the filter's last used time is updated.
 
==Memory Stored Objects==
==Memory Stored Objects==
Data handled by QPR ProcessAnalyzer memory is stored in following types of objects:
There are following types of objects managed in the QPR ProcessAnalyzer server memory:
{| class="wikitable"
{| class="wikitable"
!'''Stored object'''
!'''Stored object'''
Line 12: Line 13:
|-
|-
||Models (model eventlogs)
||Models (model eventlogs)
||Models contain all data in a model, such as events, cases, event types and variations.
||Models contain the eventlog data (events, cases), and objects calculated from the (e.g., event types, variations and flows).
||Models are never dropped automatically from the memory, even when there is a memory shortage in the server. To drop a model from memory, user needs to command the dropping in the Workspace. Note that if the server has been restarted, models may not anymore be in the memory after the restart.
||Models are never dropped automatically from the memory, even when there is a memory shortage in the server. To drop a model from memory, you need to [[QPR_ProcessAnalyzer_Project_Workspace#Loading.2C_Dropping_and_Reloading_Model|drop the model]] in the Workspace. Note that if the server has been restarted, only models that have the [[Automatic_Model_Loading_on_Server_Startup|automatic loading set]] are loaded into memory models directly after the restart.
||'''Slow'''
||'''Slow'''


Line 20: Line 21:
|-
|-
||Filters (filter eventlogs)
||Filters (filter eventlogs)
||Filters contain all filtered data in model, which includes similar data and structures as the models.
||Filters contain all filtered data in model, which includes similar structures than the models.
||Primarily defined by the ''Drop Unused Filters After'' setting in the Model settings dialog. If this not set for a model, the server level setting in the [[Web.config_file|web.config]] file is used. If the server setting is not defined, a default value of '''1 hour''' is used.
||Primarily defined by the ''Drop Unused Filters After'' setting in the Model settings dialog. If this not set for a model, the server level setting in the [[Web.config_file|web.config]] file is used. If the server setting is not defined, a default value of '''1 hour''' is used.


Line 30: Line 31:
|-
|-
||Calculation results
||Calculation results
||Results of the calculations made e.g. for charts.
||Results of the calculations made in dashboards.
||Unused calculation results are kept '''30 minutes''' in the memory. When the is a memory shortage in the server, they may be dropped from the memory earlier to free memory.
||Unused calculation results are kept '''30 minutes''' in the memory. When the is a memory shortage in the server, they may be dropped from the memory earlier to free memory.
||'''Fast'''
||'''Fast'''
Line 44: Line 45:
|}
|}


If the QPR ProcessAnalyzer server doesn't have enough memory to store more objects, already stored filters, calculation results and datatables are dropped from the memory starting from the one having the longest time since the last usage. Thus, when there is a memory shortage, objects may be dropped earlier than their settings define. Filter eventlogs, calculation results and datatables are dropped during the memory shortage, and model eventlog are never dropped prematurely. This is because recalculating filters and calculation results is usually faster than loading models from the database. That is why, when trying to load more models than there is available memory in the server, an out of memory error situation may occur.
If the QPR ProcessAnalyzer server doesn't have enough memory to store more objects, already stored filters, calculation results and datatables are dropped from the memory starting from the one having the longest time since the last usage. Thus, when there is a memory shortage, objects may be dropped earlier than their settings define. Filters, calculation results and datatables are dropped during the memory shortage, but model are never dropped automatically. This is because recalculating filters and calculation results is usually faster than loading models from the database. That is why, when trying to load more models than there is available memory in the server, an out of memory error situation may occur.


[[Category: QPR ProcessAnalyzer]]
[[Category: QPR ProcessAnalyzer]]

Revision as of 20:03, 6 September 2022

Memory management settings determine, how long models and calculation results are kept in the memory, balancing between the performance and memory consumption. The longer the items are in the memory, the more memory is reserved, but on the other hand users perceive better performance as models and calculation results are more often available in the memory cache (i.e., higher hit rate). There are memory settings that can be defined for the system level in the web.config file and for each model separately.

For models, there is the Drop Unused Filters After setting available in the Model properties dialog. The setting determines the duration after which the unused filters in the model are dropped from the memory. The Drop Unused Filters After setting is defined in format HH:mm:ss or d.HH:mm:ss, for example 01:00:00 (one hour), 00:30:00 (30 minutes) or 1.00:00:00 (24 hours). If this setting is not defined, the server level default setting is used. When any calculation is performed using a filter, the filter's last used time is updated.

Memory Stored Objects

There are following types of objects managed in the QPR ProcessAnalyzer server memory:

Stored object Contents Unused objects dropped after Time to recreate after dropping Memory consumption
Models (model eventlogs) Models contain the eventlog data (events, cases), and objects calculated from the (e.g., event types, variations and flows). Models are never dropped automatically from the memory, even when there is a memory shortage in the server. To drop a model from memory, you need to drop the model in the Workspace. Note that if the server has been restarted, only models that have the automatic loading set are loaded into memory models directly after the restart. Slow

Models are loaded from the database requiring to transfer considerable amount of data, which takes much more time than e.g. calculating filters and analyses.

High
Filters (filter eventlogs) Filters contain all filtered data in model, which includes similar structures than the models. Primarily defined by the Drop Unused Filters After setting in the Model settings dialog. If this not set for a model, the server level setting in the web.config file is used. If the server setting is not defined, a default value of 1 hour is used.

When the is a memory shortage in the server, they may be dropped from the memory earlier to free memory.

Fast

Filters are calculated from the model data that already exists in the memory. Practically, filters are subsets of models.

Medium
Calculation results Results of the calculations made in dashboards. Unused calculation results are kept 30 minutes in the memory. When the is a memory shortage in the server, they may be dropped from the memory earlier to free memory. Fast

Analysis results are calculated from filters, which already exist in memory.

Low
Datatables Contents of datatables. There is a fixed duration of 1 hour. When the is a memory shortage in the server, they may be dropped from the memory earlier to free memory. Between slow and fast, depending on how much there is data in the datatable. Between high and low, depending on how much there is data in the datatable.

If the QPR ProcessAnalyzer server doesn't have enough memory to store more objects, already stored filters, calculation results and datatables are dropped from the memory starting from the one having the longest time since the last usage. Thus, when there is a memory shortage, objects may be dropped earlier than their settings define. Filters, calculation results and datatables are dropped during the memory shortage, but model are never dropped automatically. This is because recalculating filters and calculation results is usually faster than loading models from the database. That is why, when trying to load more models than there is available memory in the server, an out of memory error situation may occur.