Big Data Chart

From QPR ProcessAnalyzer Wiki
Revision as of 14:15, 25 March 2022 by Ollvihe (talk | contribs)
Jump to navigation Jump to search

Big Data Chart performs backend calculations in the datasource (in Snowflake or SQL Server) where the eventlog data is stored, whereas the regular chart uses the in-memory processing. Depending on the model type, processing is done in Snowflake (for models using Snowflake datatables) or in SQL Server (for models using Local datatables). Snowflake powered calculation will enable practically unlimited scaling when the amount of data and users increase. The Big Data Chart needs to be used in dashboards when using Snowflake models.

Big Data Chart can be used for models using local datatables, and then processing is performed in SQL Server which is not optimal for analytics queries from performance viewpoint, though. There are still use cases when the Big Data Chart is a suitable for models using local datatables:

  • Eventlogs are filtered heavily so that the number of remaining cases and events are low (usually maximum of some thousands). Then processing may be done in the SQL Server without using the in-memory processing (which will require less memory)
  • If the model is not currently available in the memory, it's faster to use the Big data chart comparing to the in-memory chart, when the required time to load the model into memory is taken into account.

Big Data Chart can be added to dashboard by selecting the second item from the tool palette (labelled Big Data Chart).

Differences to in-memory chart

Visualization settings are the same between the Big Data Chart and in-memory chart. On the other hand, data selection settings, and measures and dimensions work differently. Differences are as follows:

  • There are different set of analyzed objects, measures and dimensions available.
  • In Big Data Chart, filtering cases and events can be done also for each measure and dimension separately. This allows to build most KPI's flexibly without using custom expressions.
  • In Big Data Chart, available measures and dimensions are equal, and they are separated only by the additional aggregation selection that measures have. Due to the similarity, in Big Data Chart measures can be moved to dimensions and vice versa by clicking the Move to dimensions and Move to measures buttons.
  • Custom expressions are written as SQL expressions which differs from the eventlog objects available in the in-memory charts. Note also that measure expressions in Big Data Chart doesn't include the aggregation logic, and thus the custom measure and dimension expressions are equal.
  • The Any datatype is not supported by the Big Data Chart in case and event attributes. Thus, when importing data, specific datatypes need to be set for each column, for case and event attributes to be available.
  • Following functionalities supported by the in-memory are not available in the Big Data Chart: Presets, Group rows exceeding maximum, Analyzed objects sample size, Find root causes, and Business calendar.
  • In addition, the following measure/dimension settings are not available: Round to decimals, Calculate measure for, Variable name, Custom aggregation expression, and Adjustment expression.
  • Big data chart cannot be used with model using ODBC or expression datasources.
  • Big Data Chart supports filtering similar to the in-memory chart, i.e., visualizations can be clicked to create filters for the shown data. Big Data Chart does not support expression based filter rules and thus there are some dimensions where filtering is not available. Other types of filter rules are same for Big Data and in-memory charts. Thus same dashboard can contain both types of chart components, and filtering between them works. Note that when an expression based filter is created from an in-memory chart, Big Data Chart cannot be shown as the filter cannot be calculated.