Generic Functions in QPR ProcessAnalyzer: Difference between revisions

From QPR ProcessAnalyzer Wiki
Jump to navigation Jump to search
 
(240 intermediate revisions by 3 users not shown)
Line 1: Line 1:
Generic Properties and Functions are available for all objects.
== Aggregation functions ==
 
Following aggregation functions are available:
=== Aggregation Functions ===
* '''Average''': Average of the items (sum of items divided by count of items). Can be used for numbers, [[QPR_ProcessAnalyzer_Objects_in_Expression_Language#DateTime|DateTimes]] and [[QPR_ProcessAnalyzer_Objects_in_Expression_Language#TimeSpan|Timespans]].
There are following aggregation functions: '''Average''', '''Count''', '''Median''', '''Min''', '''Max''' and '''Sum'''. The aggregation operations work for numbers, [[Object_Properties_and_Functions_in_QPR_ProcessAnalyzer_Expressions#DateTime|DateTimes]] and [[Object_Properties_and_Functions_in_QPR_ProcessAnalyzer_Expressions#TimeSpan|TimeSpans]], except Sum for DateTime is not possible.
* '''Count''': Number of items. Can be used for any type of data.
* '''Median''': Median, i.e. the middle value of the sorted data. If there are even number of items, the average of the two middle items. Can be used for numbers, [[QPR_ProcessAnalyzer_Objects_in_Expression_Language#DateTime|DateTimes]] and [[QPR_ProcessAnalyzer_Objects_in_Expression_Language#TimeSpan|Timespans]].
* '''Min''': Lowest/first value in the data. Can be used for data that can be sorted.
* '''Max''': Highest/last value in the data. Can be used for data that can be sorted.
*'''Percentile''': Percentile value. Can be used for numbers, [[QPR_ProcessAnalyzer_Objects_in_Expression_Language#DateTime|DateTimes]] and [[QPR_ProcessAnalyzer_Objects_in_Expression_Language#TimeSpan|Timespans]].
*'''Stdev''': Sample standard deviation. Can be calculated for numbers, [[QPR_ProcessAnalyzer_Objects_in_Expression_Language#DateTime|DateTimes]] and [[QPR_ProcessAnalyzer_Objects_in_Expression_Language#TimeSpan|Timespans]].
*'''Stdevp''': Population standard deviation. Can be calculated for numbers, [[QPR_ProcessAnalyzer_Objects_in_Expression_Language#DateTime|DateTimes]] and [[QPR_ProcessAnalyzer_Objects_in_Expression_Language#TimeSpan|Timespans]].
* '''Sum''': Sum of the items. Can be used for numbers and [[QPR_ProcessAnalyzer_Objects_in_Expression_Language#TimeSpan|Timespans]].
* '''StringJoin''': Joins array of strings together. The first parameter is a string that is placed between the joined strings.
*'''Var''': Sample variance. Can be used for numbers.
*'''Varp''': Population variance. Can be used for numbers.


Examples:
Examples:
Line 14: Line 24:
Sum([[1, 2], [3, 4, 5]])
Sum([[1, 2], [3, 4, 5]])
Returns: [3, 12]
Returns: [3, 12]
Percentile([1,2,3,4,5], 0.75)
Return: 4


Sum([])
Sum([])
Returns: null
Returns: null
StringJoin(", ", ["one", "two", "three"])
Returns: one, two, three
</pre>
</pre>


When aggregating numerical values and there are ''null'' values, nulls are treated as zeros. When aggregating DateTimes and TimeSpans, and there are null values, error is given. In the Min and Max functions, nulls are ignored. Null values can be removed before aggregating (the following example shows how).
When aggregating numerical values, ''null'' values are treated as zeros. Nulls can be removed with ''RemoveNulls'' function. In the Min and Max functions, nulls are ignored. Null values can be removed before aggregating (the following example shows how).
<pre>
<pre>
Average([1, 5, null])
Average([1, 5, null])
Returns: 2
Returns: 2


Average([1, 5, null].Where(!IsNull(_)))
Average(RemoveNulls([1, 5, null])
Returns: 3
Returns: 3
</pre>
</pre>
Line 35: Line 51:
</pre>
</pre>


=== Mathematical functions ===
== Mathematical functions ==


{| class="wikitable"
{| class="wikitable"
Line 41: Line 57:
!'''Parameters'''
!'''Parameters'''
! '''Description'''
! '''Description'''
|-
||Abs (Number)
||Number
||
Calculates the absolute value of a specified number. Examples:
<pre>
Abs(4)
Returns: 4
Abs(-2.7)
Returns: 2.7
Abs(0)
Returns: 0
</pre>
|-
||Acos (Number)
||Number
||
Calculates the angle whose cosine is the specified number. The parameter is a number representing a cosine, which must be between -1 and 1. Examples:
<pre>
Acos(0.3584)
Returns: 1.204242852965772
Acos(-0.3584)
Returns: 1.9373498006240213
</pre>
|-
||ArgMax (Object)
||
# Array or hierarchical object
# Value expression
||
Returns those objects in the array giving maximum values for the given expression when evaluated in the object's context. Examples:
<pre>
ArgMax([5, 1, 6, 4, 3, 3, 6, 5, 4, 1, 1], _)
Returns: [6, 6]
ArgMax(EventLogById(1).Cases, Duration)
Returns: An array of cases having the longest duration.
</pre>
|-
||ArgMin (Object)
||
# Array or hierarchical object
# Value expression
||
Returns those objects in the array giving minimum values for the given expression when evaluated in the object's context. Examples:
<pre>
ArgMin([5, 1, 6, 4, 3, 3, 6, 5, 4, 1, 1], _)
Returns: [1, 1, 1]
ArgMin(EventLogById(1).Cases, Duration)
Returns: An array of cases having the shortest duration.
</pre>
|-
||Asin (Number)
||Number
||
Calculates the angle whose sine is the specified number. The parameter is a number representing a sine, which must be between -1 and 1. Examples:
<pre>
Asin(-0.3584)
Returns: 0.36655347382912462
Asin(+0.3584)
Returns: -0.36655347382912462
</pre>
|-
||Atan (Number)
||Number
||
Calculates the angle whose tangent is the specified number. Examples:
<pre>
Atan(1)
Returns: 0.78539816339744828
Atan(-1)
Returns: -0.78539816339744828
</pre>
|-
|-
||Ceiling (Integer)
||Ceiling (Integer)
||
||
* Number (Float)
Number
||
||
Returns the smallest integer that is greater than or equal to the specified number. Example:
Returns the smallest integer that is greater than or equal to the specified number. Example:
Line 50: Line 145:
Ceiling(1.3)
Ceiling(1.3)
Returns: 2
Returns: 2
</pre>
|-
||Cos (Number)
||Number
||
Calculates the cosine of the specified angle measured in radians. Examples:
<pre>
Cos(1)
Returns: 0.54030230586813977
Cos(-1)
Returns: 0.54030230586813977
</pre>
|-
||Exp (Number)
||Number
||
Calculates the e raised to the specified power. Examples:
<pre>
Exp(1)
Returns: 2.7182818284590451
Exp(-1)
Returns: 0.36787944117144233
</pre>
</pre>
|-
|-
||Floor (Integer)
||Floor (Integer)
||
||
* Number (Float)
Number
||
||
Returns the largest integer that is less than or equal to the specified number. Examples:
Returns the largest integer that is less than or equal to the specified number. Example:
<pre>
<pre>
Floor(1.9)
Floor(1.9)
Returns: 1
Returns: 1
</pre>
|-
||Hash (String)
||String to hash
||
Returns 11 characters long hash value for the given string. There can be following characters in the hash: A-Za-z0-9+/. For null or empty string, "AAAAAAAAAAA" is returned.
The hashing is a repeatable operation, i.e., the same input string always returns the same hash value. Note also that hashing lead to collisions, i.e., two different input strings may give the same hash value.
The hash is calculated using Murmur3 algorithm (https://blog.teamleadnet.com/2012/08/murmurhash3-ultra-fast-hash-algorithm.html) and the first 64 bits of the 128-bit hash value is returned encoded as base64url without padding (RFC 4648 §5).
Examples:
<pre>
Hash("abcdef")
Returns: 3sqiubzVXxA
</pre>
|-
||IEEERemainder (Number)
||
# Dividend (Number)
# Divisor (Number)
||
Calculates the remainder resulting from the division of a specified number by another specified number. Uses formula:
<pre>
dividend - (divisor * Round(dividend / divisor))
</pre>
Note that the formula is different than in the % operator, which uses formula:
<pre>
(Abs(dividend) - (Abs(divisor) * (Floor(Math.Abs(dividend) / Abs(divisor))))) * Sign(dividend) 
</pre>
Examples:
<pre>
IEEERemainder(26, 4)
Returns: 2
IEEERemainder(0, 0)
Returns: NaN
</pre>
|-
||Log (Number)
||
# Number
# Logarithm base (Number)
||
Calculates the logarithm of a specified number. If the second parameter is not provided, it's the natural logarithm (base e). The second parameter can be used to define another base for the logarithm. Examples:
<pre>
Log(5)
Returns: 1.6094379124341003
Log(10, 2)
Returns: 3.3219280948873626
</pre>
|-
||Log10 (Number)
||
Number
||
Calculates the base 10 logarithm of a specified number. Examples:
<pre>
Log10(5)
Returns: 0.69897000433601886
Log10(-5)
Returns: NaN
</pre>
|-
||Pow (Number)
||
# Base number
# Exponent number
||
Calculate a specified number raised to the specified power. Examples:
<pre>
Pow(5, 3)
Returns: 125
Pow(-5, 3)
Returns: -125
</pre>
|-
||Random (Number)
||(none)
||Returns a random number value between 0 and 1, where 0 is included, but 1 is not.
Example:
<pre>
Random()
Returns: 0.1243234243 (for example)
</pre>
</pre>
|-
|-
||Round (Float)
||Round (Float)
||
||
# Number to be rounded (Float)
Number of decimals (Integer)
# Number of decimals (Integer)
||
||
For numbers, rounds a value to the nearest integer or to the specified number of fractional digits. For DateTimes, rounds given date time by given time span or given number of seconds,
For numbers, rounds the number to the specified number of decimal. Number of decimals can also be negative, in which case the number is rounded to the nearest tens, hundreds, thousands, etc.
 
For DateTimes, rounds given date time by given time span or given number of seconds.
 
Note that the number and DateTime to be rounded needs to be provided as a context object (not as a parameter).


Examples:
Examples:
Line 73: Line 285:
(1.254).Round(1)
(1.254).Round(1)
Returns: 1.3
Returns: 1.3
(162.111111).Round(-2)
Returns: 200


DateTime(2017, 1, 2, 3, 4, 5).Round(10)
DateTime(2017, 1, 2, 3, 4, 5).Round(10)
Line 79: Line 294:
DateTime(2017, 1, 2, 3, 4, 5).Round(TimeSpan(1))
DateTime(2017, 1, 2, 3, 4, 5).Round(TimeSpan(1))
Returns: DateTime(2017, 1, 2)
Returns: DateTime(2017, 1, 2)
</pre>
|-
||Sign (Number)
||
Number
||
Calculates an integer that indicates the sign of a number, either -1, 0 or 1. Examples:
<pre>
Sign(10)
Returns: 1
Sign(0)
Returns: 0
Sign(10)
Returns: -1
</pre>
|-
||Sin (Number)
||
Number
||
Calculates the sine of the specified angle. Examples:
<pre>
Sin(3.14)
Returns: 0.0015926529164868282
Sin(-3.14)
Returns: -0.0015926529164868282
</pre>
|-
||Sqrt (Number)
||
Number
||
Calculates the square root of a specified number. Examples:
<pre>
Sqrt(81)
Returns: 9
Sqrt(0.9)
Returns: 0.3
</pre>
|-
||Tan (Number)
||
Number
||
Calculates the tangent of the specified angle. Examples:
<pre>
Tan(1)
Returns: 1.5574077246549023
Tan(-1)
Returns: -1.5574077246549023
</pre>
</pre>
|}
|}


=== DateTime and TimeSpan functions ===
== Conversion functions ==


{| class="wikitable"
{| class="wikitable"
Line 89: Line 361:
! '''Description'''
! '''Description'''
|-
|-
||DateTime
||ToFloat (Number)
||
||
# Year (1-9999) (Integer)
Object to convert
# Month (1-12) (Integer)
||Converts the given object to a decimal number.
# Day (>= 1) (Integer)
|-
# Hour (>= 0) (Integer)
||ToInteger (Integer)
# Minute (>= 0) (Integer)
# Second (>= 0) (Integer)
# Millisecond (>= 0) (Integer)
||
||
Creates a new [[Object_Properties_and_Functions_in_QPR_ProcessAnalyzer_Expressions#DateTime|DateTime]] object. Only the first (year) parameter is mandatory.  
Object to convert
||
Converts the given object into an integer. If the object can not be converted into an integer, an exception will be thrown. If the object as a decimal number, it will also be rounded to the nearest integer.


Examples:
Examples:
<pre>
<pre>
DateTime(2017)
ToInteger(1.234)
Returns: A datetime for 1st January 2017 at 0:00:00.
Returns: 1
 
ToInteger(-1.5)
Returns: -2


DateTime(2017, 5)
ToInteger(1313.6)
Returns: A datetime for 5th January 2017 at 0:00:00.
Returns: 1314
 
ToInteger(123456789012345678)
Returns: 123456789012345678
 
ToInteger("5")
Returns: 5


DateTime(2017, 5, 6, 13, 34, 56, 123)
ToInteger("-5")
Returns: A date time object for 6th May 2017 at 13:34:56.123.
Returns: -5
</pre>
</pre>
|-
|-
||DateTimeFromTicks
||ToString (String)
||
||
# Number of ticks (Integer)
# Object to convert
# Date/Number/Timespan format (String)
||
||
Creates a new [[Object_Properties_and_Functions_in_QPR_ProcessAnalyzer_Expressions#DateTime|DateTime]] object from an integer representing ticks. A single tick represents one hundred nanoseconds or one ten-millionth of a second. There are 10,000 ticks in a millisecond, or 10 million ticks in a second.  
Converts the given object to a string.
 
The optional second parameter defines formatting that is needed when converting the following data types:
* Date: https://docs.microsoft.com/en-us/dotnet/standard/base-types/custom-date-and-time-format-strings
* Number: https://docs.microsoft.com/en-us/dotnet/standard/base-types/custom-numeric-format-strings
* Timespan: https://docs.microsoft.com/en-us/dotnet/standard/base-types/custom-timespan-format-strings


Examples:
Examples:
<pre>
<pre>
DateTimeFromTicks(DateTime(2018, 5, 6, 13, 34, 56, 123).Ticks)
ToString(1) + " < " + ToString(2)
Returns: A date time object for 6th May 2018 at 13:34:56.123.
Returns: "1 < 2".
 
"Case=\'" + ToString(a) + "\'"
Returns: "Case='<string representation of object a>'"
 
ToString(DateTime(2017,1,2,3,4,5,6))
Returns: "2017-01-02T03:04:05"
 
ToString(DateTime(2017,1,2,3,4,5,6), "dd.MM.yyyy")
Returns "02.01.2017"
 
ToString(1.2, "0.00")
Returns "1.20"
 
ToString(TimeSpan(16,4,3,17,250), "%d")
Returns "16"
</pre>
</pre>
|}
== Data source connectors ==
{| class="wikitable"
!'''Function'''
!'''Parameters'''
! '''Description'''
|-
|-
||TimeSpan
||CallWebService (Dictionary)
||Dictionary of parameters
||
||
# Days (Integer)
CallWebService is a generic function for any kind of http requests. Support natively REST-style web services by automatically formatting and parsing json in the request and response, so dictionary/array objects are used without converting them to strings (see examples  below). If using xml-based web services, formatting and parsing of xml needs to be taken care of manually.
# Hours (Integer)
 
# Minutes (Integer)
The function takes as a dictionary of following parameters:
# Seconds (Integer)
 
# Milliseconds (Integer)
*'''Address''' (string): Url address to connect to.
||
*'''Method''' (string): Http method to use, as one of the following: GET (default), POST, PUT, DELETE.
Creates a new [[Object_Properties_and_Functions_in_QPR_ProcessAnalyzer_Expressions#TimeSpan|TimeSpan]] object. Only the first parameter (number of days) is mandatory. By default, other parameters are assumed to be zero.
*'''Body''' (string/dictionary/array): The http request body. In case the request content type is application/json, dictionary/array can be used directly without string conversion.
*'''Content-Type''' (string): Defines the Content-Type http request header. If content type is application/x-www-form-urlencoded, form url encoding is applied for the request body.
*'''Timeout''' (integer): Time to wait before the request times out, in case no response is received earlier. Defined as integer in milliseconds. By default 60000 milliseconds.
*'''Headers''' (dictionary): Key-value pairs of HTTP request headers.
*'''QueryParameters''' (dictionary): Key-value pairs of query parameters that are added to the url. Proper encoding of the url is taken care of by the function.
*'''DefaultNetworkCredentials''' (boolean): When true, uses the default network credentials for the call. When false, the default network credentials are not used. The default network credentials can be used only if in the server configuration AllowForwardingNetworkCredentials setting is enabled.
*'''Encoding''' (string): Defines the body encoding to use. Default value is UTF8. The supported options are listed in https://msdn.microsoft.com/en-us/library/system.text.encoding%28v=vs.110%29.aspx.
 
CallWebService returns a dictionary with properties:
* '''Response''' (string/dictionary/array): Response data as string, or if content type is application/json, as dictionary/array object.
* '''StatusCode''' (integer): Http status code.
* '''Headers''' (dictionary): All http response headers.
 
The Content-Type http request header is set as follows: If the Content-Type parameter is provided, that is used. If Content-Type is not provided, and body is dictionary/array, request Content-Type is set to application/json. Otherwise no Content-Type header is set.
 
If the Content-Type parameter is application/x-www-form-urlencoded, body is expected to be a dictionary where the name/values pairs are encoded using FormUrlEncodedContent method (see examples below).


Examples:
Examples:
Get a webpage and check the status code of the response:
<pre>
<pre>
TimeSpan(1)
let resp = CallWebService(#{
Returns: Time span for the duration of 1 day.
  "Address": "https://www.google.com/search?q=QPR+ProcessAnalyzer"
});
resp.StatusCode;
</pre>


TimeSpan(12,3,4,5,6)
Get the server version in a QPR ProcessAnalyzer environment:
Returns: Time span for the duration of 12 days, 3 hours, 4 minutes, 5 seconds and 6 milliseconds.
<pre>
let resp = CallWebService(#{
  "Address": "https://processanalyzer.onqpr.com/qprpa/api/serverinfo"
});
resp.Response.serverVersion;
</pre>
 
Authenticate to QPR ProcessAnalyzer (to get an access_token) and call expression query (note the Content-Type required by the token request):
<pre>
let server = "http://processanalyzer.onqpr.com/qprpa";
let resp = CallWebService(#{
  "Address": server + "/token",
  "Method": "POST",
  "Content-Type": "application/x-www-form-urlencoded",
  "Body": #{
    "grant_type": "password",
    "username": "qpr",
    "password": "demo"
  }
});
resp = CallWebService(#{
  "Address": server + "/api/expression/query",
  "Method": "POST",
  "Content-Type": "application/json",
  "Headers": #{ "Authorization": "Bearer " + resp.Response.access_token },
  "Timeout": 120000,
  "Body": #{
    "Root": "Models",
    "Dimensions": null,
    "Values": [ #{"Name": "Name", "Expression": "Name"} ],
    "Ordering":[ #{"Name": "Name", "Direction": "Ascending"} ]
  }
});
resp.Response[0].Name
</pre>
 
Upload contents of datatable id 321 to using Web API to a datatable id 123:
<pre>
CallWebService(#{
  "Address": "https://processanalyzer.onqpr.com/qprpa/api/importfile?importMode=datatable&fileType=csv&objectId=123&append=true&csvTimeFormat=yyyy-MM-dd%20HH%3Amm%3Ass,fff",
  "Method": "POST",
  "Body": DatatableById(321).SqlDataFrame.Collect().ToCsv(),
  "Headers": #{ "Authorization": "Bearer " + resp.Response.access_token },
  "Timeout": 6000000
});
</pre>
</pre>
|-
|-
||TimeSpanFromTicks
||<span id="OpenAIChatCompletion">OpenAIChatCompletion</span> (Dictionary)
||
||Request dictionary
# Number of ticks (Integer)
||Calls OpenAI's Chat Completions API (https://platform.openai.com/docs/api-reference/chat) to generate natural language response to user prompt. Both the request and response is a dictionary following the Chat Completions API specification. As specified in the API, the ''model'' and ''messages'' are mandatory fields in the request.
||
 
Creates a new [[Object_Properties_and_Functions_in_QPR_ProcessAnalyzer_Expressions#TimeSpan|TimeSpan]] object from an integer representing ticks. A single tick represents one hundred nanoseconds or one ten-millionth of a second. There are 10,000 ticks in a millisecond, or 10 million ticks in a second.  
To use the function, the OpenAi API key needs to be configured in [[PA_Configuration_database_table#General_Settings|OpenAIAPIKey]] configuration table.


Examples:
Example: Response to a user request:
<pre>
<pre>
TimeSpanFromTicks(TimeSpan(12,3,4,5,6).Ticks)
let response = OpenAIChatCompletion(#{
Returns: A time span object representing 12 days, 3 hours, 4 minutes, 5 seconds and 6 milliseconds.
  "model":  "gpt-3.5-turbo",
  "messages": [
    #{
      "role": "system",
      "content": "You are a process mining expert making intelligent suggestions how to improve processes."
    },
    #{
      "role": "user",
      "content": "Give me three suggestions how to improve a sales process with bottlenecks."
    }
  ]
});
response["choices"][0]["message"]["content"];
</pre>
</pre>
|-
||[[Object_Properties_and_Functions_in_QPR_ProcessAnalyzer_Expressions#DateTime|Truncate]]
||
||
|}
|}


=== [[Object_Properties_and_Functions_in_QPR_ProcessAnalyzer_Expressions#String|String functions]] ===
== Miscellaneous functions ==
 
=== Array functions ===


{| class="wikitable"
{| class="wikitable"
Line 170: Line 550:
! '''Description'''
! '''Description'''
|-
|-
||Array
||Where
||
||
# Item 1 (Object)
Condition expression
# Item 2 (Object)
# ...
||
||
Creates a new array where the provided parameters are items of the array. There can be any number of parameters. Note: arrays can also be created using '''[]''' syntax.
Returns the context object if the given expression evaluates to true.


Examples:
Examples:
<pre>
<pre>
Array(1, 2, 3)
[1,2,3,4].Where(_>2)
Returns: An array having three elements which are 1, 2 and 3.
Returns: [3,4]
Also following syntax can be used: [1, 2, 3]


Array()
EventLogById(1).Cases.Where(Duration > TimeSpan(10))
Returns: An empty array.
Returns all cases where duration is more than 10 days (in eventlog id 1).
 
EventLogById(1).Events.Where(Timestamp < DateTime(2020, 1,1))
Returns all events which occurred before 1.1.2020 (in eventlog id 1).
</pre>
</pre>
|-
|-
||Concat (Array)
||<span id="If">If</span>
||
||
# Hierarchical object
# Condition expression
# True expression
# False expression
||
||
Concatenate multiple arrays together into one. Returns a single array which contains all the arrays given as parameter concatenated together in the same order they were given. If a parameter is not an array, it will be converted into an array having only the given element.
If the first parameter is true, returns the value of the second parameter. Otherwise returns the value of the third parameter. The function always evaluates only either the second or third parameter, but never both.
 
Note that the function needs to starts with a capital letter, because otherwise it's interpreted as the [[QPR_ProcessAnalyzer_Expressions#Conditional_operator_.28if.29|if operator]].


Examples:
Examples:
<pre>
<pre>
Concat(1, [DateTime(2017), 3], [4, "foo"], "bar")
If(Now.Second % 2 == 0, "Even second", "Odd second")
Returns: [1, DateTime(2017), 3, 4, "foo", "bar"]
Returns:
"Event second" or "Odd second" depending on the time of the evaluation.
 
For("i", 0, i < 10, i + 1, i).If(_ % 2 == 0, _, _remove)
Returns:
[0, 2, 4, 6, 8]
</pre>
</pre>
|-
|-
||ConcatTop (Hierarchical object)
||<span id="RemoveNulls">RemoveNulls (Array)</span>
||
||
# Hierarchical object
Array
||
||
Concatenate the topmost arrays in a given hierarchical object. Returns a new hierarchical object with the contents of the topmost level concatenated and removed.
Removes all the supported null values from given hierarchical object. Never removes arrays containing removed null values. Returns given object with null values removed.


Examples:
Examples:
<pre>
<pre>
ConcatTop([ 0, [1,2], [3], [4,[6,7]] ])
RemoveNulls([1, [], [null], [2, _remove, [_empty, 3, 4], _empty, null, 5]])
Returns: [1, 2, 3, 4, [6, 7]]
Returns: [1, [], [], [2, [3, 4], 5]]


ConcatTop([ [1,[2,3]], [4,[6,7]] ])
RemoveNulls(["foo": null, "bar": 2, "poof": [1, _empty, 3]])
Returns: [1, [2, 3], 4, [6, 7]]
Returns: [
  "foo": 0,
  "bar": 2,
  "poof": [ 1, 3 ]
]
</pre>
</pre>
|-
|-
||CountTop (Integer)
||Catch
||
||
# Hierarchical object
# Expression to calculate
# Result if exception (Object)
||
||
Get the number of elements in the topmost level array of a given array or hierarchical object. If the parameter is not an array, zero is returned.
Calculates the given expression and if any exceptions are thrown during the calculation, catches that exception and returns the given result. Note that this function does not catch any syntactical errors.


Examples:
Examples:
<pre>
<pre>
CountTop([ 0, [1,2], [3], [4,[6,7]] ])
Catch(1, 1234)
Returns: 4
Returns: 1
 
Catch(undefined, 1234)
Returns: 1234


CountTop([[1,[2,3]],[4,[6,7]]])
Catch([1,2].undefined, 1234)
Returns: 2
Returns: 1234


CountTop(0)
Catch(EventLogById(-1), 1234)
Returns: 0
Returns: 1234
</pre>
</pre>
|-
|-
||GetAt (Object)
||AsParallel
||
||
# Index (Integer)
Additional parameters
# Array
||
||
Returns element at given index from the beginning of the array. The first item has index zero. If given a hierarchical object, returns the element at given index of the root level array. Throws a calculation exception if the given index does not exist in given object or given object is not an array.
Performs all the following chaining operations in parallel to improve performance. Items are divided into parts which size is determined by the ParallelTaskSize parameter, and each part is executed as an independent task in parallel. Parallel execution has a certain cost, so it might not be optimal to run each item as a separate task (increase ParallelTaskSize to decrease number of tasks). On the other hand, to large ParallelTaskSize leads to too few parallel tasks, and then all processing capacity is not used.
 
As a parameter, takes a dictionary that accepts optional property '''ParallelTaskSize''', which is the size of the segments the root array will be split into. If the length of the input array is not divisible by ParallelTaskSize, the last segment will have less items than this configured value. The default size is 1.
 
The AsParallel function call doesn't need to be part of the chaining, and thus it can be called before the chaining.


Examples:
Examples:
<pre>
<pre>
GetAt(0, [1,2,3])
Run several ODBC queries simultaneously:
Returns: 1
let regions = ["New York", "Los Angeles", "Dallas", "London"];
AsParallel(["ParallelTaskSize": 1]);
regions.ImportOdbc("...", "SELECT * FROM [Data] Where Region = '" + _ + "'")
 
Run a complex filtering operation for cases:
Cases.AsParallel(["ParallelTaskSize": 1000]).Where(...)
 
Sum(Sum(([NumberRange(1, 100)].AsParallel(["ParallelTaskSize": 1]).For("i", 0, i < 100000, i + 1, i))[0]))
Returns: 499995000000
The same expression without parallel processing:
Sum(Sum(NumberRange(1, 100).For("i", 0, i < 100000, i + 1, i)))
Returns: 499995000000
 
Sum([NumberRange(1, 100)].AsParallel(["ParallelTaskSize": 1]).WriteLog(_))
Returns: 5050
In addition, outputs all the numbers from 1 to 100 into log file in the order in which the tasks were executed.
 
The previous example can also be written like this:
AsParallel(["ParallelTaskSize": 1]);
Sum(NumberRange(1, 100).WriteLog(_));
Returns: 5050


GetAt(1, [[1, 2], [3, 4]])
Count((([el.Cases].AsParallel(["ParallelTaskSize": 1000]):AnalyzeConformance(cm))[0]).Where(_==null))
Returns: [3, 4]
Returns the number of cases conforming to a design/conformance model cm.
The same expression without parallel processing:
Count((el.Cases:AnalyzeConformance(cm)).Where(_==null))


GetAt(1, [[[5, 6]], [[1, 2], [3, 4]]])
AsParallel(["ParallelTaskSize": 1]);
Returns: [[1, 2], [3, 4]]
el.Cases.Events.Type.Name
Returns a hierarchical object having cases of given event log (el) as root objects and array of event type names of the events in each case in the order of occurrence.
The expressions related to every case in the returned hierarchical object are processed in a separate independent task (thus, each task will process "Events.Type.Name" expression in a separate task where the root object is a case).


GetAt(1, GetAt(1, [[[5, 6]], [[1, 2], [3, 4]]]))
[[123, 456, 789]].AsParallel(["ParallelTaskSize": 1]).ScriptById(_).Run()
Returns: [3, 4]
Runs scripts with id's 123, 456 and 789 simultaneously.
</pre>
</pre>
|-
|-
||GetAtReverse (Object)
||IsNull (Boolean)
||
||
# Index (Integer)
Value to test (Object)
# Array
||
||
Same as the GetAt function, except that the index is calculated from the end of the array.
Tests whether given object is null, _empty or _remove. Returns ''true'' if it is any of those. If given a hierarchical object, applies the function as described in at the leaf level.
 
Examples:
<pre>
ForEach("item", [1, "a", null, _empty, _remove], IsNull(item))
Returns: [false, false, true, true, true]
 
IsNull(["foo":[null,"a"], "bar":[2,null]])
Returns: [
  HierarchicalArray("foo", [true, false]),
  HierarchicalArray("bar", [false, true])
]
</pre>
|-
|-
||In (Boolean)
||IsNullTop (Boolean)
||
||
# Array where to search the object
Object to test (Object)
||
||
Function checks whether the current context object is in given array, and returns either ''true'' or ''false''. Note that the searched object is not given as a parameter, but it's the context object for the function (see the examples).
Tests whether given object is null, _empty or _remove. Returns ''true'' if it is any of those. The function does not aggregate values in hierarchical objects.


Examples:
Examples:
<pre>
<pre>
1.In([1,2,3])
ForEach("item", [1, "a", null, _empty, _remove], IsNullTop(item))
Returns: true
Returns: [false, false, true, true, true]


[1].In([[1],2,3])
IsNullTop(["foo":[null,"a"], "bar":[2,null]])
Returns: false
Returns: false
</pre>
|-
||GroupBy
||
# Array to group
# Group by expressions
||
Groups given array by given expressions. Returns the given array splitted into groups in a way that each specified group expression creates one level of hierarchical arrays having the root object the same as the group expression result.
Examples:
<pre>
GroupBy([1,2,2,3,3,4,5,5,4,4,4], _)
Returns:
[[1:[1], 2:[2, 2], 3:[3, 3], 4:[4, 4, 4, 4], 5:[5,5]]
</pre>
|-
||GroupByValue
||Array to group
||
Groups all unique values in given array. Returns the given array in a format which has all the values in the original array only once as root objects and the root objects in the original array as contents of the arrays inside contexts.
In a way, this just switches root objects of hierarchical arrays to be the actual values and actual values to be root objects (without duplicates).
For every item in the array, the behavior is as follows:
* If the item is not an array, the value is used also as the root object of the item.
* If the item is an array, all the values are treated separately.
* If the item is a hierarchical array, its root object will be used as root object and each separate value treated separately as scalars (ending up being root objects of the resulting object).


DateTime(2017).In([DateTime(2016), DateTime(2017)])
Examples:
Returns: true
<pre>
GroupByValue([1,2,2,3,3,4,5,5,4,4,4])
Returns:
(The same as GroupBy([1,2,2,3,3,4,5,5,4,4,4], _))
[1:[1], 2:[2, 2], 3:[3, 3], 4:[4, 4, 4, 4], 5:[5,5]]


Count(eventLog.Events.TimeStamp.DateTime(_.Year).In([DateTime(2012)]).Where(_==true))
GroupByValue([1:["a","b"],2:["b","c"],3:["c"],4:["d","e"],5:["d","d","d"]])
Returns: The number of events there are in the given event log that have time stamp for year 2012.
Returns:
["a":[1], "b":[1, 2], "c":[2, 3], "d":[4, 5, 5, 5], "e":[4]]
 
Get count of item "2" in the array:
Let(\"groupped\", GroupByValue([1,2,4,1,4,2,3,3,2,4])); Count(GetValueOfContext(2, groupped))
Returns: 3
</pre>
</pre>
|-
|-
||IndexOfSubArray (Integer)
||Coalesce
||
||
# Optional object on which to operate
# Object to coalesce
# Sub-array to search(array)
# Result if Null
# Array from where to search the sub-array (array)
||
||
Returns the indexes of given sub-array (1. parameter) within the given array (2. parameter). If not given, the array in the current context object is used. Returns starting indexes of all the occurrences of given sub-array within given array.
Returns the second parameter if the first parameter is null or empty. If the the given object is a hierarchical object, all the leaf level values are coalesced separately.
 
Examples:
Examples:
<pre>
<pre>
[[1,2,3,4,1,2,5]].IndexOfSubArray([1,2])
Coalesce(0, 1)
IndexOfSubArray([1,2], [1,2,3,4,1,2,5])
Returns: 0
Return: [0, 4]
 
Coalesce(null, 1)
Coalesce(_empty, 1)
Coalesce(_remove, 1)
All return: 1


[[1,2,3,4,1,2,5]].IndexOfSubArray([1,2,3,4,5])
Coalesce([[null, 1], [null, null]], 3)
Returns: []
Returns: [[3, 1], [3, 3]]


[[1,2,3,4,1,2,5],[3,4],[0,1,2,3]]:IndexOfSubArray([1,2])
Coalesce([[null, 1], 2], 3)
Returns:
Returns: [[3, 1], null]
[
 
  HierarchicalArray([1,2,3,4,1,2,5], [0,4]),
Coalesce([1, [null, 2], null], 3)
  HierarchicalArray([0,1,2,3], [1])
Returns: [1, [null, 2], 3]
]
</pre>
</pre>
|-
|-
||IsArray (Boolean)
||Transpose
||
||
# Object to test
Matrix to transpose
||
||
Tests whether given object is an array or hierarchical object. Returns ''true'' if it is.
Transposes the given matrix.


Examples:
Examples:
<pre>
<pre>
ForEach("item", [1, "a", null, [1,2,3], ["foo":[null,"a"], "bar":[2,null]]], IsArray(item))
Transpose([[1,2], [3,4], [1,4]])
Returns: [false, false, false, true, true]
Returns: [[1, 3, 1], [2, 4, 4]]
</pre>
</pre>
|-
|-
||Suffle (Array)
||GarbageCollection
||
||Calling this function performs a garbage collection in the QPR ProcessAnalyzer Server. The function returns the maximum generation before the garbage collection was run. This function is an experimental functionality, which purpose is to investigate the memory consumption and memory handling behavior of the QPR ProcessAnalyzer Server.
|-
||GetContext
||
||
# Array to be shuffled
Hierarchical array
||
||
Shuffles given array into a random order. Takes an array to be suffled as a parameter. Returns a dictionary which contains exactly the same elements as the input array. However, the order of elements in the returned array is randomized.
Returns the context of given hierarchical array, i.e. list of keys in the object. If the given object is not an hierarchical array, returns ''null''.


Examples:
Examples:
<pre>
<pre>
Shuffle([1, 2])
GetContext("a":1)
Returns: Either [1, 2] or [2, 1] both in about 50% of the calls.
Returns: "a"
 
["a":1, "b":2, 2:3].GetContext(_)
Returns: ["a", "b", 2]
 
([1,2,3]:(_*2)).GetContext(_)
Returns: [1, 2, 3]


Shuffle(NumberRange(0, 99))[NumberRange(0, 9)]
["a":1, 2, 3].GetContext(_)
Returns: An array containing 10 random unique values between 0 and 99 ("Random sample of size 10").
Returns: ["a", null, null]
</pre>
</pre>
|-
|-
||StringJoin (String)
||GetValueOfContext
||
||
# Separator between joined parts (String)
# Context object
# Array of items to join (Array)
# Hierarchical array
||
||
Joins the given array of values (converted to strings) by using given separator into a single string. If given a hierarchical object, applies the function as described in at the level that is one level up from the leaf level. The depth of the result is one level less than the object that was given as parameter.
Returns the value of specified context object in given hierarchical array, i.e. the value behind the key. If the key if found multiple times, the first occurrence is returned. Returns ''_empty'' if the given key was not found.


Examples:
Examples:
<pre>
<pre>
StringJoin(", ", [1,null,"foo",DateTime(2017)])
GetValueOfContext("bar", ["foo":1, "bar":2])
Returns: 1, , foo, 01/01/2017 00:00:00
Returns: [2]
 
GetValueOfContext("bar", ["foo":1, "bar":2, "bar":"second"])
Returns: [2]


StringJoin(", ", [[1,2], [3,4]])
GetValueOfContext("test", ["foo":1, "bar":2])
Returns: ["1, 2", "3, 4"]
Returns: _empty
</pre>
</pre>
|-
|-
||Zip (Array)
||<span id="SendEmail">SendEmail</span>
||Parameters dictionary
||Sends an email message using the provided parameters. [[PA_Configuration_database_table#SMTP_Server_Settings|SMTP server settings]] need to be configured to be able to send email messages. Following parameters are supported:
* '''From''' (string) (mandatory): Email address where the email message appears to be coming from. Doesn't need to be a working address. If the From address is not defined, the system configured from address is used ([[PA Configuration database table#SMTP_Server_Settings|more information]]).
* '''To''' (string array) (mandatory): List of email addresses where to send the message to. Note that there should be only one email address per string in the array.
* '''Subject''' (string): Subject (title) of the email message.
* '''Body''' (string): Body (text contents) of the email message.
* '''Cc''' (string array): List of email addresses where to send the email as Cc.
* '''Bcc''' (string array): List of email addresses where to send the email as Bcc.
* '''IsBodyHtml''' (boolean): Defines whether the email message body is html (''true'') or plain text (''false''). By default false.
* '''ReplyTo''' (string array): Defines a list of reply-to addresses for the email message.
* '''BodyEncoding''' (string): Defines encoding for the message body. Supported encodings are ''ASCIIEncoding'', ''UTF7Encoding'', ''UTF8Encoding'' (default), ''UnicodeEncoding'' and ''UTF32Encoding''. More information: https://docs.microsoft.com/en-us/dotnet/api/system.text.encoding?redirectedfrom=MSDN&view=netcore-3.1#remarks.
* '''SubjectEncoding''' (string): Defines encoding for the message subject. Supported encodings are same as in the ''BodyEncoding'' property. Default value is ''UTF8Encoding''.
* '''Priority''' (string): Defines the email message priority. Available values are ''High'', ''Normal'' (default) and ''Low''. More information: https://docs.microsoft.com/en-us/dotnet/api/system.net.mail.mailpriority.
* '''DeliveryNotification''' (string): Defines the delivery notifications for the email message. Specified as a comma separated list of following possible values ''Delay'', ''Never'', ''None'', ''OnFailure'' and ''OnSuccess''. More information: https://docs.microsoft.com/en-us/dotnet/api/system.net.mail.deliverynotificationoptions.
* '''Sender''' (string): Defines the sender's email address. Usually the sender doesn't need to be defined. The sender is additional information for the email server.
 
Example:
<pre>
SendEmail(#{
  "From": "example.from@address.com",
  "To": ["recipient.one@address.com", "recipient.two@address.com", "recipient.three@address.com"],
  "Subject": "Example E-mail",
  "Body": "QPR ProcessAnalyzer example script started running"
});
</pre>
|-
||WriteLog
||
||
# Arrays to be combined.
message (Object)
||
||
Combines multiple arrays into one. All the arrays must be of the same length. Returns an array where the number of elements equals to the number of elements in the input arrays. All the elements in the Nth position of the array are the concatenated values of Nth elements of each of the input arrays.
When used in a script, writes the given text to the script log. The log entry is also made to the QPR ProcessAnalyzer log file. If the provided parameter is not a string, it's converted into string.
 
The return value of the function is the provided message parameter, allowing to flexibly add the WriteLog into expressions (see the examples).


Examples:
Examples:
<pre>
<pre>
Zip([1,2,3], [4,5,6], [7,8,9])
WriteLog("Calculation executed.")
Returns: [[1, 4, 7], [2, 5, 8], [3, 6, 9]]
Writes to log: Calculation executed.


Zip([])
Sum(WriteLog([1, 2, 3, 4]))
Returns: []
Returns: 10
Also writes an entry into the log showing the [1, 2, 3, 4] -array in the pretty printed fashion.


Zip([1], [4,5], [7,8,9])
NumberRange(0, 4).(WriteLog("Iteration #" + (_ + 1)), _)
Gives an error
Returns: [0, 1, 2, 3, 4]
Also writes the following entries into log:
  Iteration #1
  Iteration #2
  Iteration #3
  Iteration #4
  Iteration #5
</pre>
</pre>
|}
=== Set functions ===
{| class="wikitable"
!'''Function'''
!'''Parameters'''
! '''Description'''
|-
|-
||Distinct
||TakeSample
||
||
# Array or hierarchical object
* Input array/DataFrame
* Sample size (Integer)
* Seed (Integer)
* With replacements (Boolean)
||
||
Modify given array by filtering out all duplicate values leaving only distinct values in the input array into the result. If given a hierarchical object, applies the function at the level that is one level up from the leaf level.
Randomly chooses the defined number of items from the provided array or rows from the DataFrame, and returns a new array or DataFrame containing those items. If the sample size is larger than the number of items/rows, the TakeSample function has no effect. Note that the order of items may change from the original.
 
The seed is optional, and if not provided, the seed to be used is automatically generated.
 
The "with replacements" parameter is optional (by default false). When false, same items appear maximum of once in the resulting array. When true, same items may appear multiple times in the resulting array.


Examples:
Examples:
<pre>
<pre>
Distinct([1])
TakeSample([1,2,3,4,5,6,7,8,9,10], 4)
Returns: [1]
Returns [1, 9, 3, 4] (note: results may change)


Distinct([1, 1])
TakeSample([1,2,3,4,5,6,7,8,9,10], 5, 9, true)
Returns: [1]
Returns [5, 5, 1, 5, 3]


Distinct([1, 1, 2, 2])
TakeSample([1,2,3,4,5], 10)
Returns: [1, 2]
Returns [1,2,3,4,5]


Distinct([1, 1, 2, 2, "a", DateTime(2017), DateTime(2017), "b", DateTime(2016), "a"])
TakeSample(
Returns: [1, 2, "a", DateTime(2017), "b", DateTime(2016)]
  ToDataFrame(
    [[1,2],[3,4],[5,6]], ["a", "b"]
  ),
  2
).ToCsv()
Returns
a;b
5;6
3;4
</pre>
</pre>
|-
|-
||Intersect
||Eval
||Expression (String)
||
||
* First array
Evaluates the given expression. The Eval function can be used e.g., when the expression to be evaluated is generated during runtime, or is fetched during the script run from an external source. The expression is evaluated in the context where the Eval function exists, so it's possible to use variables defined outside the evaluated expression (see the example).
* Second array
 
* ...
Examples:
<pre>
Eval("1 + 1");
Result: 2
 
let myVar = 3;
Eval("myVar + 2");
Result: 5
</pre>
|-
||Sleep
||Duration (Timespan or integer as milliseconds)
||
||
Creates an intersection of multiple arrays. Returns a single array which contains an intersection of all the items in all the arrays given as parameter. If a parameter is not an array, it will be converted into an array having only the given element.
Suspends the expression execution for the given time duration. The parameter can either be a Timespan object or integer where the duration is specified as milliseconds.


Examples:
Examples:
<pre>
<pre>
Intersect([1, 2, "a", DateTime(2017), 5], ["a", 2, 3, DateTime(2017), 5], [DateTime(2017), "a", 2])
Sleep(Timespan(0, 0, 1)));
Returns: [2, "a", DateTime(2017)]
Result: Sleeps for 1 minute
 
Sleep(1000);
Sleep for 1 second
</pre>
</pre>
|-
|-
||Except
||ToSqlExpression
||
||Expression (String)
* First array
* Second array
* ...
||
||
 
Converts given string into SqlExpression. Note that the SqlExpression is not yet executed at this point.
Creates an substract given items from an array. Returns a single array which contains all the elements that were in the first array that were not in any of the subsequent arrays. If a parameter is not an array, it will be converted into an array having only the given element.


Examples:
Examples:
<pre>
<pre>
Except([1, 2, "a", DateTime(2017), 5, "b", 6], ["a", 2, 3, DateTime(2017), 5], [DateTime(2017), "a", 2, 6])
let expr = ToSqlExpression("Column(\"Cost\")");
Returns: [1, "b"]
</pre>
</pre>
|-
|-
||Union
||Query
||Query configuration (Object)
||
||
* First array
Runs expression query ([[Web_API:_Expression/query|Web API: Expression/query]]) and returns results as an in-memory DataFrame (for in-memory queries) or SqlDataFrame (for queries run in the datasource, e.g., in Snowflake). You can use charts to help with creating the configuration: Configure a chart, and click the ''Query'' button in the chart settings (''Advanced'' tab) to get the query configuration. Note that the json needs to be converted into expression language by adding hash characters (#) in the beginning of the curly braces of objects.
* Second array
 
* ...
Example: In-memory query:
||
<pre>
Creates an union of multiple arrays. Returns a single array which contains an union of all the items in all the arrays given as parameter. If a parameter is not an array, it will be converted into an array having only the given element.
let result = Query(#{
"ModelId": 123,
"Root": "Cases",
"MaximumRowCount": 10,
"Dimensions": [
#{ "Name": "Account manager", "Expression": "Attribute(\"Account Manager\")" }
],
"Values": [
#{ "Name": "Case count", "Expression": "Count(_)" }
],
"Ordering": [
#{ "Name": "Case count", "Direction": "Descending" }
]
});
</pre>


Examples:
Example: SqlDataFrame query ("Where" function is used additionally):
<pre>
<pre>
Union([1, 2, "a", DateTime(2017), 5], ["a", 2, 3, DateTime(2017), 5], [DateTime(2017), "a", 2, "b"])
let result = Query(#{
Returns: [1, 2, "a", DateTime(2017), 5, 3, "b"]
"ProcessingMethod": "dataframe",
"ContextType": "model",
"ModelId": 123,
"Root": "Cases",
"MaximumRowCount": 10,
"Dimensions": [
#{ "Name": "Account manager", "Expression": "Column(\"Account Manager\")" }
],
"Values": [
#{ "Name": "Case count", "AggregationFunction": "count" }
],
"Ordering": [
#{ "Name": "Case count", "Direction": "Descending" }
]
})
.Where(Like(Column("Account manager"), "Patricia%"))
.Collect();
</pre>
</pre>
|}
|}


=== Ordering functions ===
== Ordering functions ==


{| class="wikitable"
{| class="wikitable"
Line 510: Line 1,082:
|}
|}


=== Loop functions ===
== Looping functions ==
{| class="wikitable"
{| class="wikitable"
!'''Function'''
!'''Function'''
Line 541: Line 1,113:
</pre>
</pre>
|-
|-
||ForEach  
||ForEach (2 parameters)
||
# Array to iterate through
# Function to call
||
Calls given function using all the elements of given array, where the calculation is done in the current context. The function must support calling with one parameter. Returns the results of the evaluation of given function for every iteration of the input array as an array.
 
Examples:
<pre>
let sourceModel = First(Models.Where(Name == "SAP_OrderToCash"));
let attributes = sourceModel.EventAttributes;
let events = sourceModel.EventLog.Events;
let func = att => Attribute(att);
events.ForEach(attributes, func)
Returns an array containing arrays of all the event attribute values for all the events.
 
let myFunction = item => _ + item * 2;
(5).ForEach([1,2,3], myFunction)
Returns 7, 9, 11 (=5+1*2, 5+2*2, 5+3*2)
</pre>
|-
||ForEach (3 parameters)
||
||
# Variable to repeat (String)
# Variable to repeat (String)
Line 572: Line 1,165:
# Interval (Number)
# Interval (Number)
||
||
Creates an array of numbers within given range with the given interval. Interval parameter is optional, and by default it is one.
Creates an array of numbers within the given range using the given interval. Interval parameter is optional, and by default it is one. The interval can also be negative to get a sequence of decreasing numbers (then the Start needs to be greater than the End).


Examples:
Examples:
Line 578: Line 1,171:
NumberRange(1, 3)
NumberRange(1, 3)
Returns: [1, 2, 3]
Returns: [1, 2, 3]
NumberRange(-7, -1, 2)
Returns: [-7, -5, -3, -1]


NumberRange(1, 3, 0.5)
NumberRange(1, 3, 0.5)
Line 584: Line 1,180:
NumberRange(1, 3, 0.8)
NumberRange(1, 3, 0.8)
Returns: [1, 1.8, 2.6]
Returns: [1, 1.8, 2.6]
NumberRange(6, 2, -2)
Returns: [6, 4, 2]
</pre>
</pre>
|-
|-
Line 618: Line 1,217:
|}
|}


=== Recursion functions ===
== Recursion functions ==


{| class="wikitable"
{| class="wikitable"
Line 700: Line 1,299:
|}
|}


=== Variable handling functions ===
== Hierachical object functions ==
 
{| class="wikitable"
!'''Function'''
!'''Parameters'''
! '''Description'''
|-
||Def
||
# Function name (String)
# Variable 1 name (String)
# Variable 2 name (String)
# ...
# Function expression
||
Creates a new '''user defined function'''. Parameters starting from the second, are the parameters that user gives when calling the function. The last parameter is the expression to evaluate when the function is called. In that definition expression the named parameters are used. The created function is valid only within the current scope and all its child scopes.
 
Examples:
<pre>
Def("Inc", "a", a + 1); Inc(2);
Returns: 3
 
Def("Add", "a", "b", a + b); [1, 2, 3].Add(_, 2);
Returns: [3, 4, 5]
 
Def("AddSelf", "b", _ + b); [1, 2, 3].AddSelf(2);
Returns: [3, 4, 5]
 
Def("Fib", "a", If(a < 2, 1, Fib(a - 1) + Fib(a - 2))); Fib(10);
Returns: 89
</pre>
|-
||Let
||
# Variable 1 name (String)
# Variable 1 value (Object)
# Variable 2 name (String)
# Variable 2 value (Object)
# ...
# Expression to evaluate
||
Defines one or several '''user defined variables''' which can be used in the expression provided as the last parameter. Note that user defined variables cannot override existing properties.
 
Examples:
<pre>
Let("var1", "Orange"); "Value is " + var1;
Returns: Value is Orange
 
Let("var1", "Orange", "var2", "Mango"); "Values are " + var1 + " and " + var2);
Returns: Values are Orange and Mango
</pre>
|-
||Set
||
# Variable 1 name (String)
# Variable 1 value (Object)
# Variable 2 name (String)
# Variable 2 value (Object)
# ...
||
Sets a value for an existing '''user defined variable'''. Can also be used to set several variable values at once. The variable to set must have been created in some visible scope by Let function. There can be any number of variable name and variable value pairs as long as the number of parameters is even number.
 
If the number of parameters was two, the function returns that single set value. If the number of parameters is more than two, the function returns an array of all the set values.
|-
||Variable
||
# Variable name (String)
||
Function to get a variable value. The function is needed when there are spaces in a variable name, because that variable cannot be referenced otherwise in the expressions.
|}
 
=== JSON functions ===
 
{| class="wikitable"
!'''Function'''
!'''Parameters'''
! '''Description'''
|-
||ParseJson
||jsonData (String)
||
1. Converts given JSON string into an object.
1.1. Uses https://docs.microsoft.com/en-us/dotnet/api/system.web.script.serialization.javascriptserializer.deserialize?view=netframework-4.7.2 for de-serialization of the object.
 
JSON objects will be represented as StringDictionary objects in the expression language.
 
Examples:
<pre>
ParseJson("{\"a\": 1, \"b\": 2}").Get("b")
Returns: 2
 
Sum(ParseJson("[1, 2, 3, 4]"))
Returns: 10
 
ToJson(ParseJson("{\"a\": 1, \"b\": 2}"))
Returns: {"a":1,"b":2}
</pre>
|-
||ToJson
||Object
||
Converts given object into JSON. Uses https://docs.microsoft.com/en-us/dotnet/api/system.web.script.serialization.javascriptserializer.serialize?view=netframework-4.7.2 for serialization of the object.
 
It is recommended that hierarchical arrays (#29290#) and objects (#29286#) are first converted to dictionaries (#48320#) before converting to JSON since JSON does not have exactly identical construct and thus converting JSON back to expression object will not result object similar to the original.
 
Examples:
<pre>
ToJson(ParseJson("{\"a\": 1, \"b\": 2}"))
Returns: {"a":1,"b":2}
 
ToJson(ToDictionary().Set("a", 1, "b", 2))
Returns: {"a":1,"b":2}
 
ToJson([1,[[2,ToDictionary().Set("a", 1)],4]])
Returns: [1,[[2,{"a":1}],4]]
 
ToJson(null)
Returns: null
 
ToJson(_empty)
Returns: {}
 
ToJson(["a": 1, "b": 2])
Returns: [{"Root":"a","Array":[1]},{"Root":"b","Array":[2]}]
</pre>
|}
 
=== Matrix functions ===
 
{| class="wikitable"
!'''Function'''
!'''Parameters'''
! '''Description'''
|-
||BalancedKMeans
||jsonData (String)
||
Performs BalancedKMeans -clustering for given numeric matrix.
1.1. Based on: http://accord-framework.net/docs/html/T_Accord_MachineLearning_BalancedKMeans.htm
2. Parameters and return value structure identical to KMeans function (#48341#)
|-
||Codify
||Matrix to codify
||
Use Accord's Codify -functionality to encode all unique column values into unique numeric integer values. Based on: http://accord-framework.net/docs/html/T_Accord_Statistics_Filters_Codification.htm. Returns codified matrix of exactly the same dimensions as the input matrix.
 
Examples:
<pre>
Codify([[1,2], [3,4], [1,4]])
Returns: [[0, 0], [1, 1], [0, 1]]
 
Codify([[123, "foo"], [456, "bar"], [456, "foo"]])
Returns: [[0, 0], [1, 1], [1, 0]]
</pre>
|-
||KMeans
||
||
Performs KMeans -clustering for given numeric matrix. Based on: http://accord-framework.net/docs/html/T_Accord_MachineLearning_KMeans.htm
2. Parameters:
2.1. matrix: Matrix to cluster.
2.1.1. Rows (1st dimension) represent data points and columns represent feature values (2nd dimension)
2.2. k: Target number of clusters
2.3. distanceFunction: Distance function to be used in the clustering process (#48347#).
2.4. parameters: Optional key value pair collection as described in #34201#.
2.4.1. Supported keys and values:
2.4.1.1. ComputeCovariance: If true, the result will include covariance matrices.
2.4.1.1.1. Default = false
3. Returns an array having the following elements:
3.1. Element 0: An array of all the cluster labels for all the rows in the input matrix in the same order as they were given in the matrix parameter.
3.2. Element 1: An array of length 2 having the following elements:
3.2.1. Element 0: Computed final error of the clustering.
3.2.2. Element 1: Number of iterations performed in the clustering.
3.3. Element 2: Only returned if computeCovariance is True.
 
Examples:
<pre>
KMeans([[1, 2], [2, 3], [2, 2]], 2)
Returns (e.g.): [[0, 1, 0], [0.16667, 2]]
 
KMeans([[1, 2], [2, 3], [2, 2]], 3)
Returns (e.g.): [[2, 1, 0], [0, 1]]
 
KMeans([[1, 2], [2, 3], [2, 2]], 2, "manhattan", true)
Returns (e.g.): [[0, 1, 0], [0.33333, 2], <covariance matrices (k * columns * columns)>]
 
KMeans(OneHot(Codify([[123, "foo"], [456, "bar"], [456, "foo"]])), 2)
Returns (e.g.): [[0, 1, 0], [0.33333, 2]]
</pre>
|-
||KModes
||
||
Performs KModes -clustering for given numeric matrix. Based on: http://accord-framework.net/docs/html/T_Accord_MachineLearning_KModes.htm
2. Parameters:
2.1. matrix: Matrix to cluster.
2.1.1. Rows (1st dimension) represent data points and columns represent feature values (2nd dimension)
2.2. k: Target number of clusters
2.3. distanceFunction: Distance function to be used in the clustering process (#48347#).
3. Returns an array having the following elements:
3.1. Element 0: An array of all the cluster labels for all the rows in the input matrix in the same order as they were given in the matrix parameter.
3.2. Element 1: An array of length 2 having the following elements:
3.2.1. Element 0: Computed final error of the clustering.
3.2.2. Element 1: Number of iterations performed in the clustering.
 
Examples:
<pre>
KModes([[1, 2], [2, 3], [2, 2]], 2)
Returns (e.g.): [[0, 1, 0], [0, 2]]
 
KModes([[1, 2], [2, 3], [2, 2]], 3)
Returns (e.g.): [[2, 1, 0], [0, 1]]
</pre>
||
|-
||OneHot
||
||
One-hot encode all matrix columns. Based on: http://accord-framework.net/docs/html/M_Accord_Math_Jagged_OneHot_1.htm
2. Parameters:
2.1. matrix: Numeric matrix to one-hot encode.
3. Returns a matrix consisting of a concatenation of one-hot encoding of each of the input matrix columns.
3.1. The number of columns in the returned matrix is at least the same as in the input matrix.
3.2. For each input column, the corresponding one-hot vector will have all the values of 0, except for one which will be 1.
 
Examples:
<pre>
OneHot([[0], [2], [1], [3]])
Returns: [[1, 0, 0, 0], [0, 0, 1, 0], [0, 1, 0, 0], [0, 0, 0, 1]]
 
OneHot(Codify([[123, "foo"], [456, "bar"], [456, "foo"]]))
Returns: [[1, 0, 1, 0], [0, 1, 0, 1], [0, 1, 1, 0]]
</pre>
|}
 
=== Hierachical object functions ===
{| class="wikitable"
{| class="wikitable"
!'''Function'''
!'''Function'''
Line 1,091: Line 1,455:
|}
|}


=== [[Machine_Learning_Functions_in_Expression_Language|Machine Learning Functions]]===
== Snowflake connection functions ==
 
=== Other functions ===
 
{| class="wikitable"
{| class="wikitable"
!'''Function'''
!'''Function'''
Line 1,100: Line 1,461:
! '''Description'''
! '''Description'''
|-
|-
||Catch
||<span id="CallStoredProcedure">CallStoredProcedure</span>
||
# Expression to calculate
# Result if exception (Object)
||
Calculates the given expression and if any exceptions are thrown during the calculation, catches that exception and returns the given result. Note that this function does not catch any syntactical errors.
 
Examples:
<pre>
Catch(1, 1234)
Returns: 1
 
Catch(undefined, 1234)
Returns: 1234
 
Catch([1,2].undefined, 1234)
Returns: 1234
 
Catch(EventLogById(-1), 1234)
Returns: 1234
</pre>
|-
||Coalesce
||
# Object to coalesce
# Result if Null
||
Returns the second parameter if the first parameter evaluates to null or empty. If given a hierarchical object, applies the function at the leaf level. If the the given object is a hierarchical object, all its leaf level values are coalesced separately.
 
Examples:
<pre>
Coalesce(0, 1)
Returns: 0
 
Coalesce(null, 1)
Coalesce(_empty, 1)
Coalesce(_remove, 1)
All return: 1
 
Coalesce([[null, 1], [null, null]], 3)
Returns: [[3, 1], [3, 3]]
 
Coalesce([[null, 1], 2], 3)
Returns: [[3, 1], null]
 
Coalesce([1, [null, 2], null], 3)
Returns: [1, [null, 2], 3]
</pre>
|-
||DesignModelFromXml
||
* BPMN 2.0 XML document (String)
||
Creates new ([[Conformance_Checking_in_QPR_ProcessAnalyzer#DesignModel|DesignModel]] object) from BPMN 2.0 XML document provided as a string. (BPMN 2.0 specification: https://www.omg.org/spec/BPMN/).
 
In the BPMN 2.0 XML file, the ''<nowiki>http://www.omg.org/spec/BPMN/20100524/DI</nowiki>'' section is not used by the function, and thus it can be omitted to reduce transferred data.
 
Note that the quotation marks need to be escaped if the BPMN 2.0 XML is provided as a literal in the expression.
 
Example:
<pre>
Let("myConformanceModel", DesignModelFromXml("
  <?xml version=\"1.0\" encoding=\"UTF-8\"?>
    <bpmn:definitions xmlns:xsi=\"http://www.w3.org/2001/XMLSchema-instance\" xmlns:bpmn=\"http://www.omg.org/spec/BPMN/20100524/MODEL\" xmlns:bpmndi=\"http://www.omg.org/spec/BPMN/20100524/DI\">
    <bpmn:process id=\"Process_1\" isExecutable=\"false\">
    <bpmn:startEvent id=\"StartEvent_1\" />
    <bpmn:task id=\"Task_0y5w837\" name=\"abc\">
      <bpmn:outgoing>SequenceFlow_10mpkws</bpmn:outgoing>
    </bpmn:task>
    <bpmn:task id=\"Task_0c16r07\" name=\"def\">
      <bpmn:incoming>SequenceFlow_10mpkws</bpmn:incoming>
    </bpmn:task>
    <bpmn:sequenceFlow id=\"SequenceFlow_10mpkws\" sourceRef=\"Task_0y5w837\" targetRef=\"Task_0c16r07\" />
  </bpmn:process>
  <bpmndi:BPMNDiagram id=\"BPMNDiagram_1\">
    ...
  </bpmndi:BPMNDiagram>
</bpmn:definitions>
"))
</pre>
|-
||EventLogById
||
* FilterId (Integer)
||
Returns [[Object_Properties_and_Functions_in_QPR_ProcessAnalyzer_Expressions#EventLog|EventLog]] object corresponding to the provided filter Id.
|-
||GetContext
||
* Hierarchical array
||
Returns the context of given hierarchical array, i.e. list of keys in the object. If the given object is not an hierarchical array, returns ''null''.
 
Examples:
<pre>
GetContext("a":1)
Returns: "a"
 
["a":1, "b":2, 2:3].GetContext(_)
Returns: ["a", "b", 2]
 
([1,2,3]:(_*2)).GetContext(_)
Returns: [1, 2, 3]
 
["a":1, 2, 3].GetContext(_)
Returns: ["a", null, null]
</pre>
|-
||GetValueOfContext
||
# Context object
# Hierarchical array from which the value is searched
||
Returns the value of specified context object in given hierarchical array, i.e. the value behind the key. If the key if found multiple times, the first occurrence is returned. Returns ''_empty'' if the given key was not found.
 
Examples:
<pre>
GetValueOfContext("bar", ["foo":1, "bar":2])
Returns: [2]
 
GetValueOfContext("bar", ["foo":1, "bar":2, "bar":"second"])
Returns: [2]
 
GetValueOfContext("test", ["foo":1, "bar":2])
Returns: _empty
</pre>
|-
||ModelById
||
* ModelId (Integer)
||
Returns [[Object_Properties_and_Functions_in_QPR_ProcessAnalyzer_Expressions#Model|Model]] object corresponding to the provided model Id.
|-
||GroupBy
||
# Array or hierarchical object to group
# groupByExpressions
||
||
Groups given array by given expressions. Returns the given array or hierarchical object parameter split into groups in a way that each specified group expression creates one level of hierarchical arrays having the root object the same as the result of the evaluation of the grouping expression.
# Procedure name (string)
 
# Procedure parameters (dictionary)
Examples:
# Additional parameters (dictionary)
<pre>
GroupBy([1,2,2,3,3,4,5,5,4,4,4], _)
 
Returns:
[[1:[1], 2:[2, 2], 3:[3, 3], 4:[4, 4, 4, 4], 5:[5,5]]
</pre>
|-
||GroupByValue
||* Array or hierarchical object to group
||
||
Groups all unique values in given array or hierarchical object. Returns the given array or hierarchical object in a format which has all the values in the original array only once as root objects and the root objects in the original array as contents of the arrays inside contexts.
Calls a Snowflake stored procedure (it needs to pre-exist in the Snowflake when this function is called). The function returns the return value of the stored procedure. If the result is a table, its contents will be returned as an in-memory DataFrame. Object results are returned as JSON strings.
 
In a way, this just switches root objects of hierarchical arrays to be the actual values and actual values to be root objects (without duplicates).
 
For every item in the array, the behavior is as follows:
* If the item is a scalar value (not array nor hierarchical object), the value is used also as root object of the item.
* If the item is an array (not scalar), all the values are treated separately.
* If the item is a hierarchical array, its root object will be used as root object and each separate value treated separately as scalars (ending up being root objects of the resulting object).
 
Examples:
<pre>
GroupByValue([1,2,2,3,3,4,5,5,4,4,4])
Returns:
(The same as GroupBy([1,2,2,3,3,4,5,5,4,4,4], _))
[1:[1], 2:[2, 2], 3:[3, 3], 4:[4, 4, 4, 4], 5:[5,5]]


GroupByValue([1:["a","b"],2:["b","c"],3:["c"],4:["d","e"],5:["d","d","d"]])
Parameters:
Returns:
# '''Procedure name''': Name of the called stored procedure.
["a":[1], "b":[1, 2], "c":[2, 3], "d":[4, 5, 5, 5], "e":[4]]
# '''Procedure parameters''': Dictionary of parameters passed to the stored procedure. Note: If the parameter name starts with an underscore (_), it's not be passed to the procedure call.
# '''Additional parameters''': Dictionary of additional parameters affecting how the stored procedure is called. Supports parameter '''HasTabularResult''' (default ''false'') which need to be ''true'' when the stored procedure is expected to return a tabular result, and ''false'' when a scalar result. Note: When using the HasTabularResult=true, Snowflake ODBC driver version needs to be at least 3.1.0 (due to bug fix in the driver).


Get count of item "2" in the array:
Security note: All QPR ProcessAnalyzer users can use the CallStoredProcedure function and call Snowflake stored procedures that are in the schema configured to the [[PA_Configuration_database_table#General_Settings|Snowflake connection string]]. Create only stored procedures to that schema, that are safe for any user to run.
Let(\"groupped\", GroupByValue([1,2,4,1,4,2,3,3,2,4])); Count(GetValueOfContext(2, groupped))
Returns: 3
</pre>
|-
||If
||
# Condition expression
# True expression
# False expression
||
Evaluates the condition expression and if it's true, returns the value of the second parameter. Otherwise returns the value of the third parameter. Always evaluates only either the second or the third parameter, but never both.


Examples:
Example 1: Stored procedure to calculate a sum of two numbers. Create following stored procedure to Snowflake:
<pre>
<pre>
If(Now.Second % 2 == 0, "Even second", "Odd second")
CREATE OR REPLACE PROCEDURE SUMOFNUMBERS(NUMBER1 NUMBER, NUMBER2 NUMBER)
Returns:
RETURNS NUMBER
"Event second" or "Odd second" depending on the time of the evaluation.
LANGUAGE SQL
 
EXECUTE AS OWNER
For("i", 0, i < 10, i + 1, i).If(_ % 2 == 0, _, _remove)
AS '
Returns:
BEGIN
[0, 2, 4, 6, 8]
  RETURN NUMBER1 + NUMBER2;
NOTE: Is equivalent to: For("i", 0, i < 10, i + 1, i).Where(_ % 2 == 0)
END';
</pre>
</pre>
|-
||IsNull (Boolean)
||
# Value to test (Object)
||
Tests whether given object is null, _empty or _remove. Returns ''true'' if it is any of those. If given a hierarchical object, applies the function as described in at the leaf level.


Examples:
The stored procedure can be called as follows:
<pre>
<pre>
ForEach("item", [1, "a", null, _empty, _remove], IsNull(item))
CreateSnowflakeConnection()
Returns: [false, false, true, true, true]
  .CallStoredProcedure(
    "SUMOFNUMBERS",
    #{
      "NUMBER1": 3,
      "NUMBER2": 5
    }
  );


IsNull(["foo":[null,"a"], "bar":[2,null]])
Returns: 8
Returns: [
  HierarchicalArray("foo", [true, false]),
  HierarchicalArray("bar", [false, true])
]
</pre>
</pre>
|-
||IsNullTop (Boolean)
||
# Object to test (Object)
||
Tests whether given object is null, _empty or _remove. Returns ''true'' if it is any of those. The function does not aggregate values in hierarchical objects.


Examples:
Example 2: Calls a stored procedure named train_ml_model with a dictionary parameter containing InputEvents key that has SQL query string as value which, when run in the same Snowflake connection, will return all the contents of data table. ColumnMappings key that has a dict-object containing column role mappings of the InputEvents data.
<pre>
<pre>
ForEach("item", [1, "a", null, _empty, _remove], IsNullTop(item))
let dt = ModelById(1).EventsDataTable;
Returns: [false, false, true, true, true]
CreateSnowflakeConnection()
 
  .CallStoredProcedure("train_ml_model", #{
IsNullTop(["foo":[null,"a"], "bar":[2,null]])
    "ColumnMappings": dt.ColumnMappings,
Returns: false
    "InputEvents": dt.SqlDataFrame
  });
</pre>
</pre>
|-
|-
||MLModel
||<span id="CreateSnowflakeConnection">CreateSnowflakeConnection</span>
||Parameters dictionary
||
||
||
Creates an object representing connection to Snowflake. Takes as a parameter a dictionary with following settings names:
Create a new binary classification model of given type.
* '''OdbcConnectionString''': Connection to Snowflake is made with this connection string.
2. Parameters:
* '''OdbcConnectionStringKey''': [[Storing_Secrets_for_Scripts|Secret name]] containing the ODBC connection string to connect to Snowflake.
2.1. type: Type of the prediction/classification model to create. Supported values are:
* '''ProjectId''': Project id to use as the context of the secure string. If not defined, project id is queried from the current context object. Project id needs to be available when using the ''OdbcConnectionStringKey'' setting.
2.1.1. binarygbm:
2.1.1.1. Based on: ML.NET's LightGBM
2.2. parameters: Additional parameters for the model #34201#.
2.2.1. Supported keys and values:
2.2.1.1. ComputeCovariance: If true, the result will include covariance matrices.
2.2.1.1.1. Default = false
3. Returns the created MLModel object.
|-
||ToInteger (Integer)
||
# object
||
Converts given object to an integer. Internally uses .NET Convert.ToInt64 -function with invariant culture. See https://msdn.microsoft.com/en-us/library/system.convert.toint64(v=vs.110).aspx for more information. If the object can not be converted into an integer, an exception will be thrown.
 
Examples:
<pre>
ToInteger(1.234)
Returns: 1
 
ToInteger(-1.5)
Returns: -2
 
ToInteger(1313.6)
Returns: 1314
 
ToInteger(123456789012345678)
Returns: 123456789012345678
 
ToInteger("5")
Returns: 5


ToInteger("-5")
If neither ''OdbcConnectionString'' or ''OdbcConnectionStringKey'' is defined, the system configured connection string will be used.
Returns: -5
</pre>
|-
||ToString (String)
||
# object
||
Converts given object to a string object representation optimized for debugging purposes. Internally uses .NET Convert.ToString -function with invariant culture for non-array objects. See https://msdn.microsoft.com/en-us/library/system.convert.tostring(v=vs.110).aspx for more information.


Examples:
Examples:
<pre>
<pre>
ToString(1) + " < " + ToString(2)
ProjectByName("My project")
Returns: "1 < 2".
  .CreateSnowflakeConnection(
 
    #{ "OdbcConnectionStringKey": "MyConnectionStringKey" }
"Case=\'" + ToString(a) + "\'"
  );
Returns: "Case='<string representation of object a>'"
 
ToString(DateTime(2017,1,2,3,4,5,6))
Returns: "2017-01-02T03:04:05"
</pre>
</pre>
|-
|-
||Where
||ToSqlDataFrame
||
||In-memory dataframe
# Condition expression
||Same functionality as in the [[QPR_ProcessAnalyzer_Objects_in_Expression_Language#ToSqlDataFrame|Model.ToSqlDataFrame]], except the SQL dataframe is created for the connection which the function is called for.
# False expression
||
Returns the context object if the given expression evaluates to true. Otherwise returns the second parameter. The second parameter is optional and it's by default EMPTY.


Examples:
Example: Create an SQL dataframe that is passed as parameter to a Snowflake procedure:
<pre>
<pre>
[1,2,3,4].Where(_>2)
let df = ToDataFrame([[0, "zero"], [2, "two"], [3, "three"]], ["id", "right"]);
Returns: [3,4]
let connection = project.CreateSnowflakeConnection();
 
connection.CallStoredProcedure("MyProcedure", #{"input": connection.ToSqlDataFrame(df)});
[1,2,3,4].Where(_>2, _+100)
Returns: [101, 102, 3,4]
 
[1,2,3].[_,_+1]
Returns: [[1, 2], [2, 3], [3, 4]]
 
[1,2,3].[_,_+1].Where(_>=3)
Returns: [[], [3], [3, 4]]
 
[1,2,3].[_,_+1].Where(_>=3, _remove)
Returns: [[3], [3, 4]]
</pre>
</pre>
|}
|}
[[Category: QPR UI]]

Latest revision as of 14:34, 6 March 2024

Aggregation functions

Following aggregation functions are available:

  • Average: Average of the items (sum of items divided by count of items). Can be used for numbers, DateTimes and Timespans.
  • Count: Number of items. Can be used for any type of data.
  • Median: Median, i.e. the middle value of the sorted data. If there are even number of items, the average of the two middle items. Can be used for numbers, DateTimes and Timespans.
  • Min: Lowest/first value in the data. Can be used for data that can be sorted.
  • Max: Highest/last value in the data. Can be used for data that can be sorted.
  • Percentile: Percentile value. Can be used for numbers, DateTimes and Timespans.
  • Stdev: Sample standard deviation. Can be calculated for numbers, DateTimes and Timespans.
  • Stdevp: Population standard deviation. Can be calculated for numbers, DateTimes and Timespans.
  • Sum: Sum of the items. Can be used for numbers and Timespans.
  • StringJoin: Joins array of strings together. The first parameter is a string that is placed between the joined strings.
  • Var: Sample variance. Can be used for numbers.
  • Varp: Population variance. Can be used for numbers.

Examples:

Sum([3, 2, 4])
Returns: 9

Count([[1, 2], [3, 4, 5]])
Returns: [2, 3]

Sum([[1, 2], [3, 4, 5]])
Returns: [3, 12]

Percentile([1,2,3,4,5], 0.75)
Return: 4

Sum([])
Returns: null

StringJoin(", ", ["one", "two", "three"])
Returns: one, two, three

When aggregating numerical values, null values are treated as zeros. Nulls can be removed with RemoveNulls function. In the Min and Max functions, nulls are ignored. Null values can be removed before aggregating (the following example shows how).

Average([1, 5, null])
Returns: 2

Average(RemoveNulls([1, 5, null])
Returns: 3

Aggregation operations are performed to the leaf level. i.e. the deepest level in the hierarchy. When aggregating, the leaf level arrays are replaced by the aggregated values, and thus the depth of the hierarchy decreases by one. In addition to the aggregation functions, functions that modify the contents of leaf arrays (OrderBy, Distinct, ...), the operation will be performed separately for every leaf array.

OrderByValue([[4, 3], [2, 1]])
Returns: [[3, 4], [1, 2]]

Mathematical functions

Function Parameters Description
Abs (Number) Number

Calculates the absolute value of a specified number. Examples:

Abs(4)
Returns: 4

Abs(-2.7)
Returns: 2.7

Abs(0)
Returns: 0
Acos (Number) Number

Calculates the angle whose cosine is the specified number. The parameter is a number representing a cosine, which must be between -1 and 1. Examples:

Acos(0.3584)
Returns: 1.204242852965772

Acos(-0.3584)
Returns: 1.9373498006240213
ArgMax (Object)
  1. Array or hierarchical object
  2. Value expression

Returns those objects in the array giving maximum values for the given expression when evaluated in the object's context. Examples:

ArgMax([5, 1, 6, 4, 3, 3, 6, 5, 4, 1, 1], _)
Returns: [6, 6]

ArgMax(EventLogById(1).Cases, Duration)
Returns: An array of cases having the longest duration.
ArgMin (Object)
  1. Array or hierarchical object
  2. Value expression

Returns those objects in the array giving minimum values for the given expression when evaluated in the object's context. Examples:

ArgMin([5, 1, 6, 4, 3, 3, 6, 5, 4, 1, 1], _)
Returns: [1, 1, 1]

ArgMin(EventLogById(1).Cases, Duration)
Returns: An array of cases having the shortest duration.
Asin (Number) Number

Calculates the angle whose sine is the specified number. The parameter is a number representing a sine, which must be between -1 and 1. Examples:

Asin(-0.3584)
Returns: 0.36655347382912462

Asin(+0.3584)
Returns: -0.36655347382912462
Atan (Number) Number

Calculates the angle whose tangent is the specified number. Examples:

Atan(1)
Returns: 0.78539816339744828

Atan(-1)
Returns: -0.78539816339744828
Ceiling (Integer)

Number

Returns the smallest integer that is greater than or equal to the specified number. Example:

Ceiling(1.3)
Returns: 2
Cos (Number) Number

Calculates the cosine of the specified angle measured in radians. Examples:

Cos(1)
Returns: 0.54030230586813977

Cos(-1)
Returns: 0.54030230586813977
Exp (Number) Number

Calculates the e raised to the specified power. Examples:

Exp(1)
Returns: 2.7182818284590451

Exp(-1)
Returns: 0.36787944117144233
Floor (Integer)

Number

Returns the largest integer that is less than or equal to the specified number. Example:

Floor(1.9)
Returns: 1
Hash (String) String to hash

Returns 11 characters long hash value for the given string. There can be following characters in the hash: A-Za-z0-9+/. For null or empty string, "AAAAAAAAAAA" is returned.

The hashing is a repeatable operation, i.e., the same input string always returns the same hash value. Note also that hashing lead to collisions, i.e., two different input strings may give the same hash value.

The hash is calculated using Murmur3 algorithm (https://blog.teamleadnet.com/2012/08/murmurhash3-ultra-fast-hash-algorithm.html) and the first 64 bits of the 128-bit hash value is returned encoded as base64url without padding (RFC 4648 §5).

Examples:

Hash("abcdef")
Returns: 3sqiubzVXxA
IEEERemainder (Number)
  1. Dividend (Number)
  2. Divisor (Number)

Calculates the remainder resulting from the division of a specified number by another specified number. Uses formula:

dividend - (divisor * Round(dividend / divisor))

Note that the formula is different than in the % operator, which uses formula:

(Abs(dividend) - (Abs(divisor) * (Floor(Math.Abs(dividend) / Abs(divisor))))) * Sign(dividend)  

Examples:

IEEERemainder(26, 4)
Returns: 2

IEEERemainder(0, 0)
Returns: NaN
Log (Number)
  1. Number
  2. Logarithm base (Number)

Calculates the logarithm of a specified number. If the second parameter is not provided, it's the natural logarithm (base e). The second parameter can be used to define another base for the logarithm. Examples:

Log(5)
Returns: 1.6094379124341003

Log(10, 2)
Returns: 3.3219280948873626
Log10 (Number)

Number

Calculates the base 10 logarithm of a specified number. Examples:

Log10(5)
Returns: 0.69897000433601886

Log10(-5)
Returns: NaN
Pow (Number)
  1. Base number
  2. Exponent number

Calculate a specified number raised to the specified power. Examples:

Pow(5, 3)
Returns: 125

Pow(-5, 3)
Returns: -125
Random (Number) (none) Returns a random number value between 0 and 1, where 0 is included, but 1 is not.

Example:

Random()
Returns: 0.1243234243 (for example)
Round (Float)

Number of decimals (Integer)

For numbers, rounds the number to the specified number of decimal. Number of decimals can also be negative, in which case the number is rounded to the nearest tens, hundreds, thousands, etc.

For DateTimes, rounds given date time by given time span or given number of seconds.

Note that the number and DateTime to be rounded needs to be provided as a context object (not as a parameter).

Examples:

(1.254).Round(1)
Returns: 1.3

(162.111111).Round(-2)
Returns: 200

DateTime(2017, 1, 2, 3, 4, 5).Round(10)
Returns: DateTime(2017, 1, 2, 3, 4, 10)

DateTime(2017, 1, 2, 3, 4, 5).Round(TimeSpan(1))
Returns: DateTime(2017, 1, 2)
Sign (Number)

Number

Calculates an integer that indicates the sign of a number, either -1, 0 or 1. Examples:

Sign(10)
Returns: 1

Sign(0)
Returns: 0

Sign(10)
Returns: -1
Sin (Number)

Number

Calculates the sine of the specified angle. Examples:

Sin(3.14)
Returns: 0.0015926529164868282

Sin(-3.14)
Returns: -0.0015926529164868282
Sqrt (Number)

Number

Calculates the square root of a specified number. Examples:

Sqrt(81)
Returns: 9

Sqrt(0.9)
Returns: 0.3
Tan (Number)

Number

Calculates the tangent of the specified angle. Examples:

Tan(1)
Returns: 1.5574077246549023

Tan(-1)
Returns: -1.5574077246549023

Conversion functions

Function Parameters Description
ToFloat (Number)

Object to convert

Converts the given object to a decimal number.
ToInteger (Integer)

Object to convert

Converts the given object into an integer. If the object can not be converted into an integer, an exception will be thrown. If the object as a decimal number, it will also be rounded to the nearest integer.

Examples:

ToInteger(1.234)
Returns: 1

ToInteger(-1.5)
Returns: -2

ToInteger(1313.6)
Returns: 1314

ToInteger(123456789012345678)
Returns: 123456789012345678

ToInteger("5")
Returns: 5

ToInteger("-5")
Returns: -5
ToString (String)
  1. Object to convert
  2. Date/Number/Timespan format (String)

Converts the given object to a string.

The optional second parameter defines formatting that is needed when converting the following data types:

Examples:

ToString(1) + " < " + ToString(2)
Returns: "1 < 2".

"Case=\'" + ToString(a) + "\'"
Returns: "Case='<string representation of object a>'"

ToString(DateTime(2017,1,2,3,4,5,6))
Returns: "2017-01-02T03:04:05"

ToString(DateTime(2017,1,2,3,4,5,6), "dd.MM.yyyy")
Returns "02.01.2017"

ToString(1.2, "0.00")
Returns "1.20"

ToString(TimeSpan(16,4,3,17,250), "%d")
Returns "16"

Data source connectors

Function Parameters Description
CallWebService (Dictionary) Dictionary of parameters

CallWebService is a generic function for any kind of http requests. Support natively REST-style web services by automatically formatting and parsing json in the request and response, so dictionary/array objects are used without converting them to strings (see examples below). If using xml-based web services, formatting and parsing of xml needs to be taken care of manually.

The function takes as a dictionary of following parameters:

  • Address (string): Url address to connect to.
  • Method (string): Http method to use, as one of the following: GET (default), POST, PUT, DELETE.
  • Body (string/dictionary/array): The http request body. In case the request content type is application/json, dictionary/array can be used directly without string conversion.
  • Content-Type (string): Defines the Content-Type http request header. If content type is application/x-www-form-urlencoded, form url encoding is applied for the request body.
  • Timeout (integer): Time to wait before the request times out, in case no response is received earlier. Defined as integer in milliseconds. By default 60000 milliseconds.
  • Headers (dictionary): Key-value pairs of HTTP request headers.
  • QueryParameters (dictionary): Key-value pairs of query parameters that are added to the url. Proper encoding of the url is taken care of by the function.
  • DefaultNetworkCredentials (boolean): When true, uses the default network credentials for the call. When false, the default network credentials are not used. The default network credentials can be used only if in the server configuration AllowForwardingNetworkCredentials setting is enabled.
  • Encoding (string): Defines the body encoding to use. Default value is UTF8. The supported options are listed in https://msdn.microsoft.com/en-us/library/system.text.encoding%28v=vs.110%29.aspx.

CallWebService returns a dictionary with properties:

  • Response (string/dictionary/array): Response data as string, or if content type is application/json, as dictionary/array object.
  • StatusCode (integer): Http status code.
  • Headers (dictionary): All http response headers.

The Content-Type http request header is set as follows: If the Content-Type parameter is provided, that is used. If Content-Type is not provided, and body is dictionary/array, request Content-Type is set to application/json. Otherwise no Content-Type header is set.

If the Content-Type parameter is application/x-www-form-urlencoded, body is expected to be a dictionary where the name/values pairs are encoded using FormUrlEncodedContent method (see examples below).

Examples:

Get a webpage and check the status code of the response:

let resp = CallWebService(#{
  "Address": "https://www.google.com/search?q=QPR+ProcessAnalyzer"
});
resp.StatusCode;

Get the server version in a QPR ProcessAnalyzer environment:

let resp = CallWebService(#{
  "Address": "https://processanalyzer.onqpr.com/qprpa/api/serverinfo"
});
resp.Response.serverVersion;

Authenticate to QPR ProcessAnalyzer (to get an access_token) and call expression query (note the Content-Type required by the token request):

let server = "http://processanalyzer.onqpr.com/qprpa";
let resp = CallWebService(#{
  "Address": server + "/token",
  "Method": "POST",
  "Content-Type": "application/x-www-form-urlencoded",
  "Body": #{
    "grant_type": "password",
    "username": "qpr",
    "password": "demo"
  }
});
resp = CallWebService(#{
  "Address": server + "/api/expression/query",
  "Method": "POST",
  "Content-Type": "application/json",
  "Headers": #{ "Authorization": "Bearer " + resp.Response.access_token },
  "Timeout": 120000,
  "Body": #{
    "Root": "Models",
    "Dimensions": null,
    "Values": [ #{"Name": "Name", "Expression": "Name"} ],
    "Ordering":[ #{"Name": "Name", "Direction": "Ascending"} ]
  }
});
resp.Response[0].Name

Upload contents of datatable id 321 to using Web API to a datatable id 123:

CallWebService(#{
  "Address": "https://processanalyzer.onqpr.com/qprpa/api/importfile?importMode=datatable&fileType=csv&objectId=123&append=true&csvTimeFormat=yyyy-MM-dd%20HH%3Amm%3Ass,fff",
  "Method": "POST",
  "Body": DatatableById(321).SqlDataFrame.Collect().ToCsv(),
  "Headers": #{ "Authorization": "Bearer " + resp.Response.access_token },
  "Timeout": 6000000
});
OpenAIChatCompletion (Dictionary) Request dictionary Calls OpenAI's Chat Completions API (https://platform.openai.com/docs/api-reference/chat) to generate natural language response to user prompt. Both the request and response is a dictionary following the Chat Completions API specification. As specified in the API, the model and messages are mandatory fields in the request.

To use the function, the OpenAi API key needs to be configured in OpenAIAPIKey configuration table.

Example: Response to a user request:

let response = OpenAIChatCompletion(#{
  "model":  "gpt-3.5-turbo",
  "messages": [
    #{
      "role": "system",
      "content": "You are a process mining expert making intelligent suggestions how to improve processes."
    },
    #{
      "role": "user",
      "content": "Give me three suggestions how to improve a sales process with bottlenecks."
    }
  ]
});
response["choices"][0]["message"]["content"];

Miscellaneous functions

Function Parameters Description
Where

Condition expression

Returns the context object if the given expression evaluates to true.

Examples:

[1,2,3,4].Where(_>2)
Returns: [3,4]

EventLogById(1).Cases.Where(Duration > TimeSpan(10))
Returns all cases where duration is more than 10 days (in eventlog id 1).

EventLogById(1).Events.Where(Timestamp < DateTime(2020, 1,1))
Returns all events which occurred before 1.1.2020 (in eventlog id 1).
If
  1. Condition expression
  2. True expression
  3. False expression

If the first parameter is true, returns the value of the second parameter. Otherwise returns the value of the third parameter. The function always evaluates only either the second or third parameter, but never both.

Note that the function needs to starts with a capital letter, because otherwise it's interpreted as the if operator.

Examples:

If(Now.Second % 2 == 0, "Even second", "Odd second")
Returns:
"Event second" or "Odd second" depending on the time of the evaluation.

For("i", 0, i < 10, i + 1, i).If(_ % 2 == 0, _, _remove)
Returns:
[0, 2, 4, 6, 8]
RemoveNulls (Array)

Array

Removes all the supported null values from given hierarchical object. Never removes arrays containing removed null values. Returns given object with null values removed.

Examples:

RemoveNulls([1, [], [null], [2, _remove, [_empty, 3, 4], _empty, null, 5]])
Returns: [1, [], [], [2, [3, 4], 5]]

RemoveNulls(["foo": null, "bar": 2, "poof": [1, _empty, 3]])
Returns: [
  "foo": 0,
  "bar": 2,
  "poof": [ 1, 3 ]
]
Catch
  1. Expression to calculate
  2. Result if exception (Object)

Calculates the given expression and if any exceptions are thrown during the calculation, catches that exception and returns the given result. Note that this function does not catch any syntactical errors.

Examples:

Catch(1, 1234)
Returns: 1

Catch(undefined, 1234)
Returns: 1234

Catch([1,2].undefined, 1234)
Returns: 1234

Catch(EventLogById(-1), 1234)
Returns: 1234
AsParallel

Additional parameters

Performs all the following chaining operations in parallel to improve performance. Items are divided into parts which size is determined by the ParallelTaskSize parameter, and each part is executed as an independent task in parallel. Parallel execution has a certain cost, so it might not be optimal to run each item as a separate task (increase ParallelTaskSize to decrease number of tasks). On the other hand, to large ParallelTaskSize leads to too few parallel tasks, and then all processing capacity is not used.

As a parameter, takes a dictionary that accepts optional property ParallelTaskSize, which is the size of the segments the root array will be split into. If the length of the input array is not divisible by ParallelTaskSize, the last segment will have less items than this configured value. The default size is 1.

The AsParallel function call doesn't need to be part of the chaining, and thus it can be called before the chaining.

Examples:

Run several ODBC queries simultaneously:
let regions = ["New York", "Los Angeles", "Dallas", "London"];
AsParallel(["ParallelTaskSize": 1]);
regions.ImportOdbc("...", "SELECT * FROM [Data] Where Region = '" + _ + "'")

Run a complex filtering operation for cases:
Cases.AsParallel(["ParallelTaskSize": 1000]).Where(...)

Sum(Sum(([NumberRange(1, 100)].AsParallel(["ParallelTaskSize": 1]).For("i", 0, i < 100000, i + 1, i))[0]))
Returns: 499995000000
The same expression without parallel processing:
Sum(Sum(NumberRange(1, 100).For("i", 0, i < 100000, i + 1, i)))
Returns: 499995000000

Sum([NumberRange(1, 100)].AsParallel(["ParallelTaskSize": 1]).WriteLog(_))
Returns: 5050
In addition, outputs all the numbers from 1 to 100 into log file in the order in which the tasks were executed.

The previous example can also be written like this:
AsParallel(["ParallelTaskSize": 1]);
Sum(NumberRange(1, 100).WriteLog(_));
Returns: 5050

Count((([el.Cases].AsParallel(["ParallelTaskSize": 1000]):AnalyzeConformance(cm))[0]).Where(_==null))
Returns the number of cases conforming to a design/conformance model cm.
The same expression without parallel processing:
Count((el.Cases:AnalyzeConformance(cm)).Where(_==null))

AsParallel(["ParallelTaskSize": 1]);
el.Cases.Events.Type.Name
Returns a hierarchical object having cases of given event log (el) as root objects and array of event type names of the events in each case in the order of occurrence.
The expressions related to every case in the returned hierarchical object are processed in a separate independent task (thus, each task will process "Events.Type.Name" expression in a separate task where the root object is a case).

[[123, 456, 789]].AsParallel(["ParallelTaskSize": 1]).ScriptById(_).Run()
Runs scripts with id's 123, 456 and 789 simultaneously.
IsNull (Boolean)

Value to test (Object)

Tests whether given object is null, _empty or _remove. Returns true if it is any of those. If given a hierarchical object, applies the function as described in at the leaf level.

Examples:

ForEach("item", [1, "a", null, _empty, _remove], IsNull(item))
Returns: [false, false, true, true, true]

IsNull(["foo":[null,"a"], "bar":[2,null]])
Returns: [
  HierarchicalArray("foo", [true, false]),
  HierarchicalArray("bar", [false, true])
]
IsNullTop (Boolean)

Object to test (Object)

Tests whether given object is null, _empty or _remove. Returns true if it is any of those. The function does not aggregate values in hierarchical objects.

Examples:

ForEach("item", [1, "a", null, _empty, _remove], IsNullTop(item))
Returns: [false, false, true, true, true]

IsNullTop(["foo":[null,"a"], "bar":[2,null]])
Returns: false
GroupBy
  1. Array to group
  2. Group by expressions

Groups given array by given expressions. Returns the given array splitted into groups in a way that each specified group expression creates one level of hierarchical arrays having the root object the same as the group expression result.

Examples:

GroupBy([1,2,2,3,3,4,5,5,4,4,4], _)

Returns:
[[1:[1], 2:[2, 2], 3:[3, 3], 4:[4, 4, 4, 4], 5:[5,5]]
GroupByValue Array to group

Groups all unique values in given array. Returns the given array in a format which has all the values in the original array only once as root objects and the root objects in the original array as contents of the arrays inside contexts.

In a way, this just switches root objects of hierarchical arrays to be the actual values and actual values to be root objects (without duplicates).

For every item in the array, the behavior is as follows:

  • If the item is not an array, the value is used also as the root object of the item.
  • If the item is an array, all the values are treated separately.
  • If the item is a hierarchical array, its root object will be used as root object and each separate value treated separately as scalars (ending up being root objects of the resulting object).

Examples:

GroupByValue([1,2,2,3,3,4,5,5,4,4,4])
Returns:
(The same as GroupBy([1,2,2,3,3,4,5,5,4,4,4], _))
[1:[1], 2:[2, 2], 3:[3, 3], 4:[4, 4, 4, 4], 5:[5,5]]

GroupByValue([1:["a","b"],2:["b","c"],3:["c"],4:["d","e"],5:["d","d","d"]])
Returns:
["a":[1], "b":[1, 2], "c":[2, 3], "d":[4, 5, 5, 5], "e":[4]]

Get count of item "2" in the array:
Let(\"groupped\", GroupByValue([1,2,4,1,4,2,3,3,2,4])); Count(GetValueOfContext(2, groupped))
Returns: 3
Coalesce
  1. Object to coalesce
  2. Result if Null

Returns the second parameter if the first parameter is null or empty. If the the given object is a hierarchical object, all the leaf level values are coalesced separately.

Examples:

Coalesce(0, 1)
Returns: 0

Coalesce(null, 1)
Coalesce(_empty, 1)
Coalesce(_remove, 1)
All return: 1

Coalesce([[null, 1], [null, null]], 3)
Returns: [[3, 1], [3, 3]]

Coalesce([[null, 1], 2], 3)
Returns: [[3, 1], null]

Coalesce([1, [null, 2], null], 3)
Returns: [1, [null, 2], 3]
Transpose

Matrix to transpose

Transposes the given matrix.

Examples:

Transpose([[1,2], [3,4], [1,4]])
Returns: [[1, 3, 1], [2, 4, 4]]
GarbageCollection Calling this function performs a garbage collection in the QPR ProcessAnalyzer Server. The function returns the maximum generation before the garbage collection was run. This function is an experimental functionality, which purpose is to investigate the memory consumption and memory handling behavior of the QPR ProcessAnalyzer Server.
GetContext

Hierarchical array

Returns the context of given hierarchical array, i.e. list of keys in the object. If the given object is not an hierarchical array, returns null.

Examples:

GetContext("a":1)
Returns: "a"

["a":1, "b":2, 2:3].GetContext(_)
Returns: ["a", "b", 2]

([1,2,3]:(_*2)).GetContext(_)
Returns: [1, 2, 3]

["a":1, 2, 3].GetContext(_)
Returns: ["a", null, null]
GetValueOfContext
  1. Context object
  2. Hierarchical array

Returns the value of specified context object in given hierarchical array, i.e. the value behind the key. If the key if found multiple times, the first occurrence is returned. Returns _empty if the given key was not found.

Examples:

GetValueOfContext("bar", ["foo":1, "bar":2])
Returns: [2]

GetValueOfContext("bar", ["foo":1, "bar":2, "bar":"second"])
Returns: [2]

GetValueOfContext("test", ["foo":1, "bar":2])
Returns: _empty
SendEmail Parameters dictionary Sends an email message using the provided parameters. SMTP server settings need to be configured to be able to send email messages. Following parameters are supported:
  • From (string) (mandatory): Email address where the email message appears to be coming from. Doesn't need to be a working address. If the From address is not defined, the system configured from address is used (more information).
  • To (string array) (mandatory): List of email addresses where to send the message to. Note that there should be only one email address per string in the array.
  • Subject (string): Subject (title) of the email message.
  • Body (string): Body (text contents) of the email message.
  • Cc (string array): List of email addresses where to send the email as Cc.
  • Bcc (string array): List of email addresses where to send the email as Bcc.
  • IsBodyHtml (boolean): Defines whether the email message body is html (true) or plain text (false). By default false.
  • ReplyTo (string array): Defines a list of reply-to addresses for the email message.
  • BodyEncoding (string): Defines encoding for the message body. Supported encodings are ASCIIEncoding, UTF7Encoding, UTF8Encoding (default), UnicodeEncoding and UTF32Encoding. More information: https://docs.microsoft.com/en-us/dotnet/api/system.text.encoding?redirectedfrom=MSDN&view=netcore-3.1#remarks.
  • SubjectEncoding (string): Defines encoding for the message subject. Supported encodings are same as in the BodyEncoding property. Default value is UTF8Encoding.
  • Priority (string): Defines the email message priority. Available values are High, Normal (default) and Low. More information: https://docs.microsoft.com/en-us/dotnet/api/system.net.mail.mailpriority.
  • DeliveryNotification (string): Defines the delivery notifications for the email message. Specified as a comma separated list of following possible values Delay, Never, None, OnFailure and OnSuccess. More information: https://docs.microsoft.com/en-us/dotnet/api/system.net.mail.deliverynotificationoptions.
  • Sender (string): Defines the sender's email address. Usually the sender doesn't need to be defined. The sender is additional information for the email server.

Example:

SendEmail(#{
  "From": "example.from@address.com",
  "To": ["recipient.one@address.com", "recipient.two@address.com", "recipient.three@address.com"],
  "Subject": "Example E-mail",
  "Body": "QPR ProcessAnalyzer example script started running"
});
WriteLog

message (Object)

When used in a script, writes the given text to the script log. The log entry is also made to the QPR ProcessAnalyzer log file. If the provided parameter is not a string, it's converted into string.

The return value of the function is the provided message parameter, allowing to flexibly add the WriteLog into expressions (see the examples).

Examples:

WriteLog("Calculation executed.")
Writes to log: Calculation executed.

Sum(WriteLog([1, 2, 3, 4]))
Returns: 10
Also writes an entry into the log showing the [1, 2, 3, 4] -array in the pretty printed fashion.

NumberRange(0, 4).(WriteLog("Iteration #" + (_ + 1)), _)
Returns: [0, 1, 2, 3, 4]
Also writes the following entries into log:
  Iteration #1
  Iteration #2
  Iteration #3
  Iteration #4
  Iteration #5
TakeSample
  • Input array/DataFrame
  • Sample size (Integer)
  • Seed (Integer)
  • With replacements (Boolean)

Randomly chooses the defined number of items from the provided array or rows from the DataFrame, and returns a new array or DataFrame containing those items. If the sample size is larger than the number of items/rows, the TakeSample function has no effect. Note that the order of items may change from the original.

The seed is optional, and if not provided, the seed to be used is automatically generated.

The "with replacements" parameter is optional (by default false). When false, same items appear maximum of once in the resulting array. When true, same items may appear multiple times in the resulting array.

Examples:

TakeSample([1,2,3,4,5,6,7,8,9,10], 4)
Returns [1, 9, 3, 4] (note: results may change)

TakeSample([1,2,3,4,5,6,7,8,9,10], 5, 9, true)
Returns [5, 5, 1, 5, 3]

TakeSample([1,2,3,4,5], 10)
Returns [1,2,3,4,5]

TakeSample(
  ToDataFrame(
    [[1,2],[3,4],[5,6]], ["a", "b"]
  ),
  2
).ToCsv()
Returns
a;b
5;6
3;4
Eval Expression (String)

Evaluates the given expression. The Eval function can be used e.g., when the expression to be evaluated is generated during runtime, or is fetched during the script run from an external source. The expression is evaluated in the context where the Eval function exists, so it's possible to use variables defined outside the evaluated expression (see the example).

Examples:

Eval("1 + 1");
Result: 2

let myVar = 3;
Eval("myVar + 2");
Result: 5
Sleep Duration (Timespan or integer as milliseconds)

Suspends the expression execution for the given time duration. The parameter can either be a Timespan object or integer where the duration is specified as milliseconds.

Examples:

Sleep(Timespan(0, 0, 1)));
Result: Sleeps for 1 minute

Sleep(1000);
Sleep for 1 second
ToSqlExpression Expression (String)

Converts given string into SqlExpression. Note that the SqlExpression is not yet executed at this point.

Examples:

let expr = ToSqlExpression("Column(\"Cost\")");
Query Query configuration (Object)

Runs expression query (Web API: Expression/query) and returns results as an in-memory DataFrame (for in-memory queries) or SqlDataFrame (for queries run in the datasource, e.g., in Snowflake). You can use charts to help with creating the configuration: Configure a chart, and click the Query button in the chart settings (Advanced tab) to get the query configuration. Note that the json needs to be converted into expression language by adding hash characters (#) in the beginning of the curly braces of objects.

Example: In-memory query:

let result = Query(#{
	"ModelId": 123,
	"Root": "Cases",
	"MaximumRowCount": 10,
	"Dimensions": [
		#{ "Name": "Account manager", "Expression": "Attribute(\"Account Manager\")" }
	],
	"Values": [
		#{ "Name": "Case count", "Expression": "Count(_)" }
	],
	"Ordering": [
		#{ "Name": "Case count", "Direction": "Descending" }
	]
});

Example: SqlDataFrame query ("Where" function is used additionally):

let result = Query(#{
	"ProcessingMethod": "dataframe",
	"ContextType": "model",
	"ModelId": 123,
	"Root": "Cases",
	"MaximumRowCount": 10,
	"Dimensions": [
		#{ "Name": "Account manager", "Expression": "Column(\"Account Manager\")" }
	],
	"Values": [
		#{ "Name": "Case count", "AggregationFunction": "count" }
	],
	"Ordering": [
		#{ "Name": "Case count", "Direction": "Descending" }
	]
})
.Where(Like(Column("Account manager"), "Patricia%"))
.Collect();

Ordering functions

Function Parameters Description
OrderBy (array)
  1. Array to order
  2. Order expression

Orders the given array using values from the given order expression. The order expression is evaluated once for each item in the array. The order expression supports all atomic (=not collection) primitive value types.

Examples:

OrderBy(["a", "x", "b", "z", "n", "l", "a"], _)
Return:
["a", "a", "b", "l", "n", "x", "z"]

OrderBy([9,8,7,6,5,4,3,2,1], _%3)
Returns:
[9,6,3,7,4,1,8,5,2]

OrderBy([9,8,7,6,5,4,3,2,1], _%3 + _/30)
Returns:
[3,6,9,1,4,7,2,5,8]
OrderByDescending (array)
  1. Array to order
  2. Order expression
Result is same as in the OrderBy function, except the order is reverse.
OrderByTop (array)
  1. Array to order
  2. Order expression

Orders given top-level array by the value of given expression using ascending order. Supports all value types, including multi-dimensional arrays.

Order expression is evaluated in the context of each array item whose value determines the order of that item.

Examples:

OrderByTop([[1, 2, 3], [2, 2, 2], [3, 2, 1]], _[2])
Returns: [[3, 2, 1], [2, 2, 2], [1, 2, 3]]
OrderByValue (array)
  1. Array to order

Orders the given array using the natural order of items, for example numbers it's the increasing order, and for strings it's text ordering.

OrderByValue(["a", "x", "b", "z", "n", "l", "a"])
Return:
["a", "a", "b", "l", "n", "x", "z"]

OrderByValueDescending (array)
  1. Array to order

Result is same as in the OrderByValue function, except the order is reverse.

Looping functions

Function Parameters Description
For
  1. Iterated variable name (String)
  2. Initial value for iterated property (object)
  3. Iteration condition expression
  4. Next iteration step expression
  5. Expression to calculate iterated items

Iterates the given expression until the given condition is met, and returns the results of the iterated expressions for every iteration as an array.

Examples:

For("i", 0, i < 4, i + 1, i)
Returns: [0, 1, 2, 3]

For("x", 0, x < 3, x + 1, For("y", 0, y < 3, y + 1, StringJoin(",", [x, y]))
Returns: [["0,0", "0,1", "0,2"], ["1,0", "1,1", "1,2"], ["2,0", "2,1", "2,2"]]

For("i", 0, i < 4, i + 1, DateTime(2010 + i))
Returns: [DateTime(2010), DateTime(2011), DateTime(2012), DateTime(2013)]

For("str", "a", str != "aaaaa", str + "a", str)
Returns: ["a", "aa", "aaa", "aaaa"]
ForEach (2 parameters)
  1. Array to iterate through
  2. Function to call

Calls given function using all the elements of given array, where the calculation is done in the current context. The function must support calling with one parameter. Returns the results of the evaluation of given function for every iteration of the input array as an array.

Examples:

let sourceModel = First(Models.Where(Name == "SAP_OrderToCash"));
let attributes = sourceModel.EventAttributes;
let events = sourceModel.EventLog.Events;
let func = att => Attribute(att);
events.ForEach(attributes, func)
Returns an array containing arrays of all the event attribute values for all the events.

let myFunction = item => _ + item * 2;
(5).ForEach([1,2,3], myFunction)
Returns 7, 9, 11 (=5+1*2, 5+2*2, 5+3*2)
ForEach (3 parameters)
  1. Variable to repeat (String)
  2. Array to repeat
  3. Expression to calculate repeated items

Repeats the given expression as many times there are items in the given array. Item in the array is available as the given variable in the expression. Examples:

ForEach("i", [1,2,3], "Value is: " + i)
Returns:
Value is: 1
Value is: 2
Value is: 3

ForEach("item", EventLogById(1).Cases, Let("c" + item.Name, item))
Results: Creates expression variable variables like "c<casename>" for every case in the model.

ForEach("myVariable", ["a", "b", "c"], myVariable + myVariable)
Returns:
aa
bb
cc
NumberRange
  1. Start (Number)
  2. End (Number)
  3. Interval (Number)

Creates an array of numbers within the given range using the given interval. Interval parameter is optional, and by default it is one. The interval can also be negative to get a sequence of decreasing numbers (then the Start needs to be greater than the End).

Examples:

NumberRange(1, 3)
Returns: [1, 2, 3]

NumberRange(-7, -1, 2)
Returns: [-7, -5, -3, -1]

NumberRange(1, 3, 0.5)
Returns: [1, 1.5, 2, 2.5, 3]

NumberRange(1, 3, 0.8)
Returns: [1, 1.8, 2.6]

NumberRange(6, 2, -2)
Returns: [6, 4, 2]
Repeat
  1. Number of times to repeat (Integer)
  2. Expression to repeat

Repeats the defined expression the defined number of times. Examples:

Repeat(3, "Repeat me!")
Returns:
"Repeat me!"
"Repeat me!"
"Repeat me!"

Repeat(1, 5)
Returns
5
TimeRange
  1. Start (DateTime)
  2. End (DateTime)
  3. Interval (Timespan)

Generates a timestamp array starting from the start timestamp with the defined interval, and ending when the end timestamp is reached. Note that this function only creates timestamps with equal durations, so it's not possible to e.g. create timestamps for each month (to do that, you can use the loops).

Generate datetimes starting from Monday 2017-01-01 and ending to Monday 2017-12-31 including all Mondays between them:
Timerange(Datetime(2018,1,1), Datetime(2018,1,1), Timespan(7))

Recursion functions

Function Parameters Description
Recurse
  1. Expression to call recursively
  2. Stop condition expression
  3. Maximum depth (Integer)

Evaluates the given expression recursively until given condition or recursion depth is met. The function returns all the traversed objects in a single array. When the stop condition expression evaluates to false, it will stop the current recursion without including the false evaluated object into the result. Default stop condition is !IsNull(_). The default maximum depth is 1000.

Examples:

(1).Recurse(_ + 1, _ < 5)
Returns: [1, 2, 3, 4]

event.Recurse(NextInCase)
Returns: An array of all the events following given event inside the same case.

event.Recurse(NextInCase, Type.Name != "Invoice")
Returns: An array of all the events following given event inside the same case until event whose type name is "Invoice", which will itself not be included into the result.

event.Recurse(NextInCase, Type.Name != "Invoice", 2)
Returns: An array of all the events following given event inside the same case until event whose type name is "Invoice" or until the recursion depth of 2 has been reached, which will itself not be included into the result.
RecurseWithHierarchy
  1. Expression to call recursively
  2. Stop condition expression
  3. Maximum depth (Integer)

Evaluates the given expression recursively until given condition or recursion depth is met. The function returns the traversed object hierarchy. When the stop condition expression evaluates to false, it will stop the current recursion without including the false evaluated object into the result. Default stop condition is !IsNull(_). The default maximum depth is 1000.

Examples:

[1,2].RecurseWithHierarchy([1,2], false, 2)
Returns: [1:[1:[1,2],2:[1,2]],2:[1:[1,2], 2:[1,2]]]

(1).RecurseWithHierarchy(_ + 1, _ < 5)
Returns: 1:[2:[3:[4]]]

RemoveLeaves(eventLog.Flows:From.Where(IsNull(_))).To.RecurseWithHierarchy(OutgoingFlows.To, !IsNull(_), 2)
Returns: A hierarchy consisting of all the starter events of given event log and recursively all the event types reachable from them via flows until depth of 2 is reached.
RecursiveFind
  1. Expression to call recursively
  2. Find condition expression
  3. Stop condition expression
  4. Continue after finding (Boolean)
  5. Maximum depth (Integer)

Evaluates given expression recursively until given condition or recursion depth is met. The function collects all the traversed objects that match the given find expression along the way. When the find condition expression evaluates to true for the current object, it causes the following:

  • Current object is added to the result array returned by the function call
  • If continue after finding is false, the recursion will not be continued on this branch

When the stop condition expression evaluates to false, it will stop the current recursion without including the false evaluated object into the result. Default stop condition is !IsNull(_). The default maximum depth is 1000.

Continue after finding tells should the recursion be continued after a match has been found in the current branch.

Examples:

(1).RecursiveFind(_ + 1, _ == 100)
Returns: 100

eventLog.Cases:GetAt(0, Events).RecursiveFind(NextInCase, Organization=="Finance", !IsNull(_))
Returns: For each case, returns the first (by time) event whose organization equals to "Finance".

eventLog.Cases:Events.Where(Organization=="Finance")
eventLog.Cases:GetAt(0, Events).RecursiveFind(NextInCase, Organization=="Finance", true, true)
Returns: Both return for each case all events whose organization equals to "Finance".

Hierachical object functions

Function Parameters Description
FindRepeats
  1. Array
  2. Minimum repeated length (Integer)
  3. Longest repeat length (Integer)

Searches for repeating patterns in given array. Parameters:

  1. Array to search the patterns in.
  2. Minimum integer length of a repeat that can be returned by this function (default = 0).
  3. Boolean value indicating whether to return all the repeating patterns (true) or only the ones having the longest length (false) (default = false)

Returns an array of arrays indicating the repeating patterns and their positions in the array formatted as follows:

  • Every repeating pattern has its own entry in the top-level array.
  • Every repeating pattern item in the top-level array is formatted internally as an array consisting of:
    • An array constining the repeating pattern itself.
  • An array consisting of all the indexes from which this pattern was found to start from.
  • Allows overlapping patterns.
  • Results are sorted primarily by the length of the pattern and secondarily by the number of indexes the pattern was found in.

Examples:

FindRepeats([0, 1, 2, 0, 1, 2])
Returns: [
  [[0, 1, 2], [0, 3]],
  [[0, 1], [0, 3]],
  [[1, 2], [1, 4]],
  [[0], [0, 3]],
  [[1], [1, 4]],
  [[2], [2, 5]]
]

FindRepeats([0, 1, 2, 0, 1, 2], 0, true)
Returns: [[[0, 1, 2], [0, 3]]]

FindRepeats([1, "b", DateTime(2017), 1, "b", DateTime(2017)], 3)
Returns: [[1, "b", DateTime(2017)], [0, 3]]
Flatten
  1. Array or hierarchical object
  2. Include context object (boolean)

Collects all the actual leaf values from given array, array of arrays or hierarchical object and returns them in a single array. If given a hierarchical object, this function collects actual leaf values instead of leaf level values. Elements in the returned array are in the same order they were found when traversing the input object using depth first search.

When context object is included, should context objects in the internal nodes of a hierarchical object be also included into the result. Default is false.

Examples:

Flatten(1)
Returns: 1

Flatten([1, 2])
Returns: [1,2]

Flatten([[[1, 2],[3, 4]],[[5, 6]]])
Returns: [1, 2, 3, 4, 5, 6]

Flatten([[1, 2], 3])
Returns: [1, 2, 3]

Flatten([[1,2,3,4], null, [5], [1, null]])
Returns: [1, 2, 3, 4, null, 5, 1, null]

Flatten(["a":1, "b":2])
Flatten(["a":1, "b":2], false)
Returns: [1, 2]

Flatten(["a":1, "b":2], true)
Returns: ["a", 1, "b", 2]
RemoveLeaves
  1. Hierarchical object

Remove one level of hierarchy from a hierarchical object replacing all the bottom level hierarchical arrays with the context objects of the hierarchical arrays. Returns the given object with all the bottom level hierarchical arrays with the context objects of the hierarchical arrays.

Examples:

RemoveLeaves(["foo":1, "bar":2])
Returns: ["foo", "bar"]

RemoveLeaves(eventLog.Flows:From.Where(IsNull(_)))
Results: All the flows starting the cases in the event log.
ReplaceLeafValues
  1. Array or hierarchical object
  2. Variable name used in the iteration (String)
  3. Expression to get the result of each iteration
  4. Number of levels up from the leaf level to operate (Integer)

Replace all leaf values of given array or hierarchical object at given levels up from the leaf level with results of given expression.

Examples:

ReplaceLeafValues([1,2, null], "x", If(IsNull(x), null, x+1), 0)
Result: [2, 4, null]

ReplaceLeafValues([[[1,2],[2,3]],[[3,4],[4,5]]], "x", Flatten(x), 0)
Result: [[[[1],[2]],[[2],[3]]],[[[3],[4]],[[4],[5]]]]

ReplaceLeafValues([[[1,2],[2,3]],[[3,4],[4,5]]], "x", Flatten(x), 1)
Result: [[[1,2],[2,3]],[[3,4],[4,5]]]

ReplaceLeafValues([[[1,2],[2,3]],[[3,4],[4,5]]], "x", Flatten(x), 2)
Result: [[1,2,2,3],[3,4,4,5]]

ReplaceLeafValues([[[1,2],[2,3]],[[3,4],[4,5]]], "x", Flatten(x), 3)
Result: [1,2,2,3,3,4,4,5]
SliceMiddle
  1. Start index for the range to extract (Integer)
  2. End index for the range to extract (Integer)
  3. Levels up in the hierarchy (Integer)
  4. Array to slice

Extracts a continuous range of an array or hierarchical object. If given a hierarchical object, applies the function at the level that is specified levels level up from the leaf level.

Start index: Negative value means that the index is counting from the end of the array. If array does not have element at this given index, empty array is returned.

End index: not included into the extracted range. Negative value means that the index is counting from the end of the array. If array does not have element at this given index, all the elements to the end from the start index will be extracted.

Levels up: At which level of the hierarchical object are we operating (number of levels up from the leaf level). Should be at least 1 (since 0 level does not contain arrays).

Examples:

SliceMiddle(1, 2, 1, [[0, 1, 2, 3], [4, 5, 6, 7]])
Returns: [[1], [5]]

SliceMiddle(2, 4, 1, [[0, 1, 2, 3], [4, 5, 6, 7]])
Returns: [[2, 3], [6, 7]]

SliceMiddle(0, 1, 2, [[0, 1, 2, 3], [4, 5, 6, 7]])
Returns: [0, 1, 2, 3]

SliceMiddle(3, 5, 1, [0, 1, 2, 3, 4, 5, 6, 7])
Returns: [3, 4]

SliceMiddle(-3, -1, 1, [0, 1, 2, 3, 4, 5, 6, 7])
Returns: [5, 6]

SliceMiddle(3, -1, 1, [0, 1, 2, 3, 4, 5, 6, 7])
Returns: [3, 4, 5, 6]

Snowflake connection functions

Function Parameters Description
CallStoredProcedure
  1. Procedure name (string)
  2. Procedure parameters (dictionary)
  3. Additional parameters (dictionary)

Calls a Snowflake stored procedure (it needs to pre-exist in the Snowflake when this function is called). The function returns the return value of the stored procedure. If the result is a table, its contents will be returned as an in-memory DataFrame. Object results are returned as JSON strings.

Parameters:

  1. Procedure name: Name of the called stored procedure.
  2. Procedure parameters: Dictionary of parameters passed to the stored procedure. Note: If the parameter name starts with an underscore (_), it's not be passed to the procedure call.
  3. Additional parameters: Dictionary of additional parameters affecting how the stored procedure is called. Supports parameter HasTabularResult (default false) which need to be true when the stored procedure is expected to return a tabular result, and false when a scalar result. Note: When using the HasTabularResult=true, Snowflake ODBC driver version needs to be at least 3.1.0 (due to bug fix in the driver).

Security note: All QPR ProcessAnalyzer users can use the CallStoredProcedure function and call Snowflake stored procedures that are in the schema configured to the Snowflake connection string. Create only stored procedures to that schema, that are safe for any user to run.

Example 1: Stored procedure to calculate a sum of two numbers. Create following stored procedure to Snowflake:

CREATE OR REPLACE PROCEDURE SUMOFNUMBERS(NUMBER1 NUMBER, NUMBER2 NUMBER)
RETURNS NUMBER
LANGUAGE SQL
EXECUTE AS OWNER
AS '
BEGIN
  RETURN NUMBER1 + NUMBER2;
END';

The stored procedure can be called as follows:

CreateSnowflakeConnection()
  .CallStoredProcedure(
    "SUMOFNUMBERS",
    #{
      "NUMBER1": 3,
      "NUMBER2": 5
    }
  );

Returns: 8

Example 2: Calls a stored procedure named train_ml_model with a dictionary parameter containing InputEvents key that has SQL query string as value which, when run in the same Snowflake connection, will return all the contents of data table. ColumnMappings key that has a dict-object containing column role mappings of the InputEvents data.

let dt = ModelById(1).EventsDataTable;
CreateSnowflakeConnection()
  .CallStoredProcedure("train_ml_model", #{
    "ColumnMappings": dt.ColumnMappings,
    "InputEvents": dt.SqlDataFrame
  });
CreateSnowflakeConnection Parameters dictionary

Creates an object representing connection to Snowflake. Takes as a parameter a dictionary with following settings names:

  • OdbcConnectionString: Connection to Snowflake is made with this connection string.
  • OdbcConnectionStringKey: Secret name containing the ODBC connection string to connect to Snowflake.
  • ProjectId: Project id to use as the context of the secure string. If not defined, project id is queried from the current context object. Project id needs to be available when using the OdbcConnectionStringKey setting.

If neither OdbcConnectionString or OdbcConnectionStringKey is defined, the system configured connection string will be used.

Examples:

ProjectByName("My project")
  .CreateSnowflakeConnection(
    #{ "OdbcConnectionStringKey": "MyConnectionStringKey" }
  );
ToSqlDataFrame In-memory dataframe Same functionality as in the Model.ToSqlDataFrame, except the SQL dataframe is created for the connection which the function is called for.

Example: Create an SQL dataframe that is passed as parameter to a Snowflake procedure:

let df = ToDataFrame([[0, "zero"], [2, "two"], [3, "three"]], ["id", "right"]);
let connection = project.CreateSnowflakeConnection();
connection.CallStoredProcedure("MyProcedure", #{"input": connection.ToSqlDataFrame(df)});