Machine Learning Functions in Expression Language - Revision history

MarHink at 07:26, 19 November 2024

2024-11-19T07:26:49Z

← Older revision		Revision as of 07:26, 19 November 2024
Line 1:		Line 1:
	This page describes functions and properties that implement the machine learning functionality ~~for in-memory calculation~~, such as '''clustering''' and '''prediction''', that are part of the expression language. For prediction, the '''random forest''' is a supported algorithm. For clustering, the following algorithms are supported: '''KModes''', '''KMeans''' and '''BalancedKMeans'''.		This page describes functions and properties that implement the machine learning functionality, such as '''clustering''' and '''prediction''', that are part of the expression language. For prediction, the '''random forest''' is a supported algorithm. For clustering, the following algorithms are supported: '''KModes''', '''KMeans''' and '''BalancedKMeans'''.

	== Snowflake models ==		== Snowflake models ==

MarHink at 07:24, 19 November 2024

2024-11-19T07:24:28Z

← Older revision		Revision as of 07:24, 19 November 2024
Line 1:		Line 1:
	This ~~pages~~ describes functions and properties that implement the machine learning functionality, such as '''clustering''' and '''prediction''', that are part of the expression language. For prediction, the '''random forest''' is a supported algorithm. For clustering, the following algorithms are supported: '''KModes''', '''KMeans''' and '''BalancedKMeans'''.		This page describes functions and properties that implement the machine learning functionality for in-memory calculation, such as '''clustering''' and '''prediction''', that are part of the expression language. For prediction, the '''random forest''' is a supported algorithm. For clustering, the following algorithms are supported: '''KModes''', '''KMeans''' and '''BalancedKMeans'''.

			== Snowflake models ==
			For Snowflake-based models and data frames, machine learning functionalities can be found in [[Create Predicted Eventlog]] (prediction) and [[SqlDataFrame in Expression Language\|WithClusterColumn]] (clustering).

	== Clustering functions ==		== Clustering functions ==

Ollvihe: /* Clustering functions */

2022-06-14T15:49:10Z

Clustering functions

← Older revision		Revision as of 15:49, 14 June 2022
Line 86:		Line 86:
	Examples:		Examples:
	<pre>		<pre>
			Codify([["a", 4], ["c", 4], ["b", 3], ["c", 3]])
			Returns: [[0, 0], [1, 0], [2, 1], [1, 1]]

	Codify([[1,2], [3,4], [1,4]])		Codify([[1,2], [3,4], [1,4]])
	Returns: [[0, 0], [1, 1], [0, 1]]		Returns: [[0, 0], [1, 1], [0, 1]]

Ollvihe: /* Clustering functions */

2022-06-14T15:41:38Z

Clustering functions

← Older revision		Revision as of 15:41, 14 June 2022
Line 14:		Line 14:

	Parameters:		Parameters:
	# Matrix to cluster. Rows (1st dimension) represent data points and columns represent feature values (2nd dimension)		# Matrix to cluster. Rows (1st dimension) represent data points and columns represent feature values (2nd dimension).
	# Target number of clusters		# Target number of clusters.
	# Distance function to be used in the clustering process.		# Distance function to be used in the clustering process.

	Returns an array ~~having the~~ following elements:		Returns an array with following two elements:
	* ~~Element 0: An array of all the~~ cluster ~~labels for all the rows in the~~ input ~~matrix in the same order as they were given in the matrix parameter~~.		* 1. Array describing to which cluster each input data point belongs to. Clusters are defined using index starting from zero.
	* ~~Element 1: An array of length~~ 2 ~~having the~~ following elements:		* 2. Array containing following two elements:
	** ~~Element 0:~~ Computed final error of the clustering.		** 2.1. Computed final error of the clustering.
	** ~~Element 1:~~ Number of iterations performed in the clustering.		** 2.2. Number of iterations performed in the clustering.

	Examples:		Examples:

Ollvihe: /* Prediction functions */

2021-10-07T10:10:20Z

Prediction functions

← Older revision		Revision as of 10:10, 7 October 2021
Line 121:		Line 121:
	\|\|		\|\|
	* LM model type (String)		* LM model type (String)
	* ~~additional~~ parameters		* Additional parameters (key-value pairs)
	\|\|		\|\|
	Create a new machine learning model for predictions. Takes type of the ~~prediction/classification~~ model to create as a parameter. Currently the ~~only~~ supported value is '''randomforest''' which uses the Accord.NET's RandomForest algorithm.		Create a new machine learning model for making predictions. Takes type of the model to create as a parameter. Currently the supported value is '''randomforest''' which uses the Accord.NET's RandomForest algorithm. Additional parameters:
			* '''ComputeCovariance''': If true, the result will include covariance matrices. Default value is false.
	~~Parameters~~:		<pre>
	* ~~Type of the prediction/classification model to create. Only supported value is '''binarygbm''' (based on ML.NET's LightGBM)~~		let myMLModel = MLModel("randomforest", #{"ComputeCovariance": true});
	* Additional parameters as key value pairs. Only supported parameter is '''ComputeCovariance''': If true, the result will include covariance matrices. Default value is false.		</pre>
	\|-		\|-
	\|\| Train (MLModel)		\|\| Train (MLModel)

Ollvihe: /* Prediction functions */

2021-10-07T10:05:33Z

Prediction functions

← Older revision		Revision as of 10:05, 7 October 2021
Line 142:		Line 142:
	** The second dimension (columns) specifies the feature values.		** The second dimension (columns) specifies the feature values.
	* '''expected outcomes''': An array of expected outcomes for each row in the input data. Must be in the same order as the rows in the input data.		* '''expected outcomes''': An array of expected outcomes for each row in the input data. Must be in the same order as the rows in the input data.
	* '''parameters''': Additional parameters for the MLModel~~. Supported parameters~~:		* '''parameters''': Additional parameters for the MLModel:
	** ~~Number of trees~~: ~~the~~ number of trees in the random forest~~, default~~ value is 10.		** '''NumberOfTrees''': number of trees in the random forest. Default value is 10.
	** ~~Sample ratio~~: ~~the proportion~~ of samples used to train each of the trees in the decision forest~~, default~~ value is 0.632.		** '''SampleRatio''': Proportion of samples used to train each of the trees in the decision forest Default value is 0.632.
	\|-		\|-
	\|\|Transform (array)		\|\|Transform (array)

Ollvihe: /* Prediction functions */

2021-09-30T16:58:00Z

Prediction functions

← Older revision		Revision as of 16:58, 30 September 2021
Line 135:		Line 135:
	* Parameters		* Parameters
	\|\|		\|\|
	Trains given MLModel using given input data and expected outcomes.		Trains given MLModel using given input data and expected outcomes. Returns the trained MLModel object.

	Parameters:		Parameters:
Line 145:		Line 145:
	** Number of trees: the number of trees in the random forest, default value is 10.		** Number of trees: the number of trees in the random forest, default value is 10.
	** Sample ratio: the proportion of samples used to train each of the trees in the decision forest, default value is 0.632.		** Sample ratio: the proportion of samples used to train each of the trees in the decision forest, default value is 0.632.

	~~Returns the trained MLModel object.~~
	\|-		\|-
	\|\|Transform (array)		\|\|Transform (array)

Ollvihe: /* Clustering functions */

2021-09-30T16:31:06Z

Clustering functions

← Older revision		Revision as of 16:31, 30 September 2021
Line 9:		Line 9:
	\|\|KModes		\|\|KModes
	\|\|		\|\|
	* Matrix to cluster		Matrix to cluster
	\|\|		\|\|
	Performs KModes clustering for a numeric matrix. Implementation uses Accord.NET KModes method (http://accord-framework.net/docs/html/T_Accord_MachineLearning_KModes.htm).		Performs KModes clustering for a numeric matrix. Implementation uses Accord.NET KModes method (http://accord-framework.net/docs/html/T_Accord_MachineLearning_KModes.htm).
Line 95:		Line 95:
	\|\|OneHot		\|\|OneHot
	\|\|		\|\|
	* Numeric matrix		Numeric matrix
	\|\|		\|\|
	One-hot encodes all matrix columns. Implementation uses Accord.NET OneHot method (http://accord-framework.net/docs/html/M_Accord_Math_Jagged_OneHot_1.htm)		One-hot encodes all matrix columns. Implementation uses Accord.NET OneHot method (http://accord-framework.net/docs/html/M_Accord_Math_Jagged_OneHot_1.htm)

Ollvihe: /* Clustering functions */

2021-09-30T16:30:44Z

Clustering functions

← Older revision		Revision as of 16:30, 30 September 2021
Line 82:		Line 82:
	\|\|Matrix to codify		\|\|Matrix to codify
	\|\|		\|\|
	~~Use Accord's Codify -functionality to encode~~ all unique column values into unique numeric integer values. Based on: http://accord-framework.net/docs/html/T_Accord_Statistics_Filters_Codification.htm. Returns codified matrix of exactly the same dimensions as the input matrix.		Encodes all unique column values into unique numeric integer values. Based on Accord.Net codify functionality: http://accord-framework.net/docs/html/T_Accord_Statistics_Filters_Codification.htm. Returns codified matrix of exactly the same dimensions as the input matrix.

	Examples:		Examples:

Ollvihe: /* Clustering functions */

2021-09-30T16:29:10Z

Clustering functions

← Older revision		Revision as of 16:29, 30 September 2021
Line 71:		Line 71:
	\|-		\|-
	\|\|BalancedKMeans		\|\|BalancedKMeans
	\|\|~~jsonData (String)~~		\|\|
			* Matrix to cluster
			* Target number of clusters
			* Distance function
			* Additional parameters
	\|\|		\|\|
	Performs Balanced KMeans clustering for given numeric matrix. Algorithm is based on http://accord-framework.net/docs/html/T_Accord_MachineLearning_BalancedKMeans.htm. Parameters and return value structure is identical to the KMeans function.		Performs Balanced KMeans clustering for given numeric matrix. Algorithm is based on http://accord-framework.net/docs/html/T_Accord_MachineLearning_BalancedKMeans.htm. Parameters and return value structure is identical to the KMeans function.