PA Configuration database table: Difference between revisions

From QPR ProcessAnalyzer Wiki
Jump to navigation Jump to search
 
(35 intermediate revisions by the same user not shown)
Line 1: Line 1:
QPR ProcessAnalyzer database has a configuration table '''PA_Configuration''' containing settings listed in the tables below. You need '''SQL Server Management Studio''' to edit the settings in the configuration table. QPR ProcessAnalyzer Server needs to be restarted (e.g. IIS application pool recycled) for the changes to take effect.
QPR ProcessAnalyzer database has a configuration table '''PA_Configuration''' containing settings listed in the tables below. You need '''SQL Server Management Studio''' to edit the settings in the configuration table. QPR ProcessAnalyzer Server needs to be restarted (e.g. IIS application pool recycled) for the changes to take effect.
For boolean values, ''true'' and ''1'' are valid values for yes, and ''false'' and ''0'' are valid for no.


== General Settings ==
== General Settings ==
Line 40: Line 42:
||Defines whether the 12-hour clock is used by default (instead of the 24-hour clock) for the new user accounts when showing time information in the UI. Defined as '''true''' or '''false'''. More information about the 12-hour clock: https://en.wikipedia.org/wiki/12-hour_clock.
||Defines whether the 12-hour clock is used by default (instead of the 24-hour clock) for the new user accounts when showing time information in the UI. Defined as '''true''' or '''false'''. More information about the 12-hour clock: https://en.wikipedia.org/wiki/12-hour_clock.
|-
|-
||<span style="color:green;font-weight:bold;">DefaultDatasource</span>
||<span style="color:green;font-weight:bold;" id="DefaultDataSource">DefaultDataSource</span>
||
||
||Datasource where datatables data is stored when datatable is created by a script when the datasource is not explicitly specified in the script. Possible values are either '''snowflake''' or '''sqlserver'''. Value ''snowflake'' can be used when the ''SnowflakeConnectionString'' setting is defined and value sqlserver'' can be used when the setting ''SqlServerConnectionString'' is configured. This setting can be changed without affecting the existing datatables, as the setting only affect new datatables. When this setting is empty, datatables are created in the metadata database.
||Datasource where datatables data is stored when datatables are created by a script when the datasource is not explicitly specified in the script. Possible values are '''Snowflake''' and '''SqlServer'''. Value ''snowflake'' can be used when the ''SnowflakeConnectionString'' setting is defined, and value ''sqlserver'' can be used when the setting ''SqlServerConnectionString'' is configured. The setting can be changed without affecting the existing datatables, as the setting only affect new datatables. When this setting is empty, datatables are created in the metadata database.
|-
|-
||<span style="color:green;font-weight:bold;">SqlServerConnectionString</span>
||<span style="color:green;font-weight:bold;" id="SqlServerConnectionString">SqlServerConnectionString</span>
||
||
||ADO.Net connection string for the SQL Server database containing the datatables data. It's recommended to use separate database, but it's also possible to connect to the same database as the metadata (configuration data). When this setting has been configured, new datatables are created to this database instead of the metadata database. The old datatables are still located in the metadata database, and new datatables cannot be created to the metadata database anymore. Note that the connection uses ADO.Net (not ODBC), so the connection string is similar to the metadata database connection string in the [[Installing_QPR_ProcessAnalyzer_Server#Server_settings_file_(appsettings.json)|appsettings.json]] file.
||Connection string for the SQL Server database containing the datatables data. It's recommended to use a separate database, but it's also possible to connect to the same database as the configuration data. If this setting is not configured, local datatables cannot be created (SQL Server stored). Existing datatables located in the configuration datatabase still work even if this setting has not be configured. Note that the connection uses ADO.Net (not ODBC), so the connection string is similar to the configuration database ([[Installing_QPR_ProcessAnalyzer_Server#Server_settings_file_(appsettings.json)|appsettings.json]] file).
|-
|-
||<span style="color:green;font-weight:bold;">SnowflakeConnectionString</span>
||<span style="color:green;font-weight:bold;" id="SnowflakeConnectionString">SnowflakeConnectionString</span>
||
||
||ODBC connection string for the Snowflake connection, that needs to be configured to process models in the Snowflake. The connection string has following format:
||ODBC connection string for the Snowflake account. This setting is needed to make analytics calculations in the Snowflake. More information how to configure the [[Snowflake_Connection_Configuration#Set_Snowflake_ODBC_connection|Snowflake connection string]]. The Snowflake ODBC driver also needs to be installed in the machine running the QPR ProcessAnalyzer Server. When this setting has been configured, users can create Snowflake stored datatables and models using Snowflake calculation.
 
When running QPR ProcessAnalyzer in the Snowpark Container Services, leave the SnowflakeConnectionString empty because then the connection string is created automatically using the method provided by the Snowpark Container Services.
|-
||<span id="OpenAIAPIKey" style="color:green;font-weight:bold;">OpenAIAPIKey</span>
||
||API key for the OpenAI API (https://platform.openai.com/docs/api-reference) which needs to be configured to use the [[Generic_Functions_in_QPR_ProcessAnalyzer#OpenAIChatCompletion|OpenAIChatCompletion]] function and other generative AI functionality.
|-
||<span id="OpenAIDefaultModelName" style="color:green;font-weight:bold;">OpenAIDefaultModelName</span>
||gpt-4-turbo
||OpenAI large language model (LLM) to use for the [[AI_Assistant_for_QPR_ProcessAnalyzer|AI Assistant]] and [[Generic_Functions_in_QPR_ProcessAnalyzer#OpenAIChatCompletion|OpenAIChatCompletion]] function. If not defined, '''gpt-4-turbo''' model will be used in QPR ProcessAnalyzer 2024.4, and starting from QPR ProcessAnalyzer 2024.5, '''gpt-4o''' will be used. If desired to use a less expensive model, '''gpt-3.5-turbo''' can be used. Only models that support the function calling feature, are suitable for the AI Assistant. More information about OpenAI models: https://platform.openai.com/docs/models/.
<pre>
<pre>
Driver={SnowflakeDSIIDriver};Application=QPR_ProcessAnalyzer;Server=<account_identifier>.snowflakecomputing.com;Database=QPRPA;Schema=QPRPA;Warehouse=QPRPA;Role=QPRPA;uid=QPRPA;pwd=<password>
Recommendation is to use the gpt-4o model which has lower price and better performance than gpt-4-turbo.
</pre>
</pre>
where <password> is the Snowflake user password and <account_identifier> is the Snowflake account identifier. In the example, the database, schema, warehouse, role and user are named to QPRPA. In addition to this setting, the Snowflake ODBC driver needs to be installed in the machine running the QPR ProcessAnalyzer Server. When this setting has been configured, users can create Snowflake stored datatables and models using Snowflake calculation. More information about [[Snowflake_Connection_Configuration|Snowflake connection configuration]].
|-
||AllowExternalDatasources
||true
||Can be used to disallow all connections to external datasources in the expression language and SQL scripts to improve security. Disallowed operations include ODBC, OLE DB, SQL Server (Ado.Net), SAP, Salesforce, and call web service. Note that this setting does not prevent the Snowflake processing. Regardless of this setting, QPR ScriptLauncher can be used to extract data from source systems.
|-
|-
|DatabaseId
|DatabaseId
Line 79: Line 94:
||Timeout (in seconds) for requests made to /api/expression/query and /api/expression endpoints. When the timeout is exceeded, the query is stopped and a timeout error is returned. Purpose of the timeout is to protect the system against potentially too long running or even never-ending queries which might otherwise jam the system.
||Timeout (in seconds) for requests made to /api/expression/query and /api/expression endpoints. When the timeout is exceeded, the query is stopped and a timeout error is returned. Purpose of the timeout is to protect the system against potentially too long running or even never-ending queries which might otherwise jam the system.
|-
|-
||AllowExternalDatasources
||<span id="CacheOnlyPrimaryKeysForFilters">CacheOnlyPrimaryKeysForFilters</span>
||True
||false
||Determines whether the [[Generic_Functions_in_QPR_ProcessAnalyzer#Other_functions|ImportOdbc]] and CallWebService functions in the expression language and the [[QPR_ProcessAnalyzer_Model_Datasources#Loading_Data_from_ODBC_Datasource|ODBC model datasource]] can be used. This setting is for disabling external interfaces for security reasons. By default, external interfaces are allowed.
||Defines whether to include all columns in the Snowflake event cache filter tables (''false''), or only the primary key columns (''true''). When ''false'', cache table creation is slower, but the analysis calculation is faster because the original event table is not used anymore. When ''false'', also the cache tables require more storage space in Snowflake.
|-
|-
||AllowNonTemporaryETLTargetTable
||AllowNonTemporaryETLTargetTable
||False
||false
||Defined whether ETL scripts are allowed to create global temporary database tables (tables starting with ##). More information about temporary tables: https://docs.microsoft.com/en-us/sql/t-sql/statements/create-table-transact-sql?view=sql-server-ver15#temporary-tables.
||Defined whether ETL scripts are allowed to create global temporary database tables (tables starting with ##). More information about temporary tables: https://docs.microsoft.com/en-us/sql/t-sql/statements/create-table-transact-sql?view=sql-server-ver15#temporary-tables.
|-
|-

Latest revision as of 20:36, 26 October 2024

QPR ProcessAnalyzer database has a configuration table PA_Configuration containing settings listed in the tables below. You need SQL Server Management Studio to edit the settings in the configuration table. QPR ProcessAnalyzer Server needs to be restarted (e.g. IIS application pool recycled) for the changes to take effect.

For boolean values, true and 1 are valid values for yes, and false and 0 are valid for no.

General Settings

Name Default value Description
SessionIdleTimeout 3600 Idle user session expiration timeout in seconds. User session expires if the session hasn't been used after this amount of time.
SessionMaximumDuration 86400 Maximum duration for a user session in seconds. Even if a session is used so that the SessionIdleTimeout is not reached, the session is expired after this amount of time.
SandboxDatabaseConnectionString Connection string to scripting sandbox database (ETL). If not defined, SQL-based ETL scripts cannot be run. Connection string for the scripting sandbox database is similar to the QPR ProcessAnalyzer database connection string. More information: Setting up Scripting Sandbox.
DefaultUiLanguage en_US Language code for the UI language that new user accounts get by default. Thus, a created user account has this language until the user changes her/his language. Also the login page is translated using this language when QPR ProcessAnalyzer is used for the first time in that web browser (when user has changed the language, it's remembered by the browser). This setting must be one of the supported language codes (xx_XX):
  • Finnish: fi_FI
  • French: fr_FR
  • English: en_US
  • German: de_DE
  • Spanish: es_ES
  • Swedish: sv_SE
  • Polish: pl_PL
DefaultDateFormat MM/dd/yyyy Default date format that new user accounts get by default. The date format does not contain the time part (e.g. hours, minutes and seconds). Defined using the .Net date format (https://docs.microsoft.com/en-us/dotnet/standard/base-types/custom-date-and-time-format-strings).
DefaultFirstDayOfWeek 0 Default first day of the week that new user accounts get by default. 0 is Sunday and 1 is Monday. This information is used by the UI when showing e.g. calendars.
DefaultUse12HourClock false Defines whether the 12-hour clock is used by default (instead of the 24-hour clock) for the new user accounts when showing time information in the UI. Defined as true or false. More information about the 12-hour clock: https://en.wikipedia.org/wiki/12-hour_clock.
DefaultDataSource Datasource where datatables data is stored when datatables are created by a script when the datasource is not explicitly specified in the script. Possible values are Snowflake and SqlServer. Value snowflake can be used when the SnowflakeConnectionString setting is defined, and value sqlserver can be used when the setting SqlServerConnectionString is configured. The setting can be changed without affecting the existing datatables, as the setting only affect new datatables. When this setting is empty, datatables are created in the metadata database.
SqlServerConnectionString Connection string for the SQL Server database containing the datatables data. It's recommended to use a separate database, but it's also possible to connect to the same database as the configuration data. If this setting is not configured, local datatables cannot be created (SQL Server stored). Existing datatables located in the configuration datatabase still work even if this setting has not be configured. Note that the connection uses ADO.Net (not ODBC), so the connection string is similar to the configuration database (appsettings.json file).
SnowflakeConnectionString ODBC connection string for the Snowflake account. This setting is needed to make analytics calculations in the Snowflake. More information how to configure the Snowflake connection string. The Snowflake ODBC driver also needs to be installed in the machine running the QPR ProcessAnalyzer Server. When this setting has been configured, users can create Snowflake stored datatables and models using Snowflake calculation.

When running QPR ProcessAnalyzer in the Snowpark Container Services, leave the SnowflakeConnectionString empty because then the connection string is created automatically using the method provided by the Snowpark Container Services.

OpenAIAPIKey API key for the OpenAI API (https://platform.openai.com/docs/api-reference) which needs to be configured to use the OpenAIChatCompletion function and other generative AI functionality.
OpenAIDefaultModelName gpt-4-turbo OpenAI large language model (LLM) to use for the AI Assistant and OpenAIChatCompletion function. If not defined, gpt-4-turbo model will be used in QPR ProcessAnalyzer 2024.4, and starting from QPR ProcessAnalyzer 2024.5, gpt-4o will be used. If desired to use a less expensive model, gpt-3.5-turbo can be used. Only models that support the function calling feature, are suitable for the AI Assistant. More information about OpenAI models: https://platform.openai.com/docs/models/.
Recommendation is to use the gpt-4o model which has lower price and better performance than gpt-4-turbo.
AllowExternalDatasources true Can be used to disallow all connections to external datasources in the expression language and SQL scripts to improve security. Disallowed operations include ODBC, OLE DB, SQL Server (Ado.Net), SAP, Salesforce, and call web service. Note that this setting does not prevent the Snowflake processing. Regardless of this setting, QPR ScriptLauncher can be used to extract data from source systems.
DatabaseId Unique identifier for the QPR ProcessAnalyzer environment. Any characters between a-z, A-Z, 0-9 and _ (underscore) can be used in the DatabaseId. If the DatabaseId is missing or set to null, the system will generate a new GUID during startup and use it as the DatabaseId. The DatabaseId can also be an empty string. If using several QPR ProcessAnalyzer environments, make sure each use a different DatabaseId. The DatabaseId is used as part of the table names in Snowflake and SQL Server (in the datatables database). Thus if the DatabaseId is changed, all tables in Snowflake and SQL Server named with qprpa_dt_<DatabaseId>_<DatatableId> need to be renamed.
NumberOfParallelModelReaders 4 Models and datatable contents can be loaded with multiple simultaneous connections to the database to speed up the loading. This setting determines how many parallel loaders/readers at maximum (loaders are loading at the same time). For smaller models there are less parallel loaders than the defined limit: If there are less than 100000 rows in the table, there is only one loader. If there are less than 200000 rows in the table, there are only two loaders, and so on.

The more there are parallel loaders, the more processor load and network bandwidth is consumed, and other operations in QPR ProcessAnalyzer might slow down. Note also that the performance optimum is achieved with a certain number of parallel loaders which differs between environment. Thus to achieve the best performance, data loading should be tested with different number of parallel loaders. Increasing number of parallel loaders beyond the optimum decreases the performance.

StartupModelLoadingMaxParallelism 2 Maximum number of QPR ProcessAnalyzer models that are loaded into memory simultaneously by the Automatic Loading on Server Startup. If there are more models to be loaded on the server startup than this setting, loading for the rest of the models is started one by one when previous model loadings are completed. If this setting is not defined, 2 is used as a default value.

Loading more models at the same time will speed up the whole model loading process, but on the other hand, it causes more load on the system, which affects the system responsiveness for users. Model loading consists of (1) transferring data from the datasource to QPR ProcessAnalyzer and (2) loaded data preprocessing into a model. The former uses mainly network bandwidth (if datasource is in a different server) and the latter uses mainly processor capacity in the QPR ProcessAnalyzer server.

This setting affects only the model loading during the server startup and it doesn't restrict models loadings initiated by users.

QueryTimeout 300 Timeout (in seconds) for requests made to /api/expression/query and /api/expression endpoints. When the timeout is exceeded, the query is stopped and a timeout error is returned. Purpose of the timeout is to protect the system against potentially too long running or even never-ending queries which might otherwise jam the system.
CacheOnlyPrimaryKeysForFilters false Defines whether to include all columns in the Snowflake event cache filter tables (false), or only the primary key columns (true). When false, cache table creation is slower, but the analysis calculation is faster because the original event table is not used anymore. When false, also the cache tables require more storage space in Snowflake.
AllowNonTemporaryETLTargetTable false Defined whether ETL scripts are allowed to create global temporary database tables (tables starting with ##). More information about temporary tables: https://docs.microsoft.com/en-us/sql/t-sql/statements/create-table-transact-sql?view=sql-server-ver15#temporary-tables.
DatabaseBulkCopyTimeout 600 BulkCopyTimeout given for QPR ProcessAnalyzer database SqlBulkCopy operations.
DatabaseBulkCopyBatchSize 5000 BulkCopyBatchSize given for QPR ProcessAnalyzer database SqlBulkCopy operations.
SandboxDatabaseBulkCopyTimeout 600 BulkCopyTimeout given for sandbox SqlBulkCopy operations.
SandboxDatabaseBulkCopyBatchSize 5000 BulkCopyBatchSize given for sandbox SqlBulkCopy operations.

SAML 2.0 Federated Authentication Settings

Note that the SAMLMetadataUrl and ServiceProviderLocation are mandatory for the federated authentication to work.

Name Description
SAMLMetadataUrl

Metadata URL of the identity provider (IdP). Check that the metadata url can actually be opened using a web browser and is publicly available. The metadata is an XML document starting with <?xml version="1.0" encoding="UTF-8"?> followed by an EntityDescriptor tag. The metadata URL might look https://your.federated.identity.provider.com/saml/metadata. This setting is mandatory for the SAML authentication to work.

ServiceProviderLocation

Specifies the QPR ProcessAnalyzer server location (the root path which contains e.g. the ui folder). It's used by the url to redirect back to QPR ProcessAnalyzer after a successful authentication from the identity provider. The setting is defined in the following form: https://<hostname>/qprpa, for example https://customer.onqpr.com/qprpa. Note that the actual redirect back url is https://<hostname>/qprpa/api/saml2/acs (/api/saml2/acs is automatically included to the url). This setting is mandatory for the SAML authentication to work. Note that if this reply url is configured the identity provider, it must match with the ServiceProviderLocation setting.

SAMLUserIdAttribute

Name of the SAML attribute in the assertion that will be used as the user's login name. If this field is not defined, the saml:Assertion > saml:Subject > saml:NameID attribute in the assertion is used. If this setting is given, one of the saml:Assertion > saml:AttributeStatement > saml:Attribute elements in the assertion is used (the Name attribute in the saml:Attribute element is used for matching). Please note that the saml:NameID element is different than the usual SAML attributes that are defined by the saml:Attribute elements. For example, if an email address is used as a user id, the value of the setting could be for example http://schemas.xmlsoap.org/ws/2005/05/identity/claims/emailaddress.

SAMLEncryptionCertificate This setting defines a PEM formatted X.509 certificate (defined in RCF 1422) used to encrypt SAML assertions. The public key of the certificate is published in the service provider metadata, where the identity provider can read it and encrypt SAML assertions. QPR ProcessAnalyzer as the service provider uses the corresponding private key of the certificate to decrypt SAML assertions. The setting needs to be a PEM formatted certificate file that is base64 encoded and it doesn't contain the BEGIN CERTIFICATE etc. header or footer lines. This setting is needed only when using the SAML assertions encryption. Even though this setting is defined, the SAML assertions are not required to be encrypted. More information how to create the certificate file (https://stackoverflow.com/questions/16480846/x-509-private-public-key) and convert it to base64 (https://stackoverflow.com/questions/46959822/base-64-encoded-form-of-the-pfx-file).
SAMLSigningCertificate This setting defines a PEM formatted X.509 certificate (defined in RCF 1422) used to sign SAML authentication requests sent from QPR ProcessAnalyzer to the identity provider. The public key of the certificate is published in the service provider metadata, where the identity provider can read it, to verify the authenticity of the SAML requests. The setting needs to be a PEM formatted certificate file that is base64 encoded and it doesn't contain the BEGIN CERTIFICATE etc. header or footer lines. If this setting is not defined, the internal hard-coded signing certificate is used. More information how to create the certificate file (https://stackoverflow.com/questions/16480846/x-509-private-public-key) and convert it to base64 (https://stackoverflow.com/questions/46959822/base-64-encoded-form-of-the-pfx-file).

SMTP Server Settings

SMTP server settings are neede for QPR ProcessAnalyzer to be able to send email messages. Email sending is used by the notifications functionality and the SendEmail function in the expression language.

Name Description
SmtpServer DNS name, host name or IP address of the SMTP server. Mandatory setting for the email sending to work.
SmtpPort TCP port number of the SMTP server. If not defined, port 25 is used by default.
SmtpAuthenticationUsername User name for authenticating to the SMTP server. If not defined, no authentication is used to connect to the SMTP server.
SmtpFromAddress Email address where email messages sent by QPR ProcessAnalyzer appear to be coming from. This doesn't need to be a real email address, although the address used may affect email spam filters. The setting configured here is the default email address to use in following cases:
  • From address is not set for the email notifications
  • From parameter is not defined for the expression language SendEmail function
  • EmailFrom parameter is not defined for the SQL Scripting SendEmail operation
SmtpAuthenticationPassword Password for authenticating to the SMTP server.
SmtpEnableSSL True or False depending whether SSL connection to the SMTP server is used or not. If not defined, False is the default value.

Readonly Information

Name Description
DatabaseVersion Database schema version. It will be updated automatically when the newer version of QPR ProcessAnalyzer Server connects to the database and performs migration for the database schema.
InitializationScriptDatabaseVersion Database version that was when the database was initialized when the software was installed. Do not change this setting.
MinimumDatabaseVersion Minimum allowed database version for QPR ProcessAnalyzer Server connecting to the database. This is a legacy setting and it should not be used.