QPR ProcessAnalyzer System Architecture - Revision history

Ollvihe: /* Bring Data to Snowflake */

2026-03-09T22:47:59Z

Bring Data to Snowflake

← Older revision		Revision as of 22:47, 9 March 2026
Line 42:		Line 42:
	== Bring Data to Snowflake ==		== Bring Data to Snowflake ==

	There are different methods to ingest data into Snowflake, e.g., using Snowflake ~~own features~~, QPR ProcessAnalyzer ETL functionality, or 3rd party tools. The following picture illustrates ~~the~~ variaty of methods:		There are different methods to ingest data into Snowflake, e.g., using features provided by Snowflake, QPR ProcessAnalyzer ETL functionality, or 3rd party tools. The following picture illustrates these variaty of methods:

	[[File:SnowflakeIntegrationOptions.png\|1200px]]		[[File:SnowflakeIntegrationOptions.png\|1200px]]

	For example, the following data import methods are available (not mentioning all):		For example, the following data import methods are available (not mentioning all):
	* Data can be pushed into AWS, Azure or GCP ~~blob storage where~~ Snowflake ~~will~~ immediately load ~~it.~~		* Data can be pushed into AWS S3, Azure Blob Storage or GCP Cloud Storage services as different types of files (such as CSV, Parquet, etc.). Snowflake immediately detects and loads the data incrementally into Snowflake tables using Snowpipe. (https://docs.snowflake.com/en/user-guide/data-load-snowpipe-intro)
	* Snowflake supports Apache Iceberg tables, data from many kind of data lakes is available in Snowflake. When using Iceberg tables, the data is not phycically copied to Snowflake, so the source-of-truth remains in the external datasource.		* Snowflake supports Apache Iceberg tables, data from many kind of data lakes is available in Snowflake. When using Iceberg tables, the data is not phycically copied to Snowflake, so the source-of-truth remains in the external datasource.
	* SAP Business Data Cloud can be integrated with Snowflake using BDC Connect for Snowflake. The integration is based on the delta sharing enabling federated real time access (no data replication) through the Snowflake Horizon Catalog. (https://docs.snowflake.com/en/user-guide/data-integration/openflow/connectors/about-sap-snowflake)		* SAP Business Data Cloud can be integrated with Snowflake using BDC Connect for Snowflake. The integration is based on the delta sharing enabling federated real time access (no data replication) through the Snowflake Horizon Catalog. (https://docs.snowflake.com/en/user-guide/data-integration/openflow/connectors/about-sap-snowflake)

Ollvihe: /* Connecting to Data Sources */

2026-03-09T19:46:25Z

Connecting to Data Sources

@@ Line 39: / Line 39: @@
 * QPR ScriptLauncher
 * Data extraction from external datasources
 == Bring Data to Snowflake ==

Ollvihe: /* Bring Data to Snowflake */

2026-03-09T17:30:08Z

Bring Data to Snowflake

← Older revision		Revision as of 17:30, 9 March 2026
Line 59:		Line 59:
	[[File:SnowflakeIntegrationOptions.png\|1200px]]		[[File:SnowflakeIntegrationOptions.png\|1200px]]

	~~The~~ following methods are available (not mentioning all):		For example, the following data import methods are available (not mentioning all):
	* Data can be pushed into AWS, Azure or GCP blob storage where Snowflake will immediately load it.		* Data can be pushed into AWS, Azure or GCP blob storage where Snowflake will immediately load it.
	* Snowflake supports Apache Iceberg tables, data from many kind of data lakes is available in Snowflake. When using Iceberg tables, the data is not phycically copied to Snowflake, so the source-of-truth remains in the external datasource.		* Snowflake supports Apache Iceberg tables, data from many kind of data lakes is available in Snowflake. When using Iceberg tables, the data is not phycically copied to Snowflake, so the source-of-truth remains in the external datasource.

Ollvihe: /* Bring Data to Snowflake */

2026-03-09T17:29:20Z

Bring Data to Snowflake

Ollvihe: /* Bring Data to Snowflake */

2026-03-09T14:07:21Z

Bring Data to Snowflake

← Older revision		Revision as of 14:07, 9 March 2026
Line 64:		Line 64:
	* Salesforce provides a connector to Snowflake that can synchronize Salesforce data to Snowflake. Both the raw data from Salesforce CRM Analytics and transformed data can be pushed to Snowflake. (https://help.salesforce.com/s/articleView?id=analytics.bi_integrate_connectors_sync_out_snowflake.htm, https://help.salesforce.com/s/articleView?id=analytics.bi_integrate_connectors_snowflake_settings.htm, https://help.salesforce.com/s/articleView?id=analytics.bi_integrate_connectors_output_snowflake.htm).		* Salesforce provides a connector to Snowflake that can synchronize Salesforce data to Snowflake. Both the raw data from Salesforce CRM Analytics and transformed data can be pushed to Snowflake. (https://help.salesforce.com/s/articleView?id=analytics.bi_integrate_connectors_sync_out_snowflake.htm, https://help.salesforce.com/s/articleView?id=analytics.bi_integrate_connectors_snowflake_settings.htm, https://help.salesforce.com/s/articleView?id=analytics.bi_integrate_connectors_output_snowflake.htm).
	* Data can be ingested from Postgres or MySql databases.		* Data can be ingested from Postgres or MySql databases.
	* Snowflake Connector for Kafka reads data from one or more Apache Kafka topics and loads the data into a Snowflake table. (https://docs.snowflake.com/en/user-guide/kafka-connector)		* Snowflake Connector for Kafka reads data from one or more Apache Kafka topics and loads the data into a Snowflake table. This solution is ideal if using the Kafka distributed event store and stream-processing platform. (https://docs.snowflake.com/en/user-guide/kafka-connector)
	* [[QPR_ProcessAnalyzer_ScriptLauncher\|QPR ScriptLauncher]] can be installed to on-premise machine where it can access and extract data from on-premise systems behind firewall and load the data to Snowflake through QPR ProcessAnalyzer. Supported datasources are ODBC, web service API's, SAP and Salesforce.		* [[QPR_ProcessAnalyzer_ScriptLauncher\|QPR ScriptLauncher]] can be installed to on-premise machine where it can access and extract data from on-premise systems behind firewall and load the data to Snowflake through QPR ProcessAnalyzer. Supported datasources are ODBC, web service API's, SAP and Salesforce.
	* [[Importing_Data_to_Datatable_from_CSV_File\|CSV files can imported]] using QPR ProcessAnalyzer user interface. The import can add new rows and new columns on top of an exiting data. The import is also able the update existing data by matching the existing and new rows using a row identifier column.		* [[Importing_Data_to_Datatable_from_CSV_File\|CSV files can imported]] using QPR ProcessAnalyzer user interface. The import can add new rows and new columns on top of an exiting data. The import is also able the update existing data by matching the existing and new rows using a row identifier column.

Ollvihe: /* Bring Data to Snowflake */

2026-03-09T13:54:00Z

Bring Data to Snowflake

← Older revision		Revision as of 13:54, 9 March 2026
Line 60:		Line 60:

	The following methods are available (not mentioning all):		The following methods are available (not mentioning all):
	* Data can be pushed into AWS, Azure or GCP blob storage where Snowflake will immediately load it.		* Data can be pushed into AWS, Azure or GCP blob storage where Snowflake will immediately load it.
	* Snowflake Connector for Kafka reads data from one or more Apache Kafka topics and loads the data into a Snowflake table.
	* SAP Business Data Cloud can be integrated with Snowflake using BDC Connect for Snowflake. The integration is based on the delta sharing enabling federated real time access (no data replication) through the Snowflake Horizon Catalog. (https://docs.snowflake.com/en/user-guide/data-integration/openflow/connectors/about-sap-snowflake)		* SAP Business Data Cloud can be integrated with Snowflake using BDC Connect for Snowflake. The integration is based on the delta sharing enabling federated real time access (no data replication) through the Snowflake Horizon Catalog. (https://docs.snowflake.com/en/user-guide/data-integration/openflow/connectors/about-sap-snowflake)
	* Salesforce provides a connector to Snowflake that can synchronize Salesforce data to Snowflake. Both the raw data from Salesforce CRM Analytics and transformed data can be pushed to Snowflake. (https://help.salesforce.com/s/articleView?id=analytics.bi_integrate_connectors_sync_out_snowflake.htm, https://help.salesforce.com/s/articleView?id=analytics.bi_integrate_connectors_snowflake_settings.htm, https://help.salesforce.com/s/articleView?id=analytics.bi_integrate_connectors_output_snowflake.htm).		* Salesforce provides a connector to Snowflake that can synchronize Salesforce data to Snowflake. Both the raw data from Salesforce CRM Analytics and transformed data can be pushed to Snowflake. (https://help.salesforce.com/s/articleView?id=analytics.bi_integrate_connectors_sync_out_snowflake.htm, https://help.salesforce.com/s/articleView?id=analytics.bi_integrate_connectors_snowflake_settings.htm, https://help.salesforce.com/s/articleView?id=analytics.bi_integrate_connectors_output_snowflake.htm).
	* Data can be ingested from Postgres or MySql databases.		* Data can be ingested from Postgres or MySql databases.
			* Snowflake Connector for Kafka reads data from one or more Apache Kafka topics and loads the data into a Snowflake table. (https://docs.snowflake.com/en/user-guide/kafka-connector)
	* [[QPR_ProcessAnalyzer_ScriptLauncher\|QPR ScriptLauncher]] can be installed to on-premise machine where it can access and extract data from on-premise systems behind firewall and load the data to Snowflake through QPR ProcessAnalyzer. Supported datasources are ODBC, web service API's, SAP and Salesforce.		* [[QPR_ProcessAnalyzer_ScriptLauncher\|QPR ScriptLauncher]] can be installed to on-premise machine where it can access and extract data from on-premise systems behind firewall and load the data to Snowflake through QPR ProcessAnalyzer. Supported datasources are ODBC, web service API's, SAP and Salesforce.
	* [[Importing_Data_to_Datatable_from_CSV_File\|CSV files can imported]] using QPR ProcessAnalyzer user interface. The import can add new rows and new columns on top of an exiting data. The import is also able the update existing data by matching the existing and new rows using a row identifier column.		* [[Importing_Data_to_Datatable_from_CSV_File\|CSV files can imported]] using QPR ProcessAnalyzer user interface. The import can add new rows and new columns on top of an exiting data. The import is also able the update existing data by matching the existing and new rows using a row identifier column.

Ollvihe: /* Bring Data to Snowflake */

2026-03-09T11:48:09Z

Bring Data to Snowflake

← Older revision		Revision as of 11:48, 9 March 2026
Line 61:		Line 61:
	The following methods are available (not mentioning all):		The following methods are available (not mentioning all):
	* Data can be pushed into AWS, Azure or GCP blob storage where Snowflake will immediately load it.		* Data can be pushed into AWS, Azure or GCP blob storage where Snowflake will immediately load it.
	* Snowflake ~~has native connector to~~ Kafka ~~for real time event stream ingestion~~.		* Snowflake Connector for Kafka reads data from one or more Apache Kafka topics and loads the data into a Snowflake table.
	* SAP Business Data Cloud can be integrated with Snowflake using BDC Connect for Snowflake. The integration is based on the delta sharing enabling federated real time access (no data replication) through the Snowflake Horizon Catalog. (https://docs.snowflake.com/en/user-guide/data-integration/openflow/connectors/about-sap-snowflake)		* SAP Business Data Cloud can be integrated with Snowflake using BDC Connect for Snowflake. The integration is based on the delta sharing enabling federated real time access (no data replication) through the Snowflake Horizon Catalog. (https://docs.snowflake.com/en/user-guide/data-integration/openflow/connectors/about-sap-snowflake)
	* Salesforce provides a connector to Snowflake that can synchronize Salesforce data to Snowflake. Both the raw data from Salesforce CRM Analytics and transformed data can be pushed to Snowflake. (https://help.salesforce.com/s/articleView?id=analytics.bi_integrate_connectors_sync_out_snowflake.htm, https://help.salesforce.com/s/articleView?id=analytics.bi_integrate_connectors_snowflake_settings.htm, https://help.salesforce.com/s/articleView?id=analytics.bi_integrate_connectors_output_snowflake.htm).		* Salesforce provides a connector to Snowflake that can synchronize Salesforce data to Snowflake. Both the raw data from Salesforce CRM Analytics and transformed data can be pushed to Snowflake. (https://help.salesforce.com/s/articleView?id=analytics.bi_integrate_connectors_sync_out_snowflake.htm, https://help.salesforce.com/s/articleView?id=analytics.bi_integrate_connectors_snowflake_settings.htm, https://help.salesforce.com/s/articleView?id=analytics.bi_integrate_connectors_output_snowflake.htm).

Ollvihe: /* Bring Data to Snowflake */

2026-03-08T23:12:16Z

Bring Data to Snowflake

← Older revision		Revision as of 23:12, 8 March 2026
Line 63:		Line 63:
	* Snowflake has native connector to Kafka for real time event stream ingestion.		* Snowflake has native connector to Kafka for real time event stream ingestion.
	* SAP Business Data Cloud can be integrated with Snowflake using BDC Connect for Snowflake. The integration is based on the delta sharing enabling federated real time access (no data replication) through the Snowflake Horizon Catalog. (https://docs.snowflake.com/en/user-guide/data-integration/openflow/connectors/about-sap-snowflake)		* SAP Business Data Cloud can be integrated with Snowflake using BDC Connect for Snowflake. The integration is based on the delta sharing enabling federated real time access (no data replication) through the Snowflake Horizon Catalog. (https://docs.snowflake.com/en/user-guide/data-integration/openflow/connectors/about-sap-snowflake)
	* Salesforce provides a connector to Snowflake that can synchronize Salesforce data to Snowflake. Both the raw data from Salesforce CRM Analytics and transformed data can be pushed to Snowflake. (https://help.salesforce.com/s/articleView?id=analytics.bi_integrate_connectors_sync_out_snowflake.htm, https://help.salesforce.com/s/articleView?id=analytics.bi_integrate_connectors_snowflake_settings.htm, https://help.salesforce.com/s/articleView?id=analytics.bi_integrate_connectors_output_snowflake.htm).		* Salesforce provides a connector to Snowflake that can synchronize Salesforce data to Snowflake. Both the raw data from Salesforce CRM Analytics and transformed data can be pushed to Snowflake. (https://help.salesforce.com/s/articleView?id=analytics.bi_integrate_connectors_sync_out_snowflake.htm, https://help.salesforce.com/s/articleView?id=analytics.bi_integrate_connectors_snowflake_settings.htm, https://help.salesforce.com/s/articleView?id=analytics.bi_integrate_connectors_output_snowflake.htm).
			* Data can be ingested from Postgres or MySql databases.
	* [[QPR_ProcessAnalyzer_ScriptLauncher\|QPR ScriptLauncher]] can be installed to on-premise machine where it can access and extract data from on-premise systems behind firewall and load the data to Snowflake through QPR ProcessAnalyzer. Supported datasources are ODBC, web service API's, SAP and Salesforce.		* [[QPR_ProcessAnalyzer_ScriptLauncher\|QPR ScriptLauncher]] can be installed to on-premise machine where it can access and extract data from on-premise systems behind firewall and load the data to Snowflake through QPR ProcessAnalyzer. Supported datasources are ODBC, web service API's, SAP and Salesforce.
	* [[Importing_Data_to_Datatable_from_CSV_File\|CSV files can imported]] using QPR ProcessAnalyzer user interface. The import can add new rows and new columns on top of an exiting data. The import is also able the update existing data by matching the existing and new rows using a row identifier column.		* [[Importing_Data_to_Datatable_from_CSV_File\|CSV files can imported]] using QPR ProcessAnalyzer user interface. The import can add new rows and new columns on top of an exiting data. The import is also able the update existing data by matching the existing and new rows using a row identifier column.

Ollvihe: /* Bring Data to Snowflake */

2026-03-08T23:11:41Z

Bring Data to Snowflake

← Older revision		Revision as of 23:11, 8 March 2026
Line 58:		Line 58:

	[[File:SnowflakeIntegrationOptions.png\|1200px]]		[[File:SnowflakeIntegrationOptions.png\|1200px]]

	The following methods are available (not mentioning all):		The following methods are available (not mentioning all):
	* Data can be pushed into AWS, Azure or GCP blob storage where Snowflake will immediately load it.		* Data can be pushed into AWS, Azure or GCP blob storage where Snowflake will immediately load it.

Ollvihe: /* Bring Data to Snowflake */

2026-03-08T23:10:20Z

Bring Data to Snowflake

← Older revision		Revision as of 23:10, 8 March 2026
Line 57:		Line 57:
	There are various methods to bring data to Snowflake, e.g., using Snowflake own features, QPR ProcessAnalyzer ETL functionality, or 3rd party tools. The following picture illustrate the variaty of methods:		There are various methods to bring data to Snowflake, e.g., using Snowflake own features, QPR ProcessAnalyzer ETL functionality, or 3rd party tools. The following picture illustrate the variaty of methods:

	[[File:SnowflakeIntegrationOptions.png]]		[[File:SnowflakeIntegrationOptions.png\|1200px]]
	The following methods are available (not mentioning all):		The following methods are available (not mentioning all):
	* Data can be pushed into AWS, Azure or GCP blob storage where Snowflake will immediately load it.		* Data can be pushed into AWS, Azure or GCP blob storage where Snowflake will immediately load it.

← Older revision		Revision as of 17:29, 9 March 2026
Line 55:		Line 55:
	== Bring Data to Snowflake ==		== Bring Data to Snowflake ==

	There are ~~various~~ methods to ~~bring~~ data to Snowflake, e.g., using Snowflake own features, QPR ProcessAnalyzer ETL functionality, or 3rd party tools. The following picture ~~illustrate~~ the variaty of methods:		There are different methods to ingest data into Snowflake, e.g., using Snowflake own features, QPR ProcessAnalyzer ETL functionality, or 3rd party tools. The following picture illustrates the variaty of methods:

	[[File:SnowflakeIntegrationOptions.png\|1200px]]		[[File:SnowflakeIntegrationOptions.png\|1200px]]
Line 61:		Line 61:
	The following methods are available (not mentioning all):		The following methods are available (not mentioning all):
	* Data can be pushed into AWS, Azure or GCP blob storage where Snowflake will immediately load it.		* Data can be pushed into AWS, Azure or GCP blob storage where Snowflake will immediately load it.
			* Snowflake supports Apache Iceberg tables, data from many kind of data lakes is available in Snowflake. When using Iceberg tables, the data is not phycically copied to Snowflake, so the source-of-truth remains in the external datasource.
	* SAP Business Data Cloud can be integrated with Snowflake using BDC Connect for Snowflake. The integration is based on the delta sharing enabling federated real time access (no data replication) through the Snowflake Horizon Catalog. (https://docs.snowflake.com/en/user-guide/data-integration/openflow/connectors/about-sap-snowflake)		* SAP Business Data Cloud can be integrated with Snowflake using BDC Connect for Snowflake. The integration is based on the delta sharing enabling federated real time access (no data replication) through the Snowflake Horizon Catalog. (https://docs.snowflake.com/en/user-guide/data-integration/openflow/connectors/about-sap-snowflake)
	* Salesforce provides a connector to Snowflake that can synchronize Salesforce data to Snowflake. Both the raw data from Salesforce CRM Analytics and transformed data can be pushed to Snowflake. (https://help.salesforce.com/s/articleView?id=analytics.bi_integrate_connectors_sync_out_snowflake.htm, https://help.salesforce.com/s/articleView?id=analytics.bi_integrate_connectors_snowflake_settings.htm, https://help.salesforce.com/s/articleView?id=analytics.bi_integrate_connectors_output_snowflake.htm).		* Salesforce provides a connector to Snowflake that can synchronize Salesforce data to Snowflake. Both the raw data from Salesforce CRM Analytics and transformed data can be pushed to Snowflake. (https://help.salesforce.com/s/articleView?id=analytics.bi_integrate_connectors_sync_out_snowflake.htm, https://help.salesforce.com/s/articleView?id=analytics.bi_integrate_connectors_snowflake_settings.htm, https://help.salesforce.com/s/articleView?id=analytics.bi_integrate_connectors_output_snowflake.htm).
	* Data can be ingested from ~~Postgres~~ or ~~MySql databases~~.		* Data can be ingested into Snowflake from PostgreSQL or MySQL database systems.
	* Snowflake Connector for Kafka reads data from one or more Apache Kafka topics and loads the data into a Snowflake table. This solution is ideal if using the Kafka distributed event store and stream-processing platform. (https://docs.snowflake.com/en/user-guide/kafka-connector)		* Snowflake Connector for Kafka reads data from one or more Apache Kafka topics and loads the data into a Snowflake table. This solution is ideal if using the Kafka distributed event store and stream-processing platform. (https://docs.snowflake.com/en/user-guide/kafka-connector)
	* [[QPR_ProcessAnalyzer_ScriptLauncher\|QPR ScriptLauncher]] can be installed to on-premise machine where it can access and extract data from on-premise systems behind firewall and load the data to Snowflake through QPR ProcessAnalyzer. Supported datasources are ODBC, web service API's, SAP and Salesforce.		* [[QPR_ProcessAnalyzer_ScriptLauncher\|QPR ScriptLauncher]] can be installed to an on-premise machine where it can access and extract data from on-premise systems that are located behind a firewall and load the data into Snowflake through QPR ProcessAnalyzer. Supported datasources are ODBC, web service API's, SAP and Salesforce.
	* [[Importing_Data_to_Datatable_from_CSV_File\|CSV files can imported]] using QPR ProcessAnalyzer user interface. The import can add new rows and new columns on top of an exiting data. The import is also able the update existing data by matching the existing and new rows using a row identifier column.		* [[Importing_Data_to_Datatable_from_CSV_File\|CSV files can imported]] using QPR ProcessAnalyzer user interface. This is the manual import method available for ad-hoc imports, but it still supports CSV files of unlimited sizes (even gigabytes). The import can add new rows and new columns on top of an exiting data. The import is also able the update existing data by matching the existing and new rows using a row identifier column.

	== Authentication, Authorization and Security Model ==		== Authentication, Authorization and Security Model ==