Importing Data to Datatable from CSV File: Difference between revisions

From QPR ProcessAnalyzer Wiki
Jump to navigation Jump to search
No edit summary
Line 21: Line 21:
|-
|-
||String
||String
||Data type for textual data. Also suitable for distinct values that are not numerical, e.g. low, medium high.
||Data type for textual data. Also suitable for distinct values that are not numerical, e.g. low, medium high. The length of the string is unlimited, but in practice it's more efficient to use shorter strings (e.g. maximum of 100 characters) for process mining purposes.
||
||
|-
|-
||Integer
||Integer
||Data type for whole numbers, i.e. numbers that don't need decimals.  
||Data type for whole numbers, i.e. numbers that don't need decimals. Integers are also suitable for storing distinct values that can be ordered (e.g. 1=low, 2=medium, 3=high), which makes it easier to visualize data in a logical way. Integers have a advantage over decimal numbers: it's possible to make equality comparisons for integers.
||Unit
||Unit
|-
|-
||Decimal number
||Decimal number
||
||Data type that stores numerical data with decimals. It's not possible to make equality comparisons for decimal numbers (i.e. equals than, not equals than, less or equal than, greater or equal than). Only more than and less than comparison can be made for example when filtering data.
||
||
|-
|-
||Date
||Date
||Data type for precise timestamps and
||Data type stores a precise timestamp, which is an essential data type for process mining. This data type can also be used to store less fine graining time units, such as days, weeks, months or years. In that case the stored timestamp is the point of time in the beginning of the day, week etc.
||
||
|-
|-
Line 40: Line 40:
||
||
|-
|-
||Timespan
||Duration
||
||Data type that contains a time duration (e.g. 5 hours, 14 days, 3 seconds, etc.). Corresponding expression language data type is Timespan. Duration is quite close to decimal number, except its benefit is that it also contains the duration unit (thus it doesn't need to be agreed separately). Duration type of data is also very easy to show in any other duration unit with a simple selection in the dashboard. Note that the duration data type does not store information when the duration started or ended, but only the duration itself.
||
||
|}
|}


== Supported CSV Formats ==
== Supported CSV Formats ==

Revision as of 13:38, 21 June 2021

Data can be imported to datatables from CSV files using the Workspace screen. Import is done as follows:

  1. On the left side projects hierarchy, select the project where the target datatable is located.
  2. Open the Datatables tab.
  3. Select the datatable where to import data.
  4. Click the Import button.
  5. Select the CSV file to be imported, adjust data type and conversion settings if needed, and click Start import.

Import Settings

The data import dialog has the following settings:

  • Select CSV file: Button to select the CSV file. You can use this button to change the CSV file if you want to change the file.
  • Column separator: Column separator character used by the CSV file. The column separator is automatically detected when the file is selected. In some cases, the automatic detection does not find the correct column separator, and this selection allows to change the column separator. If you want to use space as a column separator, select <space>, or if you want to use tabulator, select <tab>. Note that when changing the column separator, all column specific settings in the table are reset.
  • Skip errors: Allows to continue the import when encountering errors and import null values in place of cells where the conversion could not be made. If error skipping is not in use, the import will stop to the first encountered error. The data conversions are checked before importing actual data to the datatables, so data conversion errors will not lead to partially done imported.
  • Character encoding: Character encoding used by the CSV file. Most text files use UTF-8 encoding which is the default. If some of the characters in the CSV file look incorrect in the table preview, try to change other character encoding to find the correct one. Note that there is no automatic detection of the character encoding. Note also that changing the character encoding will reset all column specific settings

Data types

It's very important to set the most suitable data types for each column, as it greatly affects the usage of the data in the analyses. Incorrect data type usually make the analysis impossible. The following data types are available:

Data type Use Settings
String Data type for textual data. Also suitable for distinct values that are not numerical, e.g. low, medium high. The length of the string is unlimited, but in practice it's more efficient to use shorter strings (e.g. maximum of 100 characters) for process mining purposes.
Integer Data type for whole numbers, i.e. numbers that don't need decimals. Integers are also suitable for storing distinct values that can be ordered (e.g. 1=low, 2=medium, 3=high), which makes it easier to visualize data in a logical way. Integers have a advantage over decimal numbers: it's possible to make equality comparisons for integers. Unit
Decimal number Data type that stores numerical data with decimals. It's not possible to make equality comparisons for decimal numbers (i.e. equals than, not equals than, less or equal than, greater or equal than). Only more than and less than comparison can be made for example when filtering data.
Date Data type stores a precise timestamp, which is an essential data type for process mining. This data type can also be used to store less fine graining time units, such as days, weeks, months or years. In that case the stored timestamp is the point of time in the beginning of the day, week etc.
Boolean Data type that contains only two values, true and false. The null' value is also available (like in other data types) which in practise is the third possible value for boolean type of data. If no more values are needed, boolean is the most efficient data type to use. Note that in dashboards, the true and false values can be translated into user understandable texts, such as "yes" and "no".
Duration Data type that contains a time duration (e.g. 5 hours, 14 days, 3 seconds, etc.). Corresponding expression language data type is Timespan. Duration is quite close to decimal number, except its benefit is that it also contains the duration unit (thus it doesn't need to be agreed separately). Duration type of data is also very easy to show in any other duration unit with a simple selection in the dashboard. Note that the duration data type does not store information when the duration started or ended, but only the duration itself.

Supported CSV Formats