Uploading data

Before being able to analyze and process your data using the getML software, you have to upload it to the engine first.

You already uploaded and saved your data once?

Refer to Lifecycle of a DataFrame for information on how to restore them.

You got some fresh data the engine has never seen before?

Read on and we will get your data science project started.

At the end of this step you will have your data in data frame objects in the getML engine and will be ready to annotate them (see Annotating data).

Unified import interface

The getML Python API provides an unified import interface requiring similar arguments and resulting in the same output format, regardless of the data source.

You can use one of the dedicated from_csv(), from_pandas(), from_db(), and from_json() class methods to construct a data frame object in the getML engine, fill it with the provided data, and retrieve a DataFrame handle in the Python API.

If you already have a data frame object in place, you can use the read_csv(), read_pandas(), read_db(), or read_json() methods of the corresponding DataFrame handle to either replace its content with new data or append to it.

All those functions also have their counterparts for exporting called to_csv(), to_pandas(), to_db(), and to_json().

The particularities of the individual formats will be covered in the following sections:

• CSV interface
• Pandas interface
• JSON interface
• SQLite3 interface
• MySQL interface
• MariaDB interface
• PostgreSQL interface
• Greenplum interface

Data Frames

The resulting DataFrame instance in the Python API represents a handle to the corresponding data frame object in the getML engine. The mapping between the two is done based on the name of the object, which has to be unique. Similarly, the names of the columns are required to be unique within the data frame they are associated with.

Handling of NULL values

Unfortunately, data sources often contain missing or corrupt data - also called NULL values. getML is able to work with missing values except for the target variable, which must not contain any NULL values (because having NULL targets does not make any sense). See Join key for details about their handling during the construction of the data model.

During import a NULL value is automatically inserted at all occurrences of the strings “nan”, “None”, “NA”, or an empty string as well as at all occurrences of None and NaN.