Reference
About data types¶
Amperity supports the following data types when structured data is made available to Amperity. Complex types–arrays, maps, and structs–must be converted to individual fields prior to making them available to the Stitch process.
Array¶
A composite fully-structred data type that stores a sequence of elements where each element is a set of fields and values.
Important
Amperity will load a data source with an array to a domain table.
For data sources with an array, you must use a custom domain table to convert all elements in the array to individual fields in the table output.
After all elements are converted to individual fields in the custom domain table output you assign semantic tags and keys to individual fields in the table.
An array may be used as part of a common table expression (CTE) in Spark SQL and Presto SQL as long as output of the query does not have a field with an array data type.
Boolean¶
A value that is TRUE, FALSE, or NULL.
Date¶
An ISO-8601 compliant date value.
Datetime¶
An ISO-8601 compliant date and time value.
Compare to
Databricks: INTERVAL and TIMESTAMP
Google BigQuery: DATETIME , TIME , and TIMESTAMP
Snowflake: DATETIME , TIME , TIMESTAMP , TIMESTAMP_LTZ , TIMESTAMP_NTZ , and TIMESTAMP_TZ
Decimal¶
A fixed point number with a configurable number of characters before and after a decimal point.
Compare to
Databricks: DECIMAL (p,s)
Google BigQuery: BIGNUMERIC , DECIMAL (p,s) , and NUMERIC (p,s)
Snowflake: NUMBER (p,s)
Float¶
A floating point number with a configurable number of characters before and after a decimal point.
Integer¶
A numeric value of various lengths.
Map¶
A composite fully-structured data type that stores data as key-value pairs.
Important
Amperity will load a data source with a map to a domain table.
For data sources with a map, you must use a custom domain table to convert all elements in the map to individual fields in the table output.
After all elements are converted to individual fields in the custom domain table output you assign semantic tags and keys to individual fields in the table.
A map may be used as part of a common table expression (CTE) in Spark SQL and Presto SQL as long as output of the query does not have a field with a map data type.
String¶
A sequence of characters.
Struct¶
A composite fully-structured data type that groups data as a sequence of fields within the structure.
Important
Amperity will load a data source with a struct to a domain table.
For data sources with a struct, you must use a custom domain table to convert all elements in the struct to individual fields in the table output.
After all elements are converted to individual fields in the custom domain table output you assign semantic tags and keys to individual fields in the table.
A struct may be used as part of a common table expression (CTE) in Spark SQL and Presto SQL as long as output of the query does not have a field with a struct data type.
Unsupported¶
Semi-structured data types are unsupported and must be converted to a fully-structured data type.