Guided setup
Databricks¶
Amperity Bridge for Databricks is a first-class integration that enables bi-directional data access between Amperity and Databricks without copying data or scheduling ETL workloads.
A connection between Databricks and Amperity requires configuration steps to be made in both Amperity and Databricks.
About encryption, credentials, and audit logs
Delta Sharing uses end-to-end TLS encryption from client to server to storage account along with short-lived credentials, such as pre-signed URLs, to access data.
Review Databricks documentation for security best practices , including setting token lifetimes for metastores, rotating credentials, applying granularity for shares and recipients, configuring IP access lists, and audit logging.
Audit logging occurs in Databricks and in Amperity.
Audit logging in Amperity shows each users actions and interactions along with access to a history of workflows with tasks that use a bridge to connect Amperity and Databricks.
Databricks captures Delta Sharing provider events , which includes logging for when a recipient (Amperity) accesses data.
Note
Review Connect Amperity Bridge to Databricks for more information about prerequisites, requirements, and optional configurations for Databricks. All configuration prerequisites must be completed before stepping through this workflow.
To add an inbound bridge
|
On the Identity resolution page in Quick start, from the Add data source dropdown select Databricks. 
This opens the Add bridge dialog box. 
Add a name and description for the bridge or select an existing bridge, and then click Confirm. |
|
Connect the bridge to Databricks by uploading the credential file that was downloaded from the activation link . There are two ways to upload the credential file:
After the credential file is uploaded, click Continue. Important You can download the credential file only once. Recipients should treat the download credential as a secret and must not share it outside of their organization. If you have concerns that a credential may have been handled insecurely you can rotate credentials at any time. When finished, click Continue. This opens the Select tables dialog box. |
|
Use the Select tables dialog box to select any combination of schemas and tables to be connected to Amperity. 
If you select a schema, all tables in that schema are connected. Any new tables added later need to manually added to the connection. When finished, click Next. This opens the Domain table mapping dialog box. |
|
Map the tables connected from Databricks to domain tables in Amperity. 
Tables connected with Amperity must have unique names among all domain tables. Primary keys are not assigned. Semantic tags are not applied. Tip Use a custom domain table to assign primary keys, apply semantic tags, and shape data within connected tables to the requirements of your workflows in Amperity. When finished, click Save and sync. This will start a workflow that synchronizes data from Databricks to Amperity and creates the mapped table names. You can manually sync tables connected with Amperity using the Sync option from the Actions menu for the inbound bridge. |
Data types¶
Most Databricks data types are supported by Amperity Bridge.
Important
If an upstream data type changes, edit the bridge in Amperity, accept the changes, and then save and sync the bridge.
The following table describes how Databricks data types map to Amperity data types.
Databricks data type |
Amperity data type |
|---|---|
|
A sequence of elements with the type of |
Array A sequence of elements with data types and values supported by Amperity data types. |
|
A 64-bit integer. |
Integer A numeric value, such as the quantity of items purchased. For example:
|
|
Variable-length byte sequence values. |
Warning The Databricks BINARY data type is unsupported. Use a view in Databricks to exclude fields with BINARY data types from tables before sharing them with Amperity. |
|
A value that can be TRUE, FALSE, or NULL. |
Boolean A value that can be TRUE, FALSE, or NULL. |
|
Year, month, and day, without a time zone. |
Date An ISO-8601 compliant date value, such as a birthdate. For example:
|
|
A number with a specified maximum precision and fixed scale. Precision is 1-38 and scale is the number of digits to the right of the decimal point. Default precision is 10. |
Decimal A fixed point number, such as for prices or message sizes. The number of characters in the decimal value is configurable. For example:
|
|
8-byte double-precision floating point numbers. |
Float A floating point number. For example:
|
|
4-byte single-precision floating point numbers. |
Float A floating point number. For example:
|
|
A 4-byte signed integer. |
Integer A numeric value, such as the quantity of items purchased. For example:
|
|
Intervals of time on a scale of seconds or months. |
Datetime
|
|
A set of key-value pairs. Each pair must have a data type that maps to a supported Amperity data type. |
Map A set of key-value pairs that map to supported Amperity data types. |
|
A 2-byte signed integer ranging from -32,768 to 32,767. |
Integer A numeric value, such as the quantity of items purchased. For example:
|
|
Character string values. |
String A sequence of characters, such as first and last names, email addresses, physical addresses, UUIDs and other IDs, phone numbers, zip codes, product names, and descriptions. May be empty. For example:
|
|
A structure described by a sequence of fields. Each field must have a data type that maps to a supported Amperity data type. |
Struct A sequence of fields. |
|
Values for year, month, day, hour, minute, and second with the session local timezone. |
Datetime ISO-8601 compliant date and time values, such as a purchase or transaction, the time at which data was last updated, or a campaign launch date. For example:
|
|
A 1-byte signed integer ranging from -128 to 127. |
Integer A numeric value, such as the quantity of items purchased. For example:
|