About DataGrid

DataGrid pulls all your customer data from any source-–online, offline, historical, streaming–without requiring schema planning or ETL. DataGrid sends results of any size and type to any of your downstream applications and workflows.

Important

DataGrid is configured by users of Amperity with administrative privileges. These users may belong to your organization and manage Amperity directly or they may be representatives from Amperity who manage DataGrid for you on your behalf. This documentation is written for the administrative users who belong to your organization. Some steps, such as configuring Google Ads and Facebook Ads as destinations, or configuring the SFTP site that is included with Amperity, require additional steps to be done by Amperity representatives and should be initiated by filing a support ticket or by contacting your Amperity representative and asking them to start that process on your behalf.

DataGrid is the multi-patented infrastructure that powers Amperity. DataGrid handles your customer data with speed, scale, and accuracy and provides optimal performance and interoperability when using any combination of AmpID, Amp360, and AmpIQ.

DataGrid is a cloud-native high-performance infrastructure that runs at scale on your choice of Amazon AWS or Microsoft Azure.

• 100+ integration options to handle any type of data.

• Pull unlimited amounts of structured and semi-structured raw data to Amperity from any source without the need for schema planning or pre-shaping data.

• Use multiple databases, SQL database querying, and the Amperity data explorer to manage your workflows and data transformation options.

• Send data shaped for any destination in any format, such as sending full databases to analytics environments, segments to campaign tools, or attributes to personalization engines.

• Use the sandbox environment to safely make changes with zero downtime to the production environment, including data sources, data models, and workflows.

• Rely on security features, such as SOC2 certification, SSO integration, PII obfuscation, user actions auditing, and more, to keep your data safe.

Common workflows

The most common workflows for DataGrid involve pulling data to Amperity from a customer data source or sending data from Amperity to a downstream workflow. There are five main areas:

• Pulling data to Amperity

• Configuring and running Stitch

• Configuring the customer 360 database, along with any other custom databases and tables your data requires

• Defining queries that interact with databases and tables in the Customer 360 tab, some of which are used for QA purposes, others are used to generate results to be sent to downstream workflows outside of Amperity

• Sending data from Amperity

After configuring DataGrid to pull data to and send data from Amperity, use these components to configure more complex and more valuable use cases, such as:

• Consolidate data across brands

• Consolidate historical data

• Enterprise change management

• Manage customer data directly

• Reshape data for downstream workflows

Configure DataGrid

The following sections provide an overview of configuring DataGrid. The person who configures DataGrid depends on how you have chosen to run Amperity. This might be a member of your organization who has completed the DataGrid certification process or it might be a representative from Amperity who works closely with you during implementation to configure Amperity exactly as you require, but then over time as Amperity runs on a daily basis.

Tip

Some information about your configuration must be shared with Amperity, such as a username and passcode required to authenticate and access to various cloud storage services or REST APIs. This information, when it must be shared with an Amperity representative, should be shared using SnapPass.

SnapPass allows secrets to be shared in a secure, ephemeral way. Input a single or multi-line secret, along with an expiration time, and then generate a one-time use URL that may be shared with anyone. Amperity uses SnapPass for sharing credentials to systems with customers.

Open Snappass to send information to your Amperity representative.

Sandboxes

A sandbox is a snapshot of the configuration state of your production tenant that is made available as a copy. Use a sandbox to safely make configuration changes, and then promote those changes back to your production tenant.

A sandbox workflow provides a version history of all changes made in the sandbox, including when they were made, who made them, and which components within Amperity were changed. A record is maintained of all changes that are promoted from the sandbox to the production tenant.

Important

Only Sandbox Administrators are granted permission to merge promoted changes to a production tenant. This means that in many cases, promoted changes must be reviewed and accepted prior to completing the promotion workflow.

Make a request to your Amperity representative and/or to Amperity support for help with getting changes in a sandbox environment promoted to your production environment.

A sandbox workflow is similar to the workflows software engineers use with version control systems: a series of small, iterative changes are submitted and approved for use in the production environment.

Configuration states go in two directions:

1. Configuration states are pulled from the production tenant to the sandbox.

   Note

   Data is not moved from your production environment to a sandbox environment.

   A banner within a sandbox notifies you if there are states that can be pulled from the production environment or promoted from the sandbox environment, along with notifying you of any conflicts that may exist between the configuration states.

   When updates for the sandbox environment are available, a banner similar to the following is shown:

   

2. Changes to configuration states that are made in a sandbox may be promoted to your production tenant.

   Note

   Data is not moved from a sandbox environment to your production environment.

   A detailed list of all changes to all components and data sources is available for each promotion workflow.

   When changes in your sandbox environment are ready to be promoted, a banner similar to the following is shown:

   

3. If there are conflicts between the two configuration states, one of the configuration states must be chosen.

4. Amperity will run a series of validations against the changes you have made in the sandbox environment against all workflows within the sandbox that depend on the changes you have made.

   In some cases, a change will require additional updates to upstream or downstream dependencies within the sandbox that must be addressed before you can promote those changes to your production environment.

   For example: a schema mismatch may exist between tables, a database table may be missing, or a data source may be unavailable.

   Each validation error will be shown. Click on a validation error to view more details about how you can resolve the validation error.

A sandbox must be enabled for use in Amperity. To request a sandbox for your tenant, contact your Amperity support representative directly, by using the Amperity Support Portal, or by sending email to support@amperity.com.

Important

When making changes to a tenant which has an SLA workflow, you must make changes in a sandbox to ensure that workflows in your production environment not interrupted.

Pull data to Amperity

Pulling data to Amperity falls into four broad categories:

• Pull data from cloud storage

• Pull data from file transfer

• Pull data from REST API

• Pull data from warehouse

Another option for specific use cases is to use the Streaming Ingest REST API to pull data to Amperity.

The process of pulling data to Amperity is managed from the Sources tab in the Amperity user interface.

The Sources tab contains the following components:

• Saved queries reshape data after pulling it to Amperity and before making it available to a feed.

• Feeds define the schema for each individual data source.

• Domain tables represent each data source after it has been processed against a feed.

• Domain transforms use existing domain tables and Spark SQL to add a custom domain table.

• Couriers define how data is pulled to Amperity, along with specifying the location from which it is pulled.

• Courier groups define schedules for pulling data to Amperity.

Configure and run Stitch

Stitch must be configured to run in a way that ensures that all data sources that contain customer records (names, email addresses, physical addresses, and other PII data) are made available to the Stitch process. The outcome of the Stitch process generates an Amperity ID for each unique customer record across all of your data.

Stitch is largely a transparent process, but there are ways to tune how it understands your data. And you can explore the results of the Stitch process against your data directly from the Stitch tab.

Build databases

The customer 360 database starts with the output of the Stitch process, which is a collection of database tables from which you can build your customer 360.

The tables that the Stitch process outputs include:

• Transaction_Attributes

• Unified_Transactions

• Unified_Itemized_Transactions

• Unified_Profiles

• Unified_Scores

• Unified_Customer

In addition to these tables, you must build a Merged_Customers table that defines certain rollup behaviors for profile data and, if using AmpIQ, transactions, itemized transactions, and customer profile data.

The databases and tables that may be present in the Customer 360 tab are not limited to only those output by the Stitch process. You can configure domain tables to be directly passed through to the customer 360 database and, using Spark SQL, you can build any custom database or table that you require.

All databases are managed from the Customer 360 tab.

After you have build your customer 360 database and it is running against a representative collection of your data sources, you can start to look at extending the database for additional workflows and use cases, such as:

1. Blocklisting values from Stitch or from customer 360 data

2. Applying specific labels to data

3. Adding calculations

4. Extending data to focus on address-based householding, first-party, third-party data

5. Adding CCPA or GDPR privacy rights workflows

6. Extending for customer interactions, such as order-level and item-level transactions data and product catalogs

7. Workflow-specific databases or tables to support PII consolidation or master data management (MDM) use cases

8. Adding support to enable additional Amperity features

Run queries

The Queries tab uses Presto SQL to interact with any database and table that is present in the Customer 360 database. You can use a visual SQL editor for simple queries and a SQL editor for more complex queries. Amperity supports nearly all of the functionality of Presto SQL that you would use when building a SELECT statement. See the Amperity Presto SQL reference for specific reference, but you may also refer to the Presto SQL documentation for anything not covered in the Amperity reference.

Use the Queries tab to review the quality of Stitch output, the quality of transactions and itemized transactions data, and to build queries, the results of which can be sent from Amperity any downstream workflow.

Send data from Amperity

Sending data from Amperity falls into similar categories:

• Send data to cloud storage

• Send data to file transfer

• Send data to REST API

• Send data to warehouse

The process of sending data from Amperity is managed from the Destinations tab in the Amperity user interface.

The Destinations tab contains the following components:

• Destinations, which define how data is sent from Amperity and the location to which it is sent

• Data templates, which map fields in the customer 360 database to the fields that are required by the downstream workflow

• Orchestrations, which define schedules for sending data from Amperity