Azure Blob Storage

Amperity can pull data in a wide variety of file types, file formats, and file sizes from Azure Blob Storage, which is a common data source across all Amperity tenants.

Note

Review Pull from Azure Blob Storage for more information about prerequisites, requirements, and optional configurations for Azure Blob Storage. All configuration prerequisites must be completed before stepping through this workflow.

To add an inbound bridge

Step one.

On the Identity resolution page in Quick start, from the Add data source dropdown select Azure Blob Storage.

Add a data source.

This opens the Add courier dialog box.
Step two.

Credentials allow Amperity to connect to Azure Blob Storage and must exist before a courier can be configured to pull data from Azure Blob Storage. Select an existing credential from the Credential dropdown, and then click Continue.

Tip

A courier that has credentials that are configured correctly shows a “Connection successful” status, similar to:

Add.
Step three.

Add a name for the data source to the Source name field.

Select the file that is pulled to Amperity, either by browsing into storage and selecting it or by providing a filename pattern.

Add a data source.

Click Browse to open the File browser. Select the file that is pulled to Amperity, and then click Accept.

Use a filename pattern to define files that will be loaded on a recurring basis, but have small changes to the filename over time, such as having a datestamp appended to the filename.

Note

For a new data source, this file is also used as the sample file that is used to define the schema. For an existing data source, this file must match the schema that has already been defined.

Add a data source.

Use the PGP credential setting to specify the credentials to use for an encrypted file.

Add a data source.
Step four.

Review the file.

Add.

The contents of the file may be viewed as a table and in the raw format. Switch between these views using the Table and Raw buttons, and then click Refresh to view the file in that format.

Note

PGP encrypted files can be previewed. Apache Parquet PGP encrypted files must be less than 500 MB to be previewed.

Amperity will infer formatting details, and then add these details to a series of settings located along the left side of the file view. File settings include:

  • Delimiter

  • Compression

  • Escape character

  • Quote character

  • Header row

Review the file, and then update these settings, if necessary.

Note

Amperity supports the following file types: Apache Avro, Apache Parquet, CSV, DSV, JSON, NDJSON, PSV, TSV, and XML.

Refer to those reference pages for details about each of the individual file formats.

Files that contain nested JSON (or “complex JSON”) or XML may require using the legacy courier configuration.

Click Save and continue.
Step five.

Select a date from the calendar picker that is before today, but after the date on which the file was added to the Azure Blob Storage file system.

Add.

Leave the load options unselected, and then click Run.