Send data to SFTP

Note

This topic contains information about configuring a destination that sends query results to SFTP using orchestrations. To configure a destination that sends audiences to SFTP using campaigns see this topic .

Secure File Transfer Protocol (SFTP) is a network protocol that provides file access, file transfer, and file management over any reliable data stream.

Amperity can be configured to send Apache Parquet, CSV, JSON, NDJSON, PSV, or TSV files to any SFTP site.

Get details

Review the following details before configuring credentials for SFTP and before configuring Amperity to send Apache Parquet, CSV, JSON, NDJSON, PSV, or TSV files to any SFTP site.

Credential settings Choose “passphrase” or “private-key” as the credential type, and then configure a combination of hostname, username and passphrase, host public key, port number, and private key settings as required by the credential type.

Required configuration settings File format Configure Amperity to send Apache Parquet, CSV, JSON, NDJSON, PSV, or TSV files to any SFTP site. Some file formats allow a custom delimiter. Choose the “Custom delimiter” file format, and then add a single character to represent the custom delimiter. Remote folder The remote folder to which Amperity will send data. A “/” (forward slash) must be used as the first character for this value. For example: “/folder/name”. For example: “/folder/name”. If the path to the remote folder contains a space, use a backslash (\) character to escape that space. For example, a folder named “From ACME” should be entered as “From\ ACME”. Note All other Amperity file format settings for SFTP are optional.

Included SFTP site

Every Amperity tenant includes an SFTP site with a hostname of <tenant>.sftp.amperity.com. For example, if your company name is ACME, then your tenant’s SFTP hostname is acme.sftp.amperity.com. (The hostname is always all lowercase.)

Note

The SFTP site is provisioned by an Amperity administrator after the initial tenant creation. As such, the SFTP site is not immediately available, but this is not a long process. When the SFTP site is ready, Amperity will use SnapPass to send you the connection details. If you wish to use RSA key-based authentication, please provide the public key to your Amperity administrator when requesting SFTP access.

Once this site is provisioned, you may configure the SFTP site as an SFTP source or an SFTP destination to support any desired workflow. External customer processes can be configured to connect to the site using SFTP, after which they can add data to or pick up data from the site to support any external workflow.

Caution

The SFTP server has a 30-day limit on data and after 30 days, data may be moved automatically to an archive location. This location is still accessible to Amperity in case it becomes necessary to reuse it.

Important

The hostname for the SFTP site is always <tenant-name>.sftp.amperity.com. Some older tenants may still be using the legacy address sftp.amperity.com, if so, please contact your Amperity administrator about migrating.

Configure credentials

Configure credentials for SFTP before adding a destination.

An individual with access to SFTP should use SnapPass to securely share credential details with the individual who will configure Amperity.

To configure credentials for SFTP

	From the Settings page, select the Credentials tab, and then click the Add credential button.
	In the Credentials settings dialog box, do the following: From the Plugin dropdown, select SFTP. Assign the credential a name and description that ensures other users of Amperity can recognize when to use this destination. From the Credential type dropdown, select “passphrase” or “private-key”.
	The settings that are available for a credential are determined by the credential type and by the settings that are required by the SFTP site. The following credential settings are available for SFTP sites. Configure the settings that are required by the SFTP site to which Amperity will connect, and then click Save. Host public key The host public key for any SFTP site. Hostname Required Required. The hostname for any SFTP site. For example: “sftp.socktown.com”. Passphrase The passphrase that is associated with the username. Port The port number required by any SFTP site. Use this setting only when the SFTP site uses a non-default port value. Private key A private key that is used to encrypt files that are sent to any SFTP site. This setting is only required when the “private-key” credential type is selected. Username Required. A username with access to any SFTP site. This username must be allowed to write data to the remote folder to which Amperity will send data.

Add destination

Use a sandbox to configure a destination for SFTP. Before promoting your changes, send a test audience, and then verify the the results in SFTP. After the end-to-end workflow has been verified, push the destination from the sandbox to production.

To add a destination for SFTP

	Open the Destinations page, and then click the Add destination button. To configure a destination for SFTP, do one of the following: Click the row in which SFTP is located. Destinations are listed alphabetically and you can scroll up and down the list. Search for SFTP. Start typing “sftp”. The list will filter to show only matching destinations. Select “SFTP”.
	Select the credential for SFTP from the Credential dropdown, and then click Continue. Tip Click the “Test connection” link on the “Configure destination” page to verify that Amperity can connect to SFTP.
	In the “Destination settings” dialog box, assign the destination a name and description that ensures other users of Amperity can recognize when to use this destination. Configure business user access By default a destination is available to all users who have permission to view personally identifiable information (PII). Enable the Admin only checkbox to restrict access to only users assigned to the Datagrid Operator and Datagrid Administrator policies. Enable the PII setting checkbox to allow users with limited access to PII access to this destination. Restricted PII access is enabled when the Restrict PII access policy option that prevents users who are assigned to that option from viewing data that is marked as PII anywhere in Amperity and from sending that data to any downstream workflow.
	Configure the following settings, and then click “Save”. Compression The compression format to apply to the file. May be one of “GZIP”, “None”, “TAR”, “TGZ”, or “ZIP”. Escape character The escape character to use in the file output. Applies to CSV, TSV, PSV, and custom delimiter file types. When an escape character is not specified and the quote mode is “None” files may be sent with unescaped and unquoted data. When an escape character is not specified, you should select a non-“None” option as the quote mode. File format Required Configure Amperity to send Apache Parquet, CSV, JSON, NDJSON, PSV, or TSV files to any SFTP site. Some file formats allow a custom delimiter. Choose the “Custom delimiter” file format, and then add a single character to represent the custom delimiter. Apache Parquet files only The extension for Apache Parquet files may be excluded from the directory name. Filename template A filename template defines the naming pattern for files that are sent from Amperity. Specify the name of the file, and then use Jinja-style string formatting to append a date or timestamp to the filename. Header Enable to include header rows in output files. Line ending The line ending to use in file output. May be one of “LF” or “CRLF”. PGP public key The PGP public key that Amperity will use to encrypt files. Quote mode The quote mode to use within the file. May be one of “all fields”, “all non-NULL fields”, “fields with special characters only”, “all non-numeric fields” or “None”. Unescaped, unquoted files may occur when quote mode is set to “None” and an escape character is not specified. Remote folder Required The remote folder to which Amperity will send data. A “/” (forward slash) must be used as the first character for this value. For example: “/folder/name”. For example: “/folder/name”. If the path to the remote folder contains a space, use a backslash (\) character to escape that space. For example, a folder named “From ACME” should be entered as “From\ ACME”. Success file Enable to send a “.DONE” file when Amperity has finished sending data. If a downstream sensor is listening for files sent from Amperity, configure that sensor to listen for the presence of the “.DONE” file. Use Zip64? Enable to apply Zip64 data compression to very large files. Row Number Select to include a row number column in the output file. Applies to CSV, TSV, PSV, and custom delimiter file types. If Row Number is enabled you may use the Column name setting to specify the name of the row number column in the output file. The name of this column must be less than 1028 characters and may only contain numbers, letters, underscores, and hyphens. Default value: “row_number”.
	After this destination is configured: Use orchestrations to send query results Use orchestrations and campaigns to send audiences Use orchestrations and campaigns to send offline events

Workflow actions

A workflow will occasionally show an error that describes what prevented a workflow from completing successfully. These first appear as alerts in the notifications pane. The alert describes the error, and then links to the Workflows tab.

Open the Workflows page to review a list of workflow actions, choose an action to resolve the workflow error, and then follow the steps that are shown.

	You may receive a notifications error for a configured SFTP destination. This appears as an alert in the notifications pane on the Destinations tab. If you receive a notification error, review the details, and then click the View Workflow link to open this notification error in the Workflows page.
	On the Workflows page, review the individual steps to determine which step(s) have errors that require your attention, and then click Show Resolutions to review the list of workflow actions that were generated for this error.
	A list of individual workflow actions are shown. Review the list to identify which action you should take. Some workflow actions are common across workflows and will often be available, such as retrying a specific task within a workflow or restarting a workflow. These types of actions can often resolve an error. In certain cases, actions are specific and are shown when certain conditions exist in your tenant. These types of actions typically must be resolved and may require steps that must be done upstream or downstream from your Amperity workflow. Amperity provides a series of workflow actions that can help resolve specific issues that may arise with SFTP, including: Invalid credentials SFTP site unavailable
	Select a workflow action from the list of actions, and then review the steps for resolving that error. After you have completed the steps in the workflow action, click Continue to rerun the workflow.

Invalid credentials

The credentials that are defined in Amperity are invalid.

To resolve this error, verify that the credentials required by this workflow are valid.

1. Open the Credentials page.

2. Review the details for the credentials used with this workflow. Update the credentials for SFTP if required.

3. Return to the workflow action, and then click Resolve to retry this workflow.

SFTP site unavailable

The SFTP site at “sftp.socktown.com” is unavailable. This may be due to any of the following issues:

• A configuration error within Amperity.

• The SFTP server is unavailable or unresponsive.

• The SFTP server does not have enough storage space.

• Too many connection attempts were made to the SFTP server.

Note

Amperity will make multiple attempts to connect to an SFTP server before showing this workflow error.

To resolve this error, do any of the following.

1. Verify the configuration for “sftp.socktown.com”.

2. Verify that the SFTP server is available and responsive.

3. Verify the amount of storage space that is available on the SFTP server.

4. Decrease the number of couriers in Amperity that are configured to connect to “sftp.socktown.com”.

5. Return to the workflow action, and then click Resolve to retry this workflow.