Send data to any SFTP site

The SSH File Transfer Protocol (also Secure File Transfer Protocol, or SFTP) is a network protocol that provides file access, file transfer, and file management over any reliable data stream.

This topic describes the steps that are required to send files to SFTP from Amperity:

  1. Get details

  2. Add destination

  3. Add data template

Get details

SFTP requires the following configuration details:

Detail one.

The username.
Detail one.

The host public key.
Detail one.

The hostname, to which data will be sent.
Detail one.

For passphrase credentials, the passphrase; for private key credentials, the private key.
Detail one.

The remote folder to which Amperity will send data.
Detail one.

The public key to use for PGP encryption.

Included SFTP site

Every Amperity tenant includes an SFTP site with a hostname of <tenant>.sftp.amperity.com. For example, if your company name is ACME, then your tenant’s SFTP hostname is acme.sftp.amperity.com. (The hostname is always all lowercase.)

Note

The SFTP site is provisioned by an Amperity administrator after the initial tenant creation. As such, the SFTP site is not immediately available, but this is not a long process. When the SFTP site is ready, Amperity will use SnapPass to send you the connection details. If you wish to use RSA key-based authentication, please provide the public key to your Amperity administrator when requesting SFTP access.

Once this site is provisioned, you may configure the SFTP site as an SFTP source or an SFTP destination to support any desired workflow. External customer processes can be configured to connect to the site using SFTP, after which they can add data to or pick up data from the site to support any external workflow.

Caution

The SFTP server has a 30-day limit on data and after 30 days, data may be moved automatically to an archive location. This location is still accessible to Amperity in case it becomes necessary to reuse it.

Important

The hostname for the SFTP site is always <tenant-name>.sftp.amperity.com. Some older tenants may still be using the legacy address sftp.amperity.com, if so, please contact your Amperity administrator about migrating.

Filedrop requirements

An SFTP location requires the following:

Add destination

Configure Amperity to send files directly to SFTP.

To add a destination to any SFTP site

Step 1.

Open the Destinations tab to configure a destination for SFTP. Click the Add Destination button to open the Destination dialog box.

Name, description, choose plugin.

Enter a name for the destination and provide a description. For example: “SFTP” and “This sends files to SFTP”.

From the Plugin drop-down, start typing “sftp” to filter the list, and then select SFTP.
Step 2.

Credentials allow Amperity to connect to SFTP.

The credential type is set automatically to passphrase or private-key. You may use an existing credential or you may add a new one.

Choose an existing credential or add credential.

Select an existing credential from the Credential drop-down.

– or –

Select Create a new credential from the Credential drop-down. This opens the Credential dialog box.

Enter the name for the credential, and then add a description. Select hostname from the HOSTNAME drop-down list. Enter the username and passphrase.

Update the settings for the SFTP destination.

When finished, click Save.
Step 3.

Each destination has settings that define how Amperity will deliver data to SFTP. These settings are listed under the Settings section of the Destination dialog box.

Settings for SFTP destinations.

Complete the following SFTP Settings:

  • The path to the Remote folder.

    Tip

    If the path to the remote folder contains a space, use a backslash () character to escape that space. For example, a folder named “From ACME” should be entered as “From\ ACME”.

  • The File format. Select the file format – File format options include Apache Parquet (recommended when supported by the downstream workflow), CSV, TSV, and PSV – from the drop-down list.

  • Optional. The Escape character that is required by SFTP.

    Note

    If an escape character is not specified and quote mode is set to “None” this may result in unescaped, unquoted files. When an escape character is not specified, you should select a non-“None” option from the Quote Mode setting.

  • Optional. The Compression format. Encoding method options include “Tar”, “Tgz”, “Zip”, “GZip”, and “None”.

  • Optional. The PGP public key that is used to encrypt files that are sent to SFTP.

  • Optional. The Quote mode that should be used within the file. From the drop-down, select one of “all fields”, “all non-NULL fields”, “fields with special characters only”, “all non-numeric fields” or “None”.

    Note

    If the quote mode is set to “None” and the Escape Character setting is empty this may result in unescaped, unquoted files. When quote mode is not set to “None”, you should specify an escape character.

  • Optional. Select Include success file upon completion to add a .DONE file that indicates when an orchestration has finished sending data.

    Tip

    If a downstream sensor is listening for files sent from Amperity, configure that sensor to listen for the presence of the .DONE file.

  • Optional. Select Include header row in output files if headers should be included in the output.
Step 4.

Business users are assigned to the Amp360 User and/or AmpIQ User policies. (Amp360 User allows access to queries and orchestrations and AmpIQ User allows access to segments and campaigns.) A business user cannot select a destination that is not visible to them.

Business users – including users assigned to the DataGrid Operator policy – may have restricted access to PII.

What is restricted access to PII?

Restricted PII access is enabled when the Restrict PII Access policy option that prevents users who are assigned to that option from viewing data that is marked as PII anywhere in Amperity and from sending that data to any downstream workflow.

You can make this destination visible to orchestrations and allow users with restricted access to PII to use this destination by enabling one (or both) of the following options:

Allow business users access to this destination.

Note

To allow business users to use this destination with campaigns, you must enable the Make available to campaigns option within the data template. This allows users to send campaign results from Amperity to SFTP.

The other two settings may be configured within the data template instead of the destination.
Step 5.

Review all settings, and then click Save.

Save the destination.

Important

You must configure a data template for this destination before you can send data to SFTP.

Add data template

A data template defines how columns in Amperity data structures are sent to downstream workflows. A data template is part of the configuration for sending query and segment results from Amperity to an external location.

To add a data template

Step 1.

From the Destinations tab, open the menu for a destination that is configured for SFTP, and then select Add data template.

This opens the Add Data Template dialog box.

Step 1

Enter the name of the data template and a description. For example: “SFTP” and “Send files to SFTP.”.
Step 2.

Verify business user access to queries and orchestrations and access to segments and campaigns.

A business user may also have restricted access to PII, which prevents them from viewing and sending customer profile data.

Step 2.

If business user access was not configured as part of the destination, you may configure access from the data template.

Important

To allow business users to use this destination with campaigns, you must enable the Make available to campaigns option. This allows users to send campaign results from Amperity to SFTP.

If you enable this option, the data extension settings require using campaign name and group name template variables to associate the name of the data extension to your campaign.
Step 3.

Verify all configuration settings.

Verify settings for the data template.

Note

When the settings required by the SFTP site were are not configured as part of the destination, you must configure them as part of the data template before making this destination available to campaigns.
Step 4.

Review all settings, and then click Save.

Save the data template.

After you have saved the data template, and depending on how you configured it, business users can send query results and/or send campaigns to SFTP.

Workflow actions

A workflow will occasionally show an error that describes what prevented a workflow from completing successfully. These first appear as alerts in the notifications pane. The alert describes the error, and then links to the Workflows tab.

Open the Workflows tab to review a list of workflow actions, choose an action to resolve the workflow error, and then follow the steps that are shown.

Step one.

You may receive a notifications error for a configured SFTP destination. This appears as an alert in the notifications pane on the Destinations tab.

Review a notifications error.

If you receive a notification error, review the details, and then click the View Workflow link to open this notification error in the Workflows page.
Step two.

On the Workflows page, review the individual steps to determine which step(s) have errors that require your attention, and then click Show Resolutions to review the list of workflow actions that were generated for this error.

The workflow tab, showing a workflow with errors.
Step three.

A list of individual workflow actions are shown. Review the list to identify which action you should take.

Choose a workflow action from the list of actions.

Some workflow actions are common across workflows and will often be available, such as retrying a specific task within a workflow or restarting a workflow. These types of actions can often resolve an error.

In certain cases, actions are specific and are shown when certain conditions exist in your tenant. These types of actions typically must be resolved and may require steps that must be done upstream or downstream from your Amperity workflow.

Amperity provides a series of workflow actions that can help resolve specific issues that may arise with SFTP, including:
Step four.

Select a workflow action from the list of actions, and then review the steps for resolving that error.

Choose a workflow action from the list of actions.

After you have completed the steps in the workflow action, click Continue to re-run the workflow.

Invalid credentials

The credentials that are defined in Amperity are invalid.

To resolve this error, verify that the credentials required by this workflow are valid.

  1. Open the Credentials page.

  2. Review the details for the credentials used with this workflow. Update the credentials for SFTP if required.

  3. Return to the workflow action, and then click Resolve to retry this workflow.

SFTP site unavailable

The SFTP site at hostname is unavailable. This may be due to any of the following issues:

  • A configuration error within Amperity.

  • The SFTP server is unavailable or unresponsive.

  • The SFTP server does not have enough storage space.

  • Too many connection attempts were made to the SFTP server.

Note

Amperity will make multiple attempts to connect to an SFTP server before showing this workflow error.

To resolve this error, do any of the following.

  1. Verify the configuration for hostname.

  2. Verify that the SFTP server is available and responsive.

  3. Verify the amount of storage space that is available on the SFTP server.

  4. Decrease the number of couriers in Amperity that are configured to connect to hostname.

  5. Return to the workflow action, and then click Resolve to retry this workflow.