Configure destination for Amazon S3¶
Note
This topic contains information about configuring a destination that sends query results to Amazon S3 using orchestrations. To configure a destination that sends audiences to Amazon S3 using campaigns see this topic .
Amazon Simple Storage Service–Amazon S3–can store data files of any size for any file format supported by Amperity.
Configure Amperity to send Apache Parquet (recommended), CSV, JSON, NDJSON, PSV, or TSV files to any Amazon S3 bucket.
Get details¶
Review the following details before configuring credentials for Amazon S3 and before configuring Amperity to send Apache Parquet (recommended), CSV, JSON, NDJSON, PSV, or TSV files to any Amazon S3 bucket.
|
Amazon S3 bucket details You will need to know the following details about the Amazon S3 bucket to which Amperity will send data.
|
|
|
Credential types and settings Amperity supports the following credential types for Amazon S3:
|
|
|
Required configuration settings
Note All other Amperity file format settings for Amazon S3 are optional. |
Configure credentials¶
Configure credentials for Amazon S3 before adding a destination.
Amperity supports the following credential types for Amazon S3:
An individual with access to Amazon S3 should use SnapPass to securely share “iam-credential” or “iam-role-to-role” details with the individual who configures Amperity.
IAM role-to-role¶
Amperity prefers to pull data from and send data to customer-managed cloud storage.
Amperity recommends using cross-account role assumption to manage access to Amazon S3. This ensures that your brand manages the security policies that control access to your data.
Using cross-account role assumption helps ensures that customers can:
Directly manage the IAM policies that control access to data
Directly manage the files that are available within the Amazon S3 bucket
Access without requiring involvement by Amperity
Revoke access at any time in either Amazon AWS account, after which data sharing ends immediately
Directly troubleshoot incomplete or missing files
Note
After setting up cross-account role assumption, make a list of files by filename and file type available to feed creation, along with any sample files. Add these files directly to the shared location after configuring cross-account role assumption.
Using an Amazon AWS Access Point?
Yes, but with the following limitations:
Amperity accesses files located in a customer-managed Amazon S3 bucket
Use a credential-free role-to-role access pattern
Do not restrict traffic to VPC-only
To configure an S3 bucket for cross-account role assumption
The following steps describe how to configure Amperity to use cross-account role assumption to pull data from or push data to a customer-managed Amazon S3 bucket.
Important
These steps require users with administrative access to configure changes to customer-managed Amazon AWS accounts.
|
From the Settings page, select the Credentials tab, and then click the Add credential button. |
|
In the Credentials settings dialog box, do the following: From the Plugin dropdown, select Amazon S3. Assign the credential a name and description that ensures other users of Amperity can recognize when to use this destination. From the Credential type dropdown, select iam-role-to-role. |
|
The settings that are available for a credential are determined by the credential type. For the iam-role-to-role credential type, configure the following settings, and then click Save. 
You must give the values for the Target Role ARN and S3 Bucket Name fields. Enter the target role ARN for the IAM role that Amperity uses to access the customer-managed Amazon S3 bucket, and then enter the name of the Amazon S3 bucket.
|
|
Review the following sample policy, and then add a policy to the customer-managed Amazon S3 bucket. Add this policy as a trusted policy to the IAM role used to manage access to the customer-managed Amazon S3 bucket. The policy for the customer-managed Amazon S3 bucket is unique, but is similar to: {
"Statement": [
{
"Sid": "AllowAmperityAccess",
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::account:role/resource"
},
"Action": "sts:AssumeRole",
"Condition": {
"StringEquals": {
"sts:ExternalId": "01234567890123456789"
}
}
}
]
}
The value for the role ARN is similar to: arn:aws:iam::1234567890:role/prod/amperity-plugin
|
|
Click Continue to test the configuration and validate the connection to the customer-managed Amazon S3 bucket, after which you can continue the steps for adding a courier. |
IAM credentials¶
IAM credentials require an access key, which is in two parts:
An access key ID
A secret access key
Both parts are required to authenticate requests to Amazon AWS resources.
To configure an S3 bucket for IAM credentials
|
|
From the Settings page, select the Credentials tab, and then click the Add credential button. |
|
|
In the Credentials settings dialog box, do the following: From the Plugin dropdown, select Amazon S3. Assign the credential a name and description that ensures other users of Amperity can recognize when to use this destination. From the Credential type dropdown, select iam-credential. |
|
|
The settings that are available for a credential are determined by the credential type. For the iam-credential credential type, configure the following settings, and then click Save. 
|
Add destination¶
Use a sandbox to configure a destination for Amazon S3. Before promoting your changes, send a test audience, and then verify the results in Amazon S3. After verifying the end-to-end workflow, push the destination from the sandbox to production.
To add a destination for Amazon S3
|
|
Open the Destinations page, and then click the Add destination button. 
To configure a destination for Amazon S3, do one of the following:
|
|
|
Select the credential for Amazon S3 from the Credential dropdown, and then click Continue. Tip Click the “Test connection” link on the “Configure destination” page to verify that Amperity can connect to Amazon S3. |
|
|
In the “Destination settings” dialog box, assign the destination a name and description that ensures other users of Amperity can recognize when to use this destination. Configure business user access By default a destination is available to all users who have permission to view personally identifiable information (PII). Enable the Admin only checkbox to restrict access to only users assigned to the Datagrid Operator and Datagrid Administrator policies. Enable the PII setting checkbox to allow limited access to PII for this destination. Use the Restrict PII access policy option to prevent users from viewing data marked as PII anywhere in Amperity and from sending data to downstream workflows. |
|
|
Configure the following settings, and then click “Save”.
|
|
|
After configuring this destination users may use:
|
Workflow actions¶
A workflow will occasionally show an error that describes what prevented a workflow from completing successfully. These first appear as alerts in the notifications pane. The alert describes the error, and then links to the Workflows tab.
Open the Workflows page to review a list of workflow actions, choose an action to resolve the workflow error, and then follow the steps that are shown.
|
|
You may receive a notifications error for a configured Amazon S3 destination. This appears as an alert in the notifications pane on the Destinations tab. 
If you receive a notification error, review the details, and then click the View Workflow link to open this notification error in the Workflows page. |
|
|
On the Workflows page, review the individual steps to determine which step(s) have errors that require your attention, and then click Show Resolutions to review the list of workflow actions that were generated for this error. 
|
|
|
A list of individual workflow actions are shown. Review the list to identify which action you should take. 
Some workflow actions are common across workflows and will often be available, such as retrying a specific task within a workflow or restarting a workflow. These types of actions can often resolve an error. In certain cases, actions are specific and are shown when certain conditions exist in your tenant. These types of actions typically must be resolved and may require steps that must be done upstream or downstream from your Amperity workflow. Amperity provides a series of workflow actions that can help resolve specific issues that may arise with Amazon S3, including: |
|
|
Select a workflow action from the list of actions, and then review the steps for resolving that error. 
After you have completed the steps in the workflow action, click Continue to rerun the workflow. |
Invalid bucket name¶
The name of the Amazon S3 bucket to which Amperity pushes data must be correctly specified in the configuration for the destination in the Destinations page.
To resolve this error, do the following.
Open the AWS management console and verify the name of the Amazon S3 bucket.
Open the Destinations page in Amperity, and then open the destination that is associated with this workflow.
Update the destination for the correct Amazon S3 bucket name.
Return to the workflow action, and then click Resolve to retry.
Invalid credentials¶
The credentials that are defined in Amperity are invalid.
To resolve this error, verify that the credentials required by this workflow are valid.
Open the Credentials page.
Review the details for the credentials used with this workflow. Update the credentials for Amazon S3 if required.
Return to the workflow action, and then click Resolve to retry this workflow.