Configure Amperity Bridge for Google BigQuery

Amperity Bridge for Google BigQuery is a first-class integration enables inbound data access to Amperity from Google BigQuery without copying data or scheduling ETL workloads.

What is BigQuery Sharing?

BigQuery Sharing publishes and shares datasets between organizations. Create an exchange, and then invite organizations to publish or subscribe to data in the exchange.

Tip

Amperity Bridge shares data directly with Google BigQuery. A bridge does not require a courier or a database export to be configured. A bridge can be configured to run automatically as part of any scheduled workflow.

From Google BigQuery

A sync from Google BigQuery to Amperity requires configuration steps to be made in both Amperity and Google BigQuery.

1. Get details

2. Configure Google BigQuery

3. Add inbound bridge

Get details

Before you can create inbound sharing between Google BigQuery and Amperity you will need to collect the following information.

	The individual who configures Amperity to share data with Google BigQuery must be assigned the Datagrid Administrator policy in Amperity or have the ability to configure a bridge for Google BigQuery in a sandbox.
	A Google Cloud account with the BigQuery API enabled .
	The user who creates the data exchange and listing in Google BigQuery must have permission to access to the Google Analytics Hub API. The following permissions are required: serviceUsage.services.get serviceUsage.services.list serviceUsage.services.enable Note roles/serviceusage.serviceUsageAdmin is a predefined IAM role that includes the permissions required to access the Analytics Hub API.
	The user who creates the data exchange and listing in Google BigQuery must have permission to create listings. The following permissions are required: bigquery.datasets.get bigquery.datasets.update Note roles/bigquery.dataOwner and roles/bigquery.admin are predefined IAM roles that include the permissions required to create listings in Google BigQuery.
	The following values are required to connect Amperity to a Google BigQuery data listing: Share link (for the data exchange) This value is available after creating the data exchange and data listing in Google BigQuery. Copy this value. It is required when adding the inbound share. https://console.cloud.google.com/bigquery/analytics-hub/\ exchanges/projects/123456789012/locations/<location>/\ dataExchanges/<exchange>/listings/<listing> Subscriber Amperity generates an account identifer to use for activating sharing in Google BigQuery. Copy this value and use it to configure permissions to the data exchange. Add a principal using this value and assign it to the Analytics Hub Subscriber role. The account identifier generated by Amperity is similar to: amperity-bridge-<bridge-id>@amperity.iam.gserviceaccount.com

Configure Google BigQuery

To configure Google BigQuery to sync data with Amperity do the following:

1. Create data exchange

2. Create data listing

3. Configure subscriber and share link

Create data exchange

A data exchange in Google BigQuery is private by default. Only users and groups that have access to the data exchange can view or subscribe to data listings within the data exchange.

Create a data exchange in Google BigQuery and then allow Amperity access to that data exchange.

To create a Google BigQuery data exchange

	Open the BigQuery Console to the Sharing page. On the Sharing page click Create exchange.
	In the Exchange configuration pane, do the following: Select the project and region for the data exchange. Enter a display name. Add your Amperity account locator. Note If you are configuring a new bridge, leave the Amperity account locator field blank. This will be added after the account identifier is created.
	Click Create exchange. Important Do not configure permissions for the data exchange. The value for the Add principal field must be the value that is generated by Amperity. Click Skip.

Create data listing

A data listing in Google BigQuery is a reference to a shared dataset that exists in a data exchange. A data listing is private by default.

A data listing within a private data exchange may be shared with Amperity.

To create a Google BigQuery data listing

	On the Sharing page click the name of the exchange. On the Listings tab for the exchange click Create listing.
	From the Resource type dropdown select “BigQuery Dataset”, and then select a dataset. Note Configuring data egress controls is optional.
	Configure a series of details for the data listing. Click Next. Required. Add a display name. Optional. Configure listing details, discoverability, and email logging. Click Next Optional. Configure the contact information for primary, provider, and publisher contacts.
	Click Publish. On the Listings tab, in the row for the listing that will be shared with Amperity, from the ellipses menu select Copy share link. Amperity generates a URL similar to: https://console.cloud.google.com/bigquery/analytics-hub/exchanges/projects/123456789012/locations/<location>/dataExchanges/<exchange>/listings/<listing> Configure this identifier in Google BigQuery as the subscriber for the data listing.

Configure subscriber and share link

A subscriber is allowed to view and access a data listing . Configure Amperity as the subscriber to the Google BigQuery data listing.

A share link allows a subscriber to connect to a data listing. Copy the share link for the data listing in Google BigQuery and configure that link in Amperity.

To configure a Google BigQuery subscriber and share link

	On the Sharing page in the row for the exchange that will share data with Amperity select Set permissions from the ellipses menu. Click Add principal Paste the Amperity account identifier into the New principal field and set the role to be Analytics Hub Subscriber . Click Save This returns you to the Sharing page.
	On the Sharing page in the row for the exchange that will share data with Amperity select Copy share link from the ellipses menu. Important This value required when adding the inbound share.

Add inbound bridge

Configure an inbound bridge to sync data from Google BigQuery to Amperity.

To add an inbound bridge for Google BigQuery

	Open the Sources page. Under Inbound shares click Add bridge. Choose Google BigQuery. This opens the Add bridge dialog box. Add a name and description for the bridge or select an existing bridge, and then click Confirm.
	Add the share link that was copied from the Google BigQuery data exchange. Amperity generates the subscriber link after the share link has been configured. The share link is a URL similar to: https://console.cloud.google.com/bigquery/analytics-hub/exchanges/projects/123456789012/locations/<location>/dataExchanges/<exchange>/listings/<listing> Configure this identifier in Google BigQuery as the subscriber for the data listing.
	Use the Select tables dialog box to select any combination of schemas and tables to be synced to Amperity. If you select a schema, all tables in that schema will be synced. Any new tables added later will need to be manually added to the sync. When finished, click Next. This will open the Domain table mapping dialog box.
	Map the tables that are synced from Google BigQuery to domain tables in Amperity. Tables that are synced with Amperity are added as domain tables. The names of synced tables must be unique among all domain tables. Primary keys are not assigned. Semantic tags are not applied. Tip Use a custom domain table to assign primary keys, apply semantic tags, and shape data within synced tables to support any of your Amperity workflows. When finished, click Save and sync. This will start a workflow that synchronizes data from Google BigQuery to Amperity and will create the mapped domain table names. You can manually sync tables that are synced with Amperity using the Sync option from the Actions menu for the inbound bridge.