Pull from HubSpot Activity

HubSpot is a CRM platform built around dedicated hubs for marketing, sales, services, content management, and operations.

HubSpot Activity pulls contacts, CRM activity, deal stage history, web events, marketing email events, and form submissions from HubSpot to Amperity, one record for each call, meeting, email event, page view, form submission, or deal stage change. Use it to build segments on how contacts engaged with your business over time, rather than on summary fields stored on the HubSpot contact.

Choose which data types to pull. Each courier run lands one NDJSON file for each selected data type. A run for a time period reads only the records that changed or happened during that period, and a run for all available data reads everything that HubSpot returns.

Note

HubSpot Activity is separate from the HubSpot source, which pulls HubSpot tables that Fivetran syncs to Snowflake, and from the HubSpot destination, which sends audiences to HubSpot lists. HubSpot Activity connects to HubSpot directly through its own HubSpot app and its own credential type, hubspot-activity. A credential for the HubSpot destination cannot be used with HubSpot Activity.

Beta

The HubSpot Activity connector is in beta. Contact your Amperity representative to learn more.

The steps that are required to pull contacts, CRM activity, deal stage history, web events, marketing email events, and form submissions to Amperity from HubSpot Activity:

1. Get details

2. Add courier

3. Get sample files

4. Add feeds

5. Add load operations

6. Run courier

7. Add to courier group

Get details

HubSpot Activity requires the following configuration details:

1. A user who can sign in to the HubSpot account that Amperity pulls from, and who can approve the authorization.

   HubSpot Activity uses OAuth, so there is no key or password to request. Amperity generates an authorization URL. A user who can sign in to your HubSpot account opens it, selects the HubSpot account, and approves the permissions that Amperity requests. The refresh token that authorizes Amperity is then stored on the credential.

   The authorization is for the Amperity HubSpot Activity app. That app is separate from the one that the HubSpot destination uses, so a credential that was created for the HubSpot destination cannot be used with HubSpot Activity.

   Every permission that Amperity requests is read-only. Reading account details and contacts is required. Reading deals, forms, logged email content, marketing email, and website analytics is optional. HubSpot leaves an optional permission off the approval screen when the selected account does not have access to that tool, so an account on any HubSpot subscription can complete the authorization. Selecting a data type in Amperity does not add a permission. A data type that needs a permission the credential was not granted fails until HubSpot is authorized again and that permission is approved.

2. The Data types to pull. Select at least one. Some data types need an optional permission that only some HubSpot subscriptions grant. Review Data types before you choose.

Note

HubSpot can show an “unverified app” warning on its approval screen for the Amperity HubSpot Activity app. Acknowledge the warning to continue. The authorization completes normally.

Add courier

A courier brings data from an external system to Amperity.

To add a courier

1. From the Sources page, click Add Courier. The Add Courier page opens.

2. Find, and then click the icon for HubSpot Activity. The Add Courier page opens.

3. Enter the name of the courier. For example: “HubSpot Activity”.

   From the Credential field, select an existing credential or select Create a new credential.

   To add a credential, enter a name and a description, and then click Generate authorization URL. Copy the URL and provide it to the HubSpot user who approves the authorization. The URL expires after 2 weeks. After that user completes the authorization, the credential is added to the Credentials page and can be selected.

   When finished click Continue.

4. Under Data types, select the data types to pull.

5. Leave Load Operations empty. You add them after the feeds are configured.

6. Click Save.

Note

Testing the connection to HubSpot Activity verifies that the credential can read from your HubSpot account, and that HubSpot granted the permission each selected data type needs. A data type whose permission is missing is named in the error, along with the permission. Every courier run makes the same permission check before it reads any data, so a missing permission fails the run at the start instead of partway through.

Get sample files

Every HubSpot Activity file that is pulled to Amperity must be configured as a feed. Before you can configure each feed you need to know the schema of that file. Run the courier without load operations to bring sample files from HubSpot Activity to Amperity, and then use each of those files to configure a feed.

Each data type lands as a file named for that data type. For example, contacts land as hubspot_contacts.ndjson and deal stage history lands as hubspot_deal_stage_history.ndjson. Data types lists every file name.

A file lands for every selected data type, even when that data type has no records for the time period of the run. To get a sample file that has records, run the courier for a longer time period or for all available data.

To get sample files

1. From the Sources tab, open the menu for a courier configured for HubSpot Activity with empty load operations, and then select Run. The Run Courier dialog box opens.

2. Select a time period that includes activity for every selected data type, or select all available data. A run for all available data reads every record in your HubSpot account and can take a long time for a large account.

3. Click Run.

   Important

   The courier run fails, but this process will successfully return a list of files from HubSpot Activity.

   These files will be available for selection as an existing source from the Add Feed dialog box.

4. Wait for the notification for this courier run to return an error similar to:

   Error running load-operations task
   Cannot find required feeds: "df-xxxxxx"
   

Add feeds

A feed defines how to load data into a domain table, including specifying required columns and columns with semantic tags for customer profile (PII) or transactions data.

Note

A feed must be added for each file that is pulled from HubSpot Activity, including all files that contain customer records and interaction records, along with any other files that is used to support downstream workflows.

Choose a primary key for each feed that identifies one record, so that a record that is read again by a later run replaces the earlier copy. Data types lists the fields that identify a record for each data type.

To add a feed

1. From the Sources tab, click Add Feed. This opens the Add Feed dialog box.

2. Under Data Source, select Create new source, and then enter “HubSpot Activity”.

3. Enter the name of the feed in Feed Name. For example: “Contacts”.

   Tip

   The name of the domain table will be “<data-source-name>:<feed-name>”. For example: “HubSpot Activity:Contacts”.

4. Under Sample File, select Select existing file, and then choose from the list of files. For example: “hubspot_contacts.ndjson”.

   Tip

   The list of files that is available from this dropdown menu is sorted from newest to oldest.

5. Select Load sample file on feed activation.

6. Click Continue. This opens the Feed Editor page.

7. Select the primary key.

8. Apply semantic tags to customer records and interaction records, as appropriate.

9. Under Last updated field, specify which field best describes when records in the table were last updated.

   Tip

   Choose Generate an “updated” field to have Amperity generate this field. This is the recommended option unless there is a field already in the table that reliably provides this data.

10. For feeds with customer records (PII data), select Make available to Stitch.

11. Click Activate. Wait for the feed to finish loading data to the domain table, and then review the sample data for that domain table from the Data Explorer.

Add load operations

After the feeds are activated and domain tables are available, add the load operations to the courier used for HubSpot Activity.

Example load operations

Load operations must specify each file that is pulled to Amperity from HubSpot Activity.

Refer to each file by the ID of its data type, not by the name of the file: calls, communications, contacts, deal-stage-history, email-events, emails, form-submissions, meetings, notes, tasks, or web-events. Add a load operation for every data type that is selected for the courier.

Do not add a truncate operation. A run for a time period lands only the records that changed or happened during that period, so a truncate would remove every record that was loaded by earlier runs.

For example:

{
  "df-A1B2C3": [
    {
      "type": "load",
      "file": "contacts"
    }
  ],
  "df-D4E5F6": [
    {
      "type": "load",
      "file": "calls"
    }
  ],
  "df-G7H8I9": [
    {
      "type": "load",
      "file": "deal-stage-history"
    }
  ]
}

To add load operations

1. From the Sources tab, open the menu for the courier that was configured for HubSpot Activity, and then select Edit. The Edit Courier dialog box opens.

2. Edit the load operations for each of the feeds that were configured for HubSpot Activity so they have the correct feed ID.

3. Click Save.

Run courier manually

Run the courier again. This time, because the load operations are present and the feeds are configured, the courier will pull data from HubSpot Activity.

Important

Run the courier for all available data before you add it to a courier group. A run for a time period reads only the records that changed or happened during that period, so history from before the first scheduled run is loaded only by a run for all available data.

To run the courier manually

1. From the Sources tab, open the menu for the courier with updated load operations that is configured for HubSpot Activity, and then select Run. The Run Courier dialog box opens.

2. Select the load option, either for a specific time period or all available data. Actual data will be loaded to a domain table because the feed is configured.

3. Click Run.

   This time the notification will return a message similar to:

   Completed in 5 minutes 12 seconds
   

Add to courier group

A courier group is a list of one or more couriers that run as a group. A courier group can act as a constraint on downstream workflows and can run automatically as part of a scheduled workflow.

To add the courier to a courier group

1. From the Sources tab, click Add Courier Group. This opens the Create Courier Group dialog box.

2. Enter the name of the courier. For example: “HubSpot Activity”.

3. Add a cron string to the Schedule field to define a schedule for the orchestration group.

   A schedule defines the frequency at which a courier group runs. All couriers in the same courier group run as a unit and all tasks must complete before a downstream process starts. Define a schedule using cron.

   Cron syntax specifies the fixed time, date, or interval at which cron runs. Each line represents a job. 30 8 * * * represents “run at 8:30 AM every day” and 30 8 * * 0 represents “run at 8:30 AM every Sunday”.

   For example:

   ┌───────── minute (0 - 59)
   │ ┌─────────── hour (0 - 23)
   │ │ ┌───────────── day of the month (1 - 31)
   │ │ │ ┌────────────── month (1 - 12)
   │ │ │ │ ┌─────────────── day of the week (0 - 6) (Sunday to Saturday)
   │ │ │ │ │
   │ │ │ │ │
   │ │ │ │ │
   * * * * * command to execute
   

   Amperity validates the cron syntax and shows you the results. You may also use crontab guru to validate cron syntax.

4. Set Status to Enabled.

5. Specify a time zone.

   A courier group schedule is associated with a time zone. The time zone determines the point at which a courier group’s scheduled start time begins. A time zone should be aligned with the time zone of system from which the data is being pulled.

   Use the Use this time zone for file date ranges checkbox to use the selected time zone to look for files. If unchecked, the courier group uses the current time in UTC to look for files to pick up.

   Note

   The time zone that is chosen for an courier group schedule should consider every downstream business processes that requires the data and also the time zones in which the consumers of that data will operate.

6. Add at least one courier to the courier group. Select the name of the courier from the Courier dropdown. Click + Add Courier to add more couriers.

7. Click Add a courier group constraint, and then select a courier group from the dropdown list.

   A wait time is a constraint placed on a courier group that defines an extended time window for data to be made available at the source location.

   Important

   A wait time is not required for a bridge.

   A courier group typically runs on an automated schedule that expects customer data to be available at the source location within a defined time window. However, in some cases, the customer data may be delayed and is not made available within that time window.

8. For each courier group constraint, apply any offsets.

   A courier can be configured to look for files within range of time that is older than the scheduled time. The scheduled time is in Coordinated Universal Time (UTC), unless the “Use this time zone for file date ranges” checkbox is enabled for the courier group.

   This range is typically 24 hours, but may be configured for longer ranges. For example, it is possible for a data file to be generated with a correct file name and datestamp appended to it, but for that datestamp to represent the previous day because of how an upstream workflow is configured. A wait time helps ensure that the data at the source location is recognized correctly by the courier.

   Warning

   This range of time may affect couriers in a courier group whether or not they run on a schedule. A manually run courier group may not take its schedule into consideration when determining the date range. Only the provided input days to load data from are used as inputs.

9. Click Save.

Data types

Each data type is read from its own HubSpot API, needs its own HubSpot permission, and links back to a contact in its own way. The permission is shown by the name that HubSpot gives it, which is also the name that appears in a missing permission error.

Data type	Each row is	HubSpot permission	Links to a contact by
Contacts	One HubSpot contact. Every other data type links back to this one.	crm.objects.contacts.read	id, which is the HubSpot contact ID
Calls, Meetings, Emails, Notes, Tasks, Communications	One activity that is logged on a HubSpot record. Emails are one-to-one emails logged in the HubSpot CRM, not marketing emails.	crm.objects.contacts.read	contact_ids, a list of the IDs of the associated contacts
Deal stage history	One change of a deal to a new pipeline stage.	crm.objects.deals.read	contact_ids, a list of the IDs of the deal’s associated contacts
Web events	One page view, form submission, or custom behavioral event that HubSpot tracked.	business-intelligence	contact_id
Email events	One event for a HubSpot marketing email, such as a send, delivery, open, click, bounce, or unsubscribe.	content	recipient, the recipient’s email address. There is no contact ID.
Form submissions	One submission of a HubSpot form.	forms	email, the email address that was submitted. There is no contact ID.

Email events need a HubSpot Marketing Hub or Content Hub subscription at the Professional or Enterprise tier. Web events need a HubSpot Professional or Enterprise subscription. The remaining data types are available on every HubSpot subscription.

Fields in each file

Field names are the names that HubSpot uses. Values, including timestamps, keep the format that HubSpot returns them in, which differs between data types. A field that HubSpot returns without a value is left out of the record, and a HubSpot property that is not defined in your account is left out of every record.

Data type	File	Fields
Contacts	hubspot_contacts.ndjson	id, created_at, updated_at, archived, hs_object_id, email, hs_additional_emails, firstname, lastname, phone, createdate, and lastmodifieddate. The primary key is id.
Calls	hubspot_calls.ndjson	The activity fields, plus hs_call_title, hs_call_direction, hs_call_disposition, hs_call_duration, and hs_call_status.
Meetings	hubspot_meetings.ndjson	The activity fields, plus hs_meeting_title, hs_meeting_start_time, hs_meeting_end_time, hs_meeting_outcome, and hs_meeting_location.
Emails	hubspot_emails.ndjson	The activity fields, plus hs_email_direction, hs_email_status, and hs_email_subject. The body of the email is not pulled.
Notes	hubspot_notes.ndjson	The activity fields only. The text of the note is not pulled.
Tasks	hubspot_tasks.ndjson	The activity fields, plus hs_task_subject, hs_task_status, hs_task_priority, and hs_task_type.
Communications	hubspot_communications.ndjson	The activity fields, plus hs_communication_channel_type and hs_communication_logged_from.
Deal stage history	hubspot_deal_stage_history.ndjson	deal_id, dealname, pipeline, dealstage (the stage that the deal moved to), changed_at, source_type, source_id, updated_by_user_id, and contact_ids. No single field identifies a row. Use deal_id and changed_at together as the primary key.
Web events	hubspot_web_events.ndjson	id, event_type, occurred_at, object_type, and contact_id, plus every property that HubSpot records for the event, such as the page address and title for a page view. contact_id holds the ID of the object that HubSpot recorded the event on, whatever object_type is. The primary key is id.
Email events	hubspot_email_events.ndjson	Every field that HubSpot returns for the event, such as id, created, type, and recipient. A field that HubSpot returns as a nested object is split into one field for each of its values, named <object>_<field>. For example: browser_name or location_city. HubSpot identifies an email event by id and created together. Use both as the primary key.
Form submissions	hubspot_form_submissions.ndjson	form_id, form_name, conversion_id, submitted_at, page_url, email, and values. values holds every field that was submitted on the form, by field name. submitted_at is in milliseconds since the Unix epoch.

The activity fields are id, created_at, updated_at, archived, hs_object_id, hs_timestamp (when the activity happened), hs_createdate, hs_lastmodifieddate, hubspot_owner_id, and contact_ids. The primary key for each activity data type is id.

contact_ids is a list, and values is a nested object. Review how these fields appear in a sample file before you configure the feed.

How data is pulled

A run for a time period reads what changed or happened in that period. Each data type is filtered on its own timestamp:

• Contacts and the activity data types are filtered on when the record was last modified.

• Deal stage history finds the deals that were modified during the time period, and then keeps only the stage changes that happened during that period. A run for all available data lands every stage change that HubSpot keeps for every deal.

• Web events are filtered on when the event happened, starting one day before the beginning of the time period.

• Email events are filtered on when the event was created.

• Form submissions are filtered on when the form was submitted.

The start of a time period is included and the end is not, so a record on the boundary between two consecutive runs lands once.

Web events overlap by one day. HubSpot can process a web event some time after it happened, so each run for a time period reads web events from one day before its start. A web event can therefore land in two consecutive runs. Use id as the primary key of the web events feed so the second copy replaces the first.

Recent changes can be missed. HubSpot search results can lag behind changes made in HubSpot by a few moments. A contact or activity that changes just before the end of a time period can be missed by that run, and the next run does not read it, because its time period starts where the previous one ended. A run for all available data reads every record, including any that were missed.

Deal stage changes can be missed by a run for a time period. Deal stage history finds deals by when they were last modified. When a deal changes stage during the time period and is modified again after the period ends but before the run reads it, that run does not find the deal. The next run finds it, but keeps only the stage changes in its own time period, so that stage change does not land. This matters most for a run of a past time period and for a run that is retried. A run for all available data lands every stage change that HubSpot keeps.

Deleted records are not removed. HubSpot does not return deleted records, so a record that is deleted in HubSpot stays in Amperity.

Some records are not pulled. Form submissions that HubSpot spam protection blocks never reach the submissions API, so they never land. contact_ids can be incomplete for a record that is associated with a very large number of contacts: for an activity in a run for a time period, and for deal stage history in every run, including a run for all available data.

Data types are pulled one after another. Every request in a run shares the request limit that HubSpot sets for the HubSpot Activity app in your HubSpot account, so data types are never pulled at the same time. Amperity paces requests below that limit: about 10 requests per second, and about 4 per second for searches. This limit is not shared with the HubSpot destination, which uses a different HubSpot app. A run for all available data in a large HubSpot account can take a long time.

Troubleshoot errors

The following errors may occur when you test the connection to HubSpot Activity or when a courier runs.

Error	Resolution
This HubSpot connection was not granted <permissions>, which <data types> need.	The error names each missing permission and the data types that need it. Authorize HubSpot again and approve every permission that HubSpot shows, or clear those data types from the courier. Some permissions exist only on paid HubSpot subscriptions, so authorizing again adds a permission only if your subscription includes it.
HubSpot rejected the connection’s refresh token, so it has expired or been revoked.	The authorization expired or was revoked. Authorize HubSpot again.
HubSpot did not accept this connection’s access token, or stopped accepting this connection while reading a data type.	Authorize HubSpot again.
HubSpot refused the request because this connection lacks a required scope, or refused this connection while reading a data type.	A permission that the request needs is missing. Authorize HubSpot again and approve every permission that HubSpot shows.
HubSpot is rate limiting this portal, or rate limited this source and did not recover.	Too many requests were made to your HubSpot account in a short time. Amperity retries rate limited requests before it reports an error. Test the connection again shortly, or run the courier again.
HubSpot returned a server error.	A problem on the HubSpot side. Amperity retries these before it reports the run as failed. Run the courier again.
Lost the connection to HubSpot.	A network interruption. Run the courier again.
HubSpot refused Amperity’s app credentials while refreshing the token.	A problem with the Amperity HubSpot Activity app, not with your credential. Contact Amperity Support.
HubSpot rejected a request. Retrying will not help.	Retrying does not resolve this. Contact Amperity Support.
Reading a data type stalled.	Nothing was read or written for that data type for 10 minutes, so the run was ended rather than left running. The error includes the number of records that had been written. Run the courier again. If it happens again, contact your Amperity representative.
Data types must be a non-empty subset of the available types.	The courier was saved with no data types selected. Select at least one data type.

If one data type fails, the entire courier run is reported as failed.

Authorize HubSpot again

To authorize HubSpot again, add a new credential for HubSpot Activity, complete the authorization, and then edit the courier to use the new credential.