AppFlow

Amazon AppFlow is a managed integration service that moves data between SaaS applications and AWS services — Salesforce, Slack, Zendesk, ServiceNow, Marketo and others on one side, S3, Redshift, Snowflake and further SaaS destinations on the other. Transfers run in both directions, and no integration code is written or hosted.

Core concepts

Flow. The unit of configuration: a source, a destination, and everything about how data moves between them.

Data mapping. How fields from the source are stored at the destination, including concatenation, masking and truncation of individual fields.

Filters. Which records are transferred, evaluated at the source so that unwanted data is never moved.

Triggers. How a flow starts:

• On demand
• On event, when the source application emits one
• On a schedule

Common use cases

1. Transfer Salesforce records to Amazon Redshift
2. Ingest and analyse Slack conversations in S3
3. Migrate Zendesk or other help-desk tickets to Snowflake
4. Land aggregated data in S3 on a schedule

Quotas

The limits that bite in practice are per source connector rather than global: Amplitude allows 25 MB per flow run, Marketo 1 GB on import and 500 MB on export, ServiceNow 100,000 records per flow run, and Salesforce 500 MB per flow run on export. A single flow run may take up to 48 hours. At the account level there is a default of 1,000 flows, 1,000 concurrent flow runs and 10 million flow runs per month.

Schedule frequency is also capped per connector — once a minute for Salesforce, S3 and Slack, once an hour for Marketo, once a day for Amplitude and Google Analytics — and every call AppFlow makes counts against the source application’s own API limits. Check the AppFlow service quotas for the connector in use before sizing a transfer; splitting a large extract across several filtered flows is the usual way around a per-run limit.