Staging bulk data in S3 before loading Aurora
When bulk data has to move from an existing estate into Aurora, the reliable shape is two steps rather than one: get the data into S3, then load Aurora from S3. Pushing directly into the database looks like it saves a step and generally costs more than it saves.
Why the intermediate hop earns its place
Section titled “Why the intermediate hop earns its place”- S3 is a durable staging area. Once the extract has landed, the source system’s part of the job is finished and can be released.
- You can validate before you load. Row counts, checksums and schema checks run against the staged files, so bad data is caught before it reaches the database rather than after.
- Restarts are cheap. A failed load resumes from the staged objects. A failed direct transfer usually resumes from the source, which may no longer be available.
- Loading parallelises. Files partitioned in S3 can be loaded concurrently, which a single streaming connection cannot do.
- You keep a copy. The staged extract is a point-in-time backup of what was migrated, which is what you will want when someone asks why a row looks wrong.
Aurora reads from S3 natively
Section titled “Aurora reads from S3 natively”Aurora is built for this path. Aurora MySQL provides LOAD DATA FROM S3 and
LOAD XML FROM S3; Aurora PostgreSQL provides the aws_s3 extension’s
table_import_from_s3. Both run inside the engine, are granted access through an IAM role
attached to the cluster, and report progress and errors through the database rather than
through an external tool. AWS Database Migration Service can also target Aurora with S3 as its
source.
Getting the data into S3
Section titled “Getting the data into S3”The physical-transfer half of this changed recently and the older advice no longer works.
- Over the network — AWS DataSync. DataSync moves data from NFS, SMB, HDFS and self-managed object storage into S3, with in-line validation, compression and encryption. A single task can saturate a 10 Gbps link. Pair it with a Direct Connect hosted connection when the internet path is the constraint.
- Physically — AWS Data Transfer Terminal, or a partner appliance. Data Transfer Terminal is a secure AWS facility you book a slot at and upload from your own drives at high speed. AWS Marketplace partners offer appliance-based alternatives.
- AWS Snowball is closing. Since 7 November 2025 Snowball Edge has been available only to existing customers, so an account that does not already use it cannot start. On 31 December 2026 AWS discontinues support for Snowball devices in all commercial Regions, after which the Snow Console and Snowball resources are no longer accessible — so an existing user needs a migration plan now rather than reassurance. AWS directs new work to DataSync, Data Transfer Terminal or partner solutions, and to AWS Outposts for the edge-compute use case.