Skip to content
PuglieseWeb Engineering HandbookSystemDox Pages

VPC routing and BGP

Every VPC has an implicit router and a main route table. You can edit the main table or create custom tables and associate them with individual subnets.

Two rules govern everything else:

  • Each route table automatically contains a local route covering the VPC CIDR block. It cannot be removed, and it always wins for traffic inside the VPC.
  • Route selection uses the most specific match. A /32 beats a /24, which beats the default route 0.0.0.0/0.

Destinations and targets

Section titled “Destinations and targets”

A destination says which network or host the packet is for:

  • a single host address, such as 192.168.1.100/32
  • a network, such as 192.168.1.0/24
  • the default route, 0.0.0.0/0, which matches anything not listed explicitly
  • a managed prefix list ID, pl-xxxxxxxx, which stands for a set of CIDR blocks — AWS publishes prefix lists for services such as Amazon S3 and DynamoDB so you do not have to track their ranges yourself

A target says where to send the packet next:

  • local — an address inside the VPC
  • igw-xxxxxxxx — an internet gateway
  • nat-xxxxxxxx — a NAT gateway
  • vgw-xxxxxxxx — a virtual private gateway, the AWS end of a Site-to-Site VPN or a Direct Connect private virtual interface
  • tgw-xxxxxxxx — a transit gateway
  • pcx-xxxxxxxx — a VPC peering connection
  • vpce-xxxxxxxx — a gateway VPC endpoint
  • eni-xxxxxxxx — a network interface, used when routing through an appliance

A worked example

Section titled “A worked example”
Destination         Target
10.0.0.0/16         local
192.168.0.0/24      vgw-xxxxxxx
0.0.0.0/0           nat-xxxxxxx
pl-xxxxxxx          vpce-xxxxxxx

Resolving four destinations against it:

10.0.45.34          local          — inside the VPC CIDR
10.0.255.255        local          — also inside the VPC CIDR; the local route
                                     is the most specific match
64.56.34.1          nat-xxxxxxx    — no specific route, so the default applies
192.168.0.7         vgw-xxxxxxx    — matches the on-premises prefix
an Amazon S3 address vpce-xxxxxxx  — matched by the S3 prefix list, which is
                                     more specific than 0.0.0.0/0

The last two lines are the point of the example: adding a gateway endpoint changes where S3 traffic goes without touching the application, and the on-premises route pulls one prefix off the default path.

Border Gateway Protocol

Section titled “Border Gateway Protocol”

BGP is how AWS and your network exchange routes over Direct Connect and dynamically routed VPNs.

Where it is used

  • Required for Direct Connect virtual interfaces.
  • Optional for Site-to-Site VPN, which can also use static routes. Dynamic routing is what makes automatic failover between tunnels and between links possible.
  • Not involved in VPC peering or in intra-VPC routing, which are not dynamic.

How it works in practice

  • BGP runs over TCP port 179, plus ephemeral ports for the return path.
  • Each side identifies itself with an Autonomous System Number (ASN). AWS uses 64512 by default for a virtual private gateway or transit gateway; you choose a private ASN from 64512–65534 or 4200000000–4294967294 for your side.
  • Local preference and weight on your routers decide which path outbound traffic takes. AWS honours BGP communities on Direct Connect for controlling the scope of advertised routes and the preference of inbound traffic.
  • AS path prepending is the usual way to make one link less attractive than another for traffic coming from AWS.

A common hybrid design advertises the same on-premises prefixes over both a Direct Connect private virtual interface and a backup VPN, and relies on Direct Connect being preferred. When the Direct Connect link fails, BGP withdraws its routes and the VPN takes over without any manual change.

Route propagation. A route table can be configured to accept routes learned by an attached virtual private gateway automatically, rather than having them entered by hand. See route tables.

Operating practice

Section titled “Operating practice”
  1. Document what each non-local route is for; a route table with unexplained entries is a route table nobody dares change.
  2. Audit route tables periodically for entries pointing at deleted resources — they show as blackhole.
  3. Use BGP communities on Direct Connect to control which routes are advertised where.
  4. Design a backup path for anything business-critical, and test the failover rather than assuming it.
Section titled “Related pages”