Elastic Fabric Adapter (EFA)
An Elastic Fabric Adapter (EFA) is a network interface for Amazon EC2 instances designed to accelerate high performance computing (HPC) and machine learning workloads.
An EFA is an Elastic Network Interface with extra capability. Ordinary IP traffic continues to work through it; the acceleration applies to applications written against the libfabric API.
Characteristics
Section titled “Characteristics”- Performance. Lower and more consistent latency, and higher throughput, than the TCP transport used by a standard network interface.
- OS-bypass. Applications talk to the network adapter directly rather than through the operating system kernel, which removes a significant share of per-message overhead.
- Scalable Reliable Datagram (SRD). EFA uses AWS’s own transport protocol rather than TCP, spreading packets across many network paths and recovering from loss in hardware.
- Ecosystem support. Works with the common HPC and machine learning stacks — MPI implementations and NCCL among them — through libfabric.
- Scale. Designed to scale to thousands of CPUs or GPUs in a single job.
Where it fits
Section titled “Where it fits”EFA suits tightly coupled workloads where nodes exchange messages constantly and the job runs only as fast as the slowest exchange: computational fluid dynamics, weather modelling, molecular dynamics, and distributed model training.
For loosely coupled work, where nodes rarely talk to each other, a standard Elastic Network Interface with enhanced networking is sufficient and simpler.
Constraints
Section titled “Constraints”- EFA is available only on selected instance types.
- It must be enabled explicitly when the interface is created; it is not on by default.
- OS-bypass traffic only flows between instances in the same subnet, and the security group attached to the EFA must allow all inbound and outbound traffic to and from itself.
- Cluster placement groups are the usual companion, because they put the instances close together on the network.