Lambda architecture patterns

This page covers how Lambda functions are composed into systems: the recurring architectural patterns, the deployment frameworks, and the event bus that usually sits between them. For the service itself and its quotas, see Lambda.

Characteristics that shape the architecture

• Code runs on demand, with no infrastructure to manage
• Execution is stateless — nothing survives an invocation except by leaving the function
• Scaling is automatic and per function
• Billing is per request and per GB-second, with nothing charged while idle

Those four properties are what the patterns below work around.

Fan-out

A single event triggers parallel work. A typical image pipeline:

1. A mobile client uploads an image, and the upload lands on an SQS queue.
2. A receipt function reads the queue and, for each image, invokes several downstream functions in parallel — one to resize, one to extract metadata, one to send an acknowledgement email.
3. The resize function writes to Amazon S3; the metadata function writes to DynamoDB; the notification function sends through Amazon SES.
4. Messages the receipt function cannot process are moved to a dead-letter queue for inspection.

Each branch scales independently and fails independently, which is the point: a broken email template does not stop images being resized. Two things make or break this pattern in practice — a dead-letter queue on every asynchronous branch, and idempotent handlers, because at-least-once delivery means every function will eventually see the same message twice.

Fan-out can be built with direct invocations as above, or with an SNS topic or an EventBridge bus in the middle, which decouples the producer from the number of consumers.

State management

Lambda functions are stateless. State has to live somewhere else:

• DynamoDB for key-value and document state, including idempotency records and job status
• Amazon S3 for objects and large payloads — pass a bucket and key between functions rather than the payload itself
• Amazon ElastiCache or MemoryDB for shared low-latency caches
• AWS Step Functions where the state is the workflow: retries, branching, parallelism and long waits belong in a state machine rather than in a function that polls

The /tmp directory and module-level variables survive between invocations that reuse an execution environment, which makes them a legitimate cache. They are not storage: the environment can be reclaimed at any time and is never shared between concurrent invocations.

AWS Serverless Application Model (SAM)

SAM is an open-source framework for serverless applications. A SAM template is a CloudFormation template with extra resource types — AWS::Serverless::Function, AWS::Serverless::Api, AWS::Serverless::StateMachine — that expand into full CloudFormation at deploy time. Running sam deploy from a workstation or a pipeline creates a CloudFormation stack, and CloudFormation creates the Lambda functions, API Gateway APIs, DynamoDB tables and IAM roles the template describes.

What it adds over hand-written CloudFormation:

1. Local testing. sam local invoke and sam local start-api run functions in a container against real event payloads before anything is deployed.
2. CLI tooling. sam build, sam deploy and sam logs cover build, deployment and log tailing.
3. Sensible defaults. A few lines of SAM expand into the IAM roles, log groups and permissions that would otherwise be written by hand.

A minimal function:

Resources:
  ImageProcessor:
    Type: AWS::Serverless::Function
    Properties:
      Runtime: nodejs22.x
      Handler: index.handler
      MemorySize: 512
      Timeout: 30

Pin the runtime to a currently supported version and revisit it on a schedule — Lambda blocks function creation and, later, function updates on deprecated runtimes.

SAM and the Serverless Framework

Both use YAML templates, both are built for serverless applications, and both generate CloudFormation when targeting AWS.

• AWS SAM is AWS-only. It tracks new AWS features as they ship, is supported by AWS, and produces stacks that behave exactly like any other CloudFormation stack.
• The Serverless Framework supports other providers — Azure, Google Cloud and others — through a plugin model, and has a larger plugin ecosystem.

The deciding question is whether you actually deploy to more than one cloud. If not, SAM is the smaller dependency; if the AWS-only constraint is the problem, or a specific plugin is doing real work, the Serverless Framework earns its place. Terraform and the AWS CDK are the other two common answers, and both are reasonable.

Amazon EventBridge

EventBridge is a serverless event bus that routes events from AWS services, SaaS applications and your own code to targets, using rules that match on event content.

Sources publish onto a bus: AWS services onto the default bus, your applications onto a custom bus, and integrated SaaS partners onto a partner event source bus.

Rules filter events with a pattern — matching on source, detail type, or any field in the event body — and route matches to targets.

Targets include Lambda functions, Step Functions state machines, SQS queues, SNS topics, Kinesis Data Firehose delivery streams, ECS tasks and API destinations.

It is the standard way to decouple producers from consumers: the producer publishes a fact, and consumers subscribe to the facts they care about without the producer knowing they exist. Adding a consumer becomes a rule rather than a code change upstream. EventBridge Scheduler covers the cron case, replacing scheduled CloudWatch Events rules.

Practices worth adopting

Design. Keep functions stateless and single-purpose. Make every handler idempotent. Put retries, branching and waits in Step Functions rather than in code. Give every asynchronous path a dead-letter queue.

Development. Test locally before deploying. Instrument with structured logs and X-Ray traces from the start; a distributed system is very hard to retrofit observability onto. Initialise SDK clients and connections at module scope so they are reused — see scaling and concurrency.

Cost. Tune memory by measuring duration at several settings rather than guessing. Set timeouts to slightly above the observed p99, not to the maximum. Watch concurrency against the account quota so one function cannot starve the rest.