AWS Lambda Scaling and Concurrency Optimization Guide

Lambda scales by creating execution environments. Understanding the two numbers that govern that — the concurrency limit and the concurrency scaling rate — is most of what is needed to size a function and to diagnose a throttle.

Concurrency

Concurrency is the number of requests a function is serving at any instant. The account has a default limit of 1,000 concurrent executions per Region, shared across all functions and increasable on request.

Reserved concurrency

Reserved concurrency carves a fixed portion of the account limit out for one function. It does two jobs at once:

• It guarantees that much concurrency is available to that function, so a noisy neighbour cannot starve it.
• It caps the function at that number, so the function cannot consume the whole account limit — useful when the function calls a downstream system with its own throughput ceiling.

Reserving concurrency costs nothing, but it is subtracted from the pool available to everything else.

{
  "FunctionName": "MyFunction",
  "ReservedConcurrentExecutions": 100
}

Provisioned concurrency

Provisioned concurrency keeps a number of execution environments initialised and waiting, so requests do not pay initialisation cost. It is the answer for latency-sensitive synchronous paths where a cold start is visible to a user.

It is charged for the whole time it is provisioned, whether or not the environments are used, and it is not covered by the free tier. Attach it to a version or alias, and scale it on a schedule or with Application Auto Scaling rather than provisioning peak capacity around the clock.

{
  "FunctionName": "MyFunction",
  "Qualifier": "live",
  "ProvisionedConcurrentExecutions": 50
}

ScalingPolicy:
  Type: AWS::ApplicationAutoScaling::ScalingPolicy
  Properties:
    PolicyName: ProvisionedConcurrencyPolicy
    PolicyType: TargetTrackingScaling
    TargetTrackingScalingPolicyConfiguration:
      TargetValue: 0.75
      PredefinedMetricSpecification:
        PredefinedMetricType: LambdaProvisionedConcurrencyUtilization

The concurrency scaling rate

Concurrency is how many environments a function may have. The scaling rate is how quickly it may acquire them, and it is a separate limit.

In each Region, and for each function, Lambda can add up to 1,000 execution environments every 10 seconds — equivalently, up to 10,000 additional requests per second every 10 seconds. This is a function-level limit, so each function scales independently of every other function in the account, up to the account concurrency limit.

Two properties are worth knowing:

• Lambda refills the allowance continuously rather than in a single burst every 10 seconds.
• Unused allowance does not accrue. Ten idle seconds do not buy 2,000 environments in the next interval; the rate is always at most 1,000 per 10 seconds.

If requests arrive faster than the function can scale, or the function is at its concurrency limit, the excess is throttled with a 429.

This replaced an older model in which a shared account-level burst of 500–3,000 environments was followed by 500 additional environments per minute. Any sizing calculation based on those figures is wrong by roughly an order of magnitude, and any advice about staggering functions to avoid competing for a shared burst pool no longer applies.

Cold starts

A cold start is the initialisation of a new execution environment: downloading the code, starting the runtime, and running everything outside the handler. It happens on the first request to a new environment and never again for that environment.

Initialise outside the handler

Lambda reuses an execution environment for subsequent invocations. Anything created at module scope is created once per environment; anything created inside the handler is created on every single invocation. SDK clients, database connections and configuration parsing therefore belong at module scope.

import boto3

# Correct: created once per execution environment and reused.
s3 = boto3.client("s3")

def handler(event, context):
    return s3.get_object(Bucket=event["bucket"], Key=event["key"])

import boto3

# Wrong: a new client, and a new TLS handshake, on every invocation.
def handler(event, context):
    s3 = boto3.client("s3")
    return s3.get_object(Bucket=event["bucket"], Key=event["key"])

The same reasoning applies to cached static assets: write them to /tmp on first use and check for them on subsequent invocations.

The caveat is that initialisation code runs before the handler and its failures are harder to attribute, so keep it to client construction and configuration, not to work that can fail in interesting ways.

Other levers

• Reduce package size. Ship only what the function imports; a smaller artefact downloads and initialises faster.
• Choose the runtime deliberately. Interpreted runtimes initialise fastest; JVM and .NET functions initialise slowest and then run quickly.
• Use provisioned concurrency where the latency is user-visible and the cost is justified.
• Use SnapStart for Java, .NET and Python functions, which restores from a snapshot of the initialised environment.

“Warming” a function by invoking it on a timer is a workaround, not a fix: it keeps a small number of environments alive and does nothing for a traffic spike, which is when cold starts actually hurt. Provisioned concurrency does the same job with a guarantee.

Memory and CPU

Lambda allocates CPU in proportion to memory: one vCPU at 1,769 MB, up to about six vCPUs at the maximum 10,240 MB. Because billing is per GB-second, raising memory does not necessarily raise cost — a function that halves its duration at twice the memory costs the same and returns sooner.

There is no universally correct setting. Profile the function at several memory sizes and pick the point where cost per invocation stops falling; AWS Lambda Power Tuning automates this. A single-threaded function stops improving past one vCPU, so the useful range for most functions is narrower than the configurable one.

Monitoring

Watch these CloudWatch metrics in the AWS/Lambda namespace:

Metric	Statistic	What it tells you
ConcurrentExecutions	Maximum	How close the function is to its limit
Throttles	Sum	Requests rejected for want of concurrency
Duration	Average and p99	Whether memory tuning is working
Errors	Sum	Function failures
IteratorAge	Maximum	Stream processing falling behind

An alarm at roughly 80% of the concurrency limit gives useful warning:

ConcurrencyAlarm:
  Type: AWS::CloudWatch::Alarm
  Properties:
    AlarmDescription: Concurrency has reached 80% of the account limit
    MetricName: ConcurrentExecutions
    Namespace: AWS/Lambda
    Statistic: Maximum
    Period: 300
    EvaluationPeriods: 2
    Threshold: 800
    ComparisonOperator: GreaterThanThreshold
    AlarmActions:
      - !Ref AlertSNSTopic

Error handling

Distinguish failures worth retrying from failures that will never succeed. Raising an exception makes Lambda retry an asynchronous invocation and keeps a stream or queue record in place; returning normally marks it handled.

def handler(event, context):
    try:
        process_event(event)
    except TransientError:
        # Let Lambda retry: the downstream system may recover.
        raise
    except PermanentError:
        # Never going to succeed. Record it and let the record be removed.
        logger.exception("permanent failure, sending to DLQ")
        send_to_dlq(event)

Configure a dead-letter queue or an on-failure destination on every asynchronous path, so a message that exhausts its retries is kept rather than dropped.

Troubleshooting

Throttling. Check whether the function has hit its own reserved concurrency or the account limit, and whether traffic is arriving faster than 1,000 new environments per 10 seconds. Buffer through SQS to convert a spike into a queue, increase reserved concurrency, or request an account limit increase.

High latency. Separate cold-start latency from execution latency using the Init Duration field in the CloudWatch Logs report line. Cold starts want provisioned concurrency or SnapStart; slow execution wants memory tuning or a look at the downstream call.

Timeouts. Set the timeout slightly above the observed p99, not to the maximum. A timeout set to 15 minutes turns a hung downstream call into 15 minutes of billed time and a blocked concurrency slot.