Designing pipeline environments

A pipeline’s shape is mostly a question of how many environments a change passes through and what changes between them. Four is the common answer; more than four usually means one of them is not earning its keep, and fewer usually means production is doing a job that a pre-production environment should have done.

The environments

Development. Where a change is first exercised. Deployed automatically on every commit, expected to be broken some of the time, sized as small as it can be. Automated tests run here first, and a failure here should stop the pipeline before anything else runs.

Quality assurance. More stable than development. This is where integration tests, performance tests and user acceptance testing run. The architecture mirrors production but at reduced scale. Deployments are automatic on promotion from development, sometimes gated.

Staging or pre-production. A faithful mirror of production, including scaling behaviour and data volumes where that is possible. This is where load testing, security testing and data migration rehearsals happen. Deployments require approval.

Production. Live traffic. Strictest access controls, manual approval on the deployment, full scaling and high availability. Nobody has standing write access to it outside the pipeline.

Two optional environments earn their place in specific circumstances:

Hotfix. A short path to production for genuine emergencies, bypassing the normal promotion sequence. Access is tightly controlled and every use is reviewed afterwards, because an unaudited fast path becomes the normal path.

Demo or training. A stable, controlled data set for customer demonstrations and onboarding. It follows production’s version but on a schedule rather than automatically.

What changes between environments

Everything that differs between environments should be configuration, not a different artefact. The build happens once; the same package is promoted.

	Dev	QA	Staging	Production
Deployment	Automatic on commit	Automatic on promotion	Manual approval	Manual approval
Instance sizing	Smallest viable	Reduced scale	Production-like	Full
Scaling	Fixed, scaled down out of hours	Fixed	Auto Scaling	Auto Scaling
Monitoring	Critical alarms only	Critical alarms only	Full	Full, with paging
Backups	Daily, short retention	Daily	Production-like	Continuous, long retention
Developer access	Full	Read/write	Read-only	Read-only, break-glass audited
Artefact retention	7 days	7 days	30 days	90 days or longer

A worked CodePipeline definition

The structure below is the shape CodePipeline actually accepts: a list of stages, each holding action objects with an actionTypeId and a provider-specific configuration. Trimmed for length, but structurally valid.

{
  "pipeline": {
    "name": "example-service",
    "roleArn": "arn:aws:iam::111122223333:role/service-role/codepipeline-example-service",
    "artifactStore": {
      "type": "S3",
      "location": "example-pipeline-artifacts",
      "encryptionKey": {
        "id": "arn:aws:kms:eu-west-2:111122223333:key/EXAMPLE-KEY-ID",
        "type": "KMS"
      }
    },
    "stages": [
      {
        "name": "Source",
        "actions": [
          {
            "name": "Source",
            "actionTypeId": {
              "category": "Source",
              "owner": "AWS",
              "provider": "CodeStarSourceConnection",
              "version": "1"
            },
            "configuration": {
              "ConnectionArn": "arn:aws:codeconnections:eu-west-2:111122223333:connection/EXAMPLE",
              "FullRepositoryId": "example-org/example-service",
              "BranchName": "main"
            },
            "outputArtifacts": [{ "name": "SourceArtifact" }]
          }
        ]
      },
      {
        "name": "Build",
        "actions": [
          {
            "name": "BuildAndUnitTest",
            "actionTypeId": {
              "category": "Build",
              "owner": "AWS",
              "provider": "CodeBuild",
              "version": "1"
            },
            "configuration": { "ProjectName": "example-service-build" },
            "inputArtifacts": [{ "name": "SourceArtifact" }],
            "outputArtifacts": [{ "name": "BuildArtifact" }]
          }
        ]
      },
      {
        "name": "DeployDev",
        "actions": [
          {
            "name": "DeployDev",
            "actionTypeId": {
              "category": "Deploy",
              "owner": "AWS",
              "provider": "CloudFormation",
              "version": "1"
            },
            "configuration": {
              "ActionMode": "CREATE_UPDATE",
              "StackName": "example-service-dev",
              "TemplatePath": "BuildArtifact::packaged.yaml",
              "Capabilities": "CAPABILITY_IAM",
              "RoleArn": "arn:aws:iam::111122223333:role/cfn-deploy-example-service"
            },
            "inputArtifacts": [{ "name": "BuildArtifact" }]
          }
        ]
      },
      {
        "name": "Production",
        "actions": [
          {
            "name": "Approve",
            "actionTypeId": {
              "category": "Approval",
              "owner": "AWS",
              "provider": "Manual",
              "version": "1"
            },
            "configuration": {
              "NotificationArn": "arn:aws:sns:eu-west-2:111122223333:release-approvals"
            },
            "runOrder": 1
          },
          {
            "name": "DeployProd",
            "actionTypeId": {
              "category": "Deploy",
              "owner": "AWS",
              "provider": "CloudFormation",
              "version": "1"
            },
            "configuration": {
              "ActionMode": "CREATE_UPDATE",
              "StackName": "example-service-prod",
              "TemplatePath": "BuildArtifact::packaged.yaml",
              "Capabilities": "CAPABILITY_IAM",
              "RoleArn": "arn:aws:iam::444455556666:role/cfn-deploy-example-service"
            },
            "inputArtifacts": [{ "name": "BuildArtifact" }],
            "runOrder": 2
          }
        ]
      }
    ]
  }
}

The production stage deploys with a role in a different account, which is what makes the separation real rather than nominal.

Environments as nested stacks

Where the environments share a template, a parent stack per environment keeps the difference in parameters rather than in the template:

Resources:
  DevEnvironment:
    Type: AWS::CloudFormation::Stack
    Properties:
      TemplateURL: https://s3.eu-west-2.amazonaws.com/example-templates/environment.yaml
      Parameters:
        EnvironmentType: Dev
        InstanceSize: t3.small

  ProdEnvironment:
    Type: AWS::CloudFormation::Stack
    Properties:
      TemplateURL: https://s3.eu-west-2.amazonaws.com/example-templates/environment.yaml
      Parameters:
        EnvironmentType: Prod
        InstanceSize: t3.large

Isolation

Separate AWS accounts per environment are the strongest boundary available and the one worth paying for: a runaway process in development cannot exhaust a production quota, and a misconfigured security group cannot expose production data. Below that, separate VPCs and environment-specific security groups give network isolation within one account, but not blast-radius isolation from IAM mistakes or service quota exhaustion.

Configuration and secrets

Non-secret configuration belongs in Systems Manager Parameter Store, namespaced by environment (/example-service/prod/log-level). Credentials that need rotation belong in AWS Secrets Manager. Neither belongs in the repository, in a CloudFormation parameter default, or in an environment variable committed to a task definition.

Cost

Development and QA: small instances, scaled to zero or stopped outside working hours.
Staging: medium instances, brought to production scale only for load tests.
Production: full Auto Scaling.
Artefacts and logs: a lifecycle policy per environment, with shorter retention in the lower environments, so the artefact bucket does not grow without bound.

Operational readiness

Disaster recovery. Test restores rather than backups. A backup that has never been restored is a hypothesis. Document the restoration procedure alongside the pipeline that creates the environment.

Auditing. CloudTrail enabled in every account, logs delivered to a separate log-archive account, and the pipeline as the only routine path to production so that CloudTrail’s record of production change is short and legible.

Approval gates. Put them where the risk changes — before staging and before production — not at every boundary, where they degrade into a rubber stamp.

Adjust the number of environments to the team and the compliance regime. The test is whether each environment has caught something the previous one did not.