HashiCorp Vault

Vault is a secret store. A secret is anything whose access must be tightly controlled — API keys, passwords, certificates, private keys. Vault puts one interface in front of all of them, with access control on every path and an audit record of every read.

It does more than store what is handed to it. Vault can act as a root or intermediate certificate authority, issuing short-lived certificates on demand; it can generate database credentials per application instance with a lease that expires; and it offers encryption as a service, encrypting and decrypting data on behalf of an application that never holds the key.

Two problems justify running it:

1. Secrets are centralised rather than scattered across repositories, CI configuration and individual machines.
2. Long-lived credentials can be eliminated in favour of short-lived ones that are issued on demand and expire on their own.

The trade-off is that every application now depends on Vault being reachable at startup, and that Vault itself becomes a component whose availability and compromise both matter a great deal.

Architecture

Between the HTTP API that clients talk to and the storage backend where data is persisted sits a cryptographic barrier. Nothing crosses it unless Vault is unsealed and the request is authenticated and authorised. Data on the storage side of the barrier is always encrypted, so the storage backend is not a trusted component — it can be a shared service without being able to read anything it holds. HashiCorp’s architecture overview covers the internals.

Seal, unseal and the root key

Vault encrypts data with an encryption key that is itself stored alongside the data. That key is protected by a root key (older documentation calls it the master key), which is held only in memory. Sealing Vault means discarding the root key from memory; nothing can be read until it is supplied again.

vault operator init performs first-time initialisation: it generates the root key, returns the unseal material, and issues an initial root token. That token exists to configure the first real authentication method and policies, and should be revoked as soon as it has done so.

There are three ways to hold the unseal material:

• Shamir seal (the default). The root key is split into shares using Shamir’s Secret Sharing, and some threshold of them must be supplied to reconstruct it. Shares are distributed among several people, and can each be encrypted to a different PGP key so that a share is useless to anyone but its holder. Unsealing is a deliberate, multi-person act.
• Auto unseal. A cloud KMS or an HSM holds the key that encrypts the root key, and Vault decrypts it at startup without human involvement. This is what makes unattended restarts possible. Recovery keys, split the same way as Shamir shares, remain for the operations that still require a quorum.
• Transit unseal. A separate Vault cluster performs the same role as the KMS, for estates that would rather not depend on a cloud provider’s key service.

Reseal deliberately when unseal material is believed to be exposed; rotate it afterwards.

Secrets engines

Secrets engines are the components that store, generate or encrypt data. Each is enabled at a path and is isolated to it. The key/value engine stores what is written to it; the database engine generates credentials on demand; the PKI engine issues certificates; the transit engine encrypts data without storing it.

Authentication methods

An authentication method establishes who or what is calling and attaches policies to them. Several are available — AppRole for applications, OIDC and LDAP for people, and platform-native methods such as Kubernetes or AWS IAM that let a workload authenticate with an identity it already has.

AppRole, the usual choice for a service, uses a Role ID and a Secret ID to obtain a token. Subsequent requests carry the token. The lifetimes of both are configured per role; short token TTLs and regularly rotated Secret IDs are the point of the mechanism, so set them deliberately rather than accepting whatever is there.

Where the platform offers it, prefer a native method over AppRole: a Kubernetes service account or an AWS IAM role removes the bootstrapping problem of how the application gets its first credential.

Paths and policies

Everything in Vault is addressed by path. The prefix tells Vault which component a request routes to: secrets engines and authentication methods are mounted at a path, and the paths available depend on what has been enabled.

Policies are written against paths and grant capabilities — create, read, update, delete, list. Because authorisation is path-based, the path layout is the access control model, and it is worth designing before the first secret is written.

Some engines define paths beneath their mount point. The database engine, for instance:

Path	Holds
database/config/<name>	Connection details for the database
database/roles/<name>	The statements that create a credential
database/creds/<name>	Reading this path generates a credential

A key/value layout that keeps per-application secrets separate from shared ones, so that a policy can grant an application its own subtree and nothing else:

flowchart LR secrets --> apps secrets --> common apps --> app1 apps --> app2 app1 --> app1u[username] app1 --> app1p[password] app2 --> app2u[username] app2 --> app2p[password] common --> apikey[api_key]

Storage backend

The storage backend holds the encrypted data. HashiCorp recommends Integrated Storage, a Raft-based store built into Vault itself, for most deployments: it is highly available, needs no second system to operate and monitor, and removes a network hop from every read. It has been the recommendation since Vault 1.4; Consul, which held that position before, remains supported but is no longer the default answer.

External backends — DynamoDB, PostgreSQL, MySQL, S3, Google Cloud Storage, Azure and others — are still available and receive limited support from HashiCorp. Not all of them support high availability. Choose one only when there is a reason not to use Integrated Storage; a development instance that needs no durability at all can use the in-memory backend.

flowchart TD A{"Production?"} -->|No| B["In-memory"] A -->|Yes| C{"Reason not to use<br/>Integrated Storage?"} C -->|No| D["Integrated Storage (Raft)"] C -->|Yes| E{"Must be highly available?"} E -->|Yes| F["Consul, DynamoDB, PostgreSQL,<br/>Spanner, etcd, and others"] E -->|No| G["Filesystem, S3, and others"]

Server configuration

The seal stanza configures auto unseal. For the AWS KMS seal, Vault needs permission to use the KMS key. Where Vault runs on EC2 or ECS, attach an IAM role and let the SDK pick up the credentials — do not put access_key and secret_key in the configuration file.

A local instance for development

Dev mode starts Vault unsealed, in memory, with a known root token and a key/value version 2 engine already mounted at secret/. It is for local development only: nothing it holds survives a restart, and it listens without TLS.

The vault Docker Official Image is gone — HashiCorp ended support in March 2023 and removed it in June 2023. Use the hashicorp/vault image and pin a version tag rather than latest:

docker run -d -p 8200:8200 --name vault \
  -e 'VAULT_DEV_ROOT_TOKEN_ID=myroot' \
  -e 'VAULT_DEV_LISTEN_ADDRESS=0.0.0.0:8200' \
  hashicorp/vault:<version>

Enter the container and authenticate:

docker exec -it vault sh
export VAULT_ADDR='http://127.0.0.1:8200'
vault login myroot

Write and read a secret. Dev mode mounts key/value version 2 at secret/, so the command is vault kv put, not vault write — the latter returns “Invalid path for a versioned K/V secrets engine”:

vault kv put secret/configclient client.pseudo.property="Property value loaded from Vault"
vault kv get secret/configclient

The browser UI is then at http://localhost:8200, using the same root token to sign in.