Secrets Management: HashiCorp Vault and AWS Secrets Manager

Exposed secrets — API keys in code, passwords in versioned .env files, tokens hardcoded into Docker images — are the root cause of a significant share of data breaches in cloud environments. This article explains how a secrets vault works, compares the two most widely adopted solutions on the market (HashiCorp Vault and AWS Secrets Manager) and describes the correct operating model for production, CI/CD and Kubernetes environments.

The problem: where secrets really live today

In organizations without a formal secrets management strategy, credentials tend to accumulate in unlikely places: hardcoded directly in the source code, in .env files that "should never have been committed," in the environment variables of manually created EC2 instances, in shared access spreadsheets, or in Slack messages from three years ago. The problem is not just that the secret is in the wrong place — it is that there is no inventory, no rotation and no audit of who accessed what.

The OWASP Top 10 2021 lists Cryptographic Failures (A02) as the second most critical category, and the exposure of secrets is one of the main vectors. GitGuardian's State of Secrets Sprawl report found, on average, more than 3 valid secrets per public repository on GitHub — including repositories of organizations that believed they had adequate controls. The attack surface is not theoretical.

What a secrets vault is

A secrets vault is a centralized system that securely stores, distributes and rotates credentials, with granular access control and a complete audit trail. Unlike a personal password manager, a secrets vault is designed to be consumed programmatically by applications, pipelines and infrastructure operators — not by humans typing passwords.

The fundamental principles of any mature solution are: (1) encryption at rest and in transit — the secret never travels or is stored in cleartext; (2) identity authentication — only authorized entities can read a specific secret; (3) immutable audit — every access is logged with timestamp, identity and result; (4) automatic rotation — credentials have a defined lifecycle and are replaced without manual intervention.

HashiCorp Vault

Vault is the reference open-source solution for secrets management in heterogeneous environments. Its architecture is based on secrets engines — plugins that understand how to communicate with external systems to generate, read or revoke credentials.

The KV Secrets Engine is the simplest: it stores encrypted key-value pairs, with versioning (KV v2). The Database Secrets Engine is where the concept of dynamic secrets becomes concrete: instead of storing a fixed PostgreSQL user, Vault connects to the database and generates a unique username/password pair for each request, with a configurable TTL — when the TTL expires, Vault revokes the credential automatically. The PKI Secrets Engine operates as an internal CA, issuing short-lived TLS certificates. The Transit Secrets Engine exposes cryptographic operations (encrypt, decrypt, sign) as an API, without the application needing to manage keys.

Authentication in Vault is done through auth methods: Kubernetes (via ServiceAccount JWT), AWS IAM (via STS), GitHub (via personal access token), LDAP, OIDC, among others. The authenticated identity receives policies written in HCL that define exactly which Vault paths the entity can read, write or list. The model is additive — access denied by default, granted by explicit policy.

Vault records every operation in a configurable audit device (file, syslog, socket). The logs include the token used, the path accessed, the result and the timestamp. In regulated environments (PCI-DSS, ISO 27001, SOC 2), this trail is a compliance requirement.

AWS Secrets Manager

AWS Secrets Manager is Amazon's managed service for storing and rotating secrets. It integrates natively with IAM (access policies by resource ARN), KMS (encryption with customer-managed CMK keys), CloudTrail (auditing of all API events) and Lambda (rotation functions).

Configuring automatic rotation is the operational differentiator of Secrets Manager: for databases managed by AWS (RDS, Aurora, Redshift, DocumentDB), AWS provides pre-built rotation Lambda functions. You simply associate the function with the secret and define the rotation window. The service handles generating the new credential, updating it in the database and in the vault, and verifying that the rotation was successful before invalidating the previous credential.

The pricing model is per stored secret (USD 0.40/secret/month) plus per API call (USD 0.05 per 10,000 requests). For architectures with many microservices reading secrets frequently, the cost can be significant and justifies using a local cache with a short TTL via the SDK.

One limitation compared with Vault: Secrets Manager does not natively support dynamic secrets for databases outside the AWS ecosystem. Credentials are stored and rotated, but not generated on demand with an individual TTL per request — this distinction is important for high-security architectures.

Comparison: Vault, AWS Secrets Manager and alternatives

Criterion HashiCorp Vault AWS Secrets Manager Azure Key Vault GCP Secret Manager
Dynamic secrets Yes (native) No (periodic rotation) No No
Multi-cloud / on-prem Yes AWS only Azure only GCP only
Internal PKI/CA Yes (PKI Engine) No (uses ACM) Yes (Certificate service) No
Operation Self-managed or HCP Vault Fully managed Fully managed Fully managed
Kubernetes integration Vault Agent, CSI Driver, ESO ESO, ASCP ESO, CSI Driver ESO, CSI Driver
Automatic rotation Via Dynamic Secrets + TTL Native Lambda (RDS/Aurora) Lambda / runbooks Requires manual automation
Auditing Audit Device (file/syslog) CloudTrail Azure Monitor / Diagnostic logs Cloud Audit Logs
Base cost Free (OSS) / USD 0.03/hour (HCP) USD 0.40/secret/month USD 0.004/10k operations USD 0.06/10k accesses

Secrets in CI/CD pipelines

CI/CD pipelines are a critical vector for secret exposure. The most common bad practice is storing credentials as static environment variables in the repository or in the CI system configuration — which means any developer with access to the repository can read the secret, and the secret never changes.

The correct model uses ephemeral OIDC authentication: GitHub Actions and GitLab CI support issuing OIDC tokens per job, which identify the repository, the branch and the workflow that is running. These tokens can be exchanged for temporary credentials in Vault (via the JWT auth method) or in AWS (via AssumeRoleWithWebIdentity). The pipeline receives a token valid for minutes — never a long-lived credential. If the pipeline log leaks, the secret has already expired.

For GitHub Actions with Vault: the official hashicorp/vault-action authenticates via OIDC and exports secrets as step environment variables. For AWS Secrets Manager: aws-actions/configure-aws-credentials with role-to-assume does the same. In both cases, the secret should not be printed to the logs — configure add_mask: true or the equivalent.

Secrets in Kubernetes

Kubernetes stores Secret objects in etcd with base64 encoding — which is encoding, not encryption. By default, any pod with access to the namespace can read the secret. For production environments, there are two established patterns for integrating an external vault:

The External Secrets Operator (ESO) is a Kubernetes controller that watches ExternalSecret objects (a CRD) and reconciles them periodically with the external vault, creating or updating native Secret objects. It supports Vault, AWS Secrets Manager, Azure Key Vault, GCP Secret Manager and others. Reconciliation ensures that if the secret rotates in the vault, the cluster's Secret will be updated without a manual restart — provided the application watches for changes or has a reload mechanism.

The Secrets Store CSI Driver with the appropriate provider (Vault Provider, AWS Provider) mounts secrets directly as volumes in the pods at runtime, without creating Secret objects in etcd. It is the most secure approach for secrets that should not exist in the cluster — useful for TLS certificates and cryptographic keys that only the pod that needs them should see.

Regardless of the method, enabling Encryption at Rest in etcd is recommended for any production cluster — available on EKS via envelope encryption with KMS.

What not to do

  • Do not commit .env files, even if they look like development files. Configuring .gitignore is not enough — use pre-commit hooks (detect-secrets, gitleaks) that block the commit if they find secret patterns.
  • Do not use long-lived environment variables in production containers. Environment variables are visible to any child process and can appear in memory dumps and debug logs.
  • Do not create IAM users with static access keys for services on AWS. IAM roles with STS and temporary credentials have been the correct standard since 2018 — there is no technical justification for IAM users in automated workloads.
  • Do not share secrets across environments (dev/staging/prod). Each environment should have its own credentials, with separate access policies.
  • Do not ignore auditing. Enabling logs without periodic review is security theater. Configure alerts for anomalous patterns: read spikes, accesses from unexpected IPs, repeated denials.
  • Do not use the secret directly in code even after fetching it from the vault. Prefer passing it as a function parameter rather than a global variable — to limit the lifetime of the value in memory.

Core concepts

Least privilege: each identity (application, pipeline, operator) should have access only to the secrets it needs, at the moment it needs them. In Vault, this translates into HCL policies by path. In Secrets Manager, into IAM policies by resource ARN with context conditions.

Encryption at rest: Vault encrypts the entire storage backend with AES-256-GCM before writing (barrier encryption). Secrets Manager uses KMS with the option of a customer-managed CMK. In both cases, the database or bucket that holds the encrypted data is useless without the encryption keys.

Encryption in transit: all communication with Vault and with Secrets Manager occurs over TLS 1.2+. In Kubernetes clusters, consider mTLS between pods via a service mesh (Istio, Linkerd) to prevent secrets injected as environment variables from being intercepted in internal transit.

Leasing and revocation: in Vault, every dynamic secret has a lease — a validity contract. The application can renew the lease as long as it needs the credential or let it expire. In case of compromise, the operator revokes all leases of a family with a single command, immediately invalidating all issued credentials.

Frequently asked questions

What is the difference between HashiCorp Vault and AWS Secrets Manager?

Vault is an open-source multi-cloud solution with support for dynamic secrets, multiple authentication backends (LDAP, Kubernetes, JWT, AWS IAM) and specialized engines (PKI, SSH, Transit). AWS Secrets Manager is a managed service natively integrated with the AWS ecosystem (IAM, KMS, Lambda), with easy automatic rotation and billing per stored secret. For workloads that are 100% on AWS, Secrets Manager reduces operational overhead. For hybrid or multi-cloud environments, Vault offers portability and granular control.

What are dynamic secrets and why do they matter?

Dynamic secrets are credentials generated on demand with a defined lifetime (TTL) — valid only for a specific session. Vault, for example, can generate a unique PostgreSQL username/password pair for each deploy, which expires automatically after 1 hour. This eliminates shared and permanent credentials: if a secret leaks, it no longer exists. It is the difference between a master key that never changes and a visitor badge that expires at the end of the day.

Is storing secrets in environment variables (.env) safe?

Not entirely. .env files solve the problem of hardcoding in the source code, but they create others: they are copied into Docker images, appear in debug logs, remain in shell history and frequently end up in repositories accidentally. The correct approach is to use a secrets vault as the canonical reference and inject variables at runtime via a sidecar, CSI driver or SDK — never persisting the value to disk or to long-lived operating-system environment variables.

How do you integrate secrets management into CI/CD pipelines?

The recommended practice involves three steps: (1) authenticate the pipeline to the vault using an ephemeral identity (a GitHub Actions or GitLab CI OIDC token, never static credentials); (2) request from the vault only the secrets needed for that specific pipeline step, with minimal scope; (3) do not export the values to logs or artifacts. HashiCorp Vault has the hashicorp/vault-action action; AWS Secrets Manager integrates via aws-actions/aws-secretsmanager-get-secrets. In both cases, the pipeline should never store the secret beyond the job's execution time.

What is automatic secret rotation and how do you configure it?

Automatic rotation is the process of replacing a credential with a new one programmatically, without human intervention and without downtime. In AWS Secrets Manager, you simply associate a Lambda function with the secret and define the rotation window (e.g., every 30 days); AWS provides ready-made templates for RDS, Redshift and other services. In Vault, the Dynamic Secrets Engine covers databases, cloud credentials and PKI with a configurable TTL. Rotation is not optional in production environments — NIST SP 800-63B and OWASP ASVS level 2 require periodic rotation of privileged credentials.

How do you use secrets management in Kubernetes?

There are two established patterns: the External Secrets Operator (ESO), which synchronizes secrets from Vault or AWS Secrets Manager into native Kubernetes Secret objects in real time; and the Secrets Store CSI Driver, which mounts secrets directly as volumes in the pods without persistence in etcd. The ESO is preferable when you need continuous reconciliation; the CSI driver is indicated when the requirement is not to store the secret in the cluster. In both cases, avoid creating Kubernetes Secrets manually — they are stored in etcd with base64 encoding, not encrypted by default, unless you enable Encryption at Rest in the cluster.

References

How Decripte implements secrets management

Decripte designs and implements secrets management architectures for organizations of any size — from startups with a single AWS environment to companies with more than 100,000 employees operating across multiple clouds and on-premises datacenters. The work covers everything from the initial inventory of secrets scattered across the environment to the full implementation of Vault or AWS Secrets Manager, integration with CI/CD pipelines, Kubernetes and a DevSecOps security posture review.

The Threat Management plan includes an assessment of credential exposure and practical remediation recommendations. For deeper implementations — vault onboarding, dynamic secrets, internal PKI, SIEM integration — the Incident Response and Compliance plans cover the full scope with continuous monitoring.

Start with the free diagnostic: in minutes you get a view of the current state of credential exposure in your environment.