MSB
All posts

DevOps Secrets Manager: One Place for the Keys

A secrets platform with envelope encryption, per-vault roles, key rotation and a CLI that injects credentials into any command — and what finishing it taught me about the bugs that hide between three clients and one API.

The problem it goes after

Every team eventually has credentials living in the wrong places: a database password pasted into a chat thread, a production .env file passed around by hand, an API key that nobody remembers rotating. DevOps Secrets Manager is my attempt at the tool that replaces that habit. It gives a team one place to store secrets, organized the way deployments actually are, with a clear record of who read what.

The work is split into vaults, usually one per service, and each vault holds environments such as development, staging, and production. A secret is a key and a value inside one environment, so the same DATABASE_URL can point somewhere different in each.

How it works

There are three pieces around a PostgreSQL database. A Go REST API owns the data and every security decision. A React and TypeScript web console is where people browse vaults, add secrets, and manage who has access. A Rust CLI brings the same vaults to the terminal, where most of the actual use happens.

The encryption uses an envelope design. A master key, supplied to the API as an environment variable and never stored in the database, encrypts a separate data key for each vault. That vault key encrypts every secret value with AES-256-GCM. The database only ever holds ciphertext, and a single vault's key can be replaced without touching every other vault.

Access is role-based. Each person has one of five roles on a vault: owner, admin, developer, on-call, or viewer. Viewers can see which secrets exist but can't read their values; on-call engineers can read values but not change them; only owners can delete a vault. Reading a value is deliberately its own action in the interface, with a notice that it's being logged and a view that hides the value again after thirty seconds.

The CLI is what makes it practical day to day. It can list vaults and environments, set a value, pull an environment into a .env file, or run a command with the secrets injected directly into that process's environment, so they never have to touch disk at all. In a pipeline it authenticates with a token scoped to one environment, and every read it makes appears in the audit log under that token's name rather than a person's.

Values are versioned, so a bad update can be rolled back to the value that worked, and an environment can be imported from or exported to a .env file — the same format the CLI writes, which is how most of these secrets arrive in the first place.

What works well

  • The encryption boundary is in the right place. The API decrypts only on an explicit request, and what's stored is useless without the master key, which lives outside the database. Both keys can be rotated: the master key without re-encrypting a single value, a vault's data key by re-encrypting that vault's values and their history in one transaction.
  • Revealing is a considered action rather than a default. The audit notice and auto-hide make reading a production credential feel like the deliberate step it is, and the log shows who read what, from where.
  • Permissions are checked against the caller's effective role on the vault that owns the resource, and things you cannot see return "not found" rather than "forbidden", so identifiers can't be probed. A test runs every vault endpoint as all five roles and compares the result against the same permission table the interface uses.
  • Running a command with injected secrets is the feature that would actually change a team's habits, since it removes the reason to keep a local .env copy at all. Pipelines use a token scoped to one environment, so a leaked CI token is not a leaked account.

Where it falls short

Setting it up for a screenshot surfaced the project's weakest point: the pieces had drifted apart. The vault handlers depended on a membership table that no database migration created. The Docker image referenced a configuration file that didn't exist. The dashboard's numbers were invented, the audit page stayed empty, and a viewer could reveal production values because the permission check used the organization role instead of the role on the vault.

I went back and finished it properly, and the same class of gap kept turning up: registration wasn't atomic, so a failure left an account that could never register again; editing a secret's description quietly wiped its value; the browser dropped the session on every page reload; the CLI wrote its fallback token file world-readable and called it encrypted; the bulk delete button logged to the console instead of deleting. Each one is in the history with the test that now covers it.

What's still missing is the part a team would notice on day two: there's no multi-factor authentication or single sign-on, and the master key comes from an environment variable rather than a cloud KMS or an HSM, so whoever controls the API host can read everything. The hosted demo is documented rather than running, the Docker images are built by CI but have never been built on my machine, and the macOS and Windows CLI binaries are produced by the release workflow without my having run them.

What I'd do next

The contract problem is solved — the console's types are generated from the OpenAPI description, integration tests run the real API against a real database, and the same specification checks every response in those tests — so the next work is about trust rather than correctness. Single sign-on and multi-factor authentication come first, because a secrets tool is only as strong as the login in front of it. Then the master key should move to a cloud KMS, so the key is never in a process's environment at all. After that, an approval flow for production reveals: the audit log records who read a credential, but it can't ask a second person first.

Key takeaways

  • Envelope encryption keeps a leaked database from being a leaked set of credentials, and makes per-vault keys cheap to rotate.
  • Making "reveal" an explicit, audited action changes how casually secrets get read.
  • Three clients over one API only work if the contract is generated and tested, not assumed: types from the specification, integration tests against a real database.
  • The dangerous bugs were the quiet ones — a permission check reading the wrong role, an update wiping the value it was meant to keep.