Lessons From Locking Myself Out of Vault (Multiple Times)
Yes, I Locked Myself Out
More than once, I managed to:
- enforce mTLS on Vault,
- let important client certs expire,
- and then stare at
tls: expired certificateerrors.
This wasn’t fun, but it was educational.
What Went Wrong
Patterns:
- Short TTLs without rotation.
- No alerting or scripts to warn me early.
- Assumption that a root token would somehow “save me” even if TLS failed.
In reality:
- TLS didn’t care about my root token,
- Vault never saw my HTTP requests,
- I had to fall back to reconfiguring listeners or restarting from a more manual state.
What I’m Changing
Because of this, I am:
- documenting maintenance procedures,
- designing a proper break-glass listener,
- writing scripts and timers for cert checks and rotation,
- and capturing the whole mess as blog posts instead of keeping it in my head.
Failing like this once is bad.
Failing the same way twice and not learning from it would be worse.