Secrets in dev vs production

Secrets in dev vs production

Your agent pipeline reads its secrets from two different places depending on where it runs: a .env file on your laptop, and Google Secret Manager in production. One pipeline, two sources, and a single if to switch between them.

var builder = WebApplication.CreateBuilder(args);

if (builder.Environment.IsProduction())
    builder.Configuration.AddGcpSecretManager(projectId); // mounted at runtime
else
    DotNetEnv.Env.Load(); // laptop only — never shipped

// From here on, code reads IConfiguration and never knows the difference:
var bullhornSecret = builder.Configuration["Bullhorn:ClientSecret"];

Dev convenience and production safety should cost you one if, not two codebases.

IAM & least privilege

The runtime service account gets the smallest set of permissions that lets the agents do their job, and nothing more. If the container is ever compromised, this is the blast radius — so keep it small.

Grant only these roles to agents-runtime@…:

# Read only the specific secrets it needs — not "all secrets"
gcloud secrets add-iam-policy-binding openai-api-key \
  --member serviceAccount:agents-runtime@$GCP_PROJECT_ID.iam.gserviceaccount.com \
  --role roles/secretmanager.secretAccessor
# ...repeat per-secret. Bind at the secret level, never project-wide.

# Pull work from the queue; write to the DLQ
gcloud projects add-iam-policy-binding $GCP_PROJECT_ID \
  --member serviceAccount:agents-runtime@$GCP_PROJECT_ID.iam.gserviceaccount.com \
  --role roles/pubsub.subscriber

# Write structured logs and traces
# roles/logging.logWriter, roles/cloudtrace.agent

The rules, in plain terms:

• Per-secret access, not project-wide. Bind secretAccessor on each individual secret. A blanket "read all secrets" grant means one compromised container reads everything.
• A dedicated runtime identity. Never the project's default service account — it tends to accumulate permissions over time and you lose track of what it can touch.
• No write access it doesn't use. The agents read the ATS and write decisions back through scoped ATS credentials, not through broad cloud roles. The ATS credentials themselves should be least-privilege in Bullhorn / JobAdder — read candidates and jobs, write back through the one resource they own (e.g. JobAdder's write_note scope), nothing else.
• CI deploys; the runtime doesn't. The identity that deploys (Cloud Build / your CI) is separate from the identity the service runs as. Deploy permissions never live in the running container.
• Rotate and audit. Rotate secrets on a schedule, and review the service account's bindings the same way you'd review who has keys to the office.

Least privilege isn't paranoia. It's deciding the size of tomorrow's incident, today.

AWS / Azure equivalents

• AWS: an IAM task role scoped to specific Secrets Manager ARNs (secretsmanager:GetSecretValue on named secrets), SQS ReceiveMessage/DeleteMessage on the one queue, and CloudWatch PutLogEvents. Deploy via a separate IAM principal.
• Azure: a managed identity with Key Vault get/list on named secrets only (via access policy or RBAC Key Vault Secrets User), Service Bus Receiver on the one queue, and the Monitoring Metrics/Logs publisher roles.

The principle survives the platform unchanged: the running service holds the fewest keys that let it do exactly its job — and it cannot deploy itself, read secrets it doesn't use, or reach data it has no business touching.

See also: the endpoint cheat-sheets, with the exact Bullhorn and JobAdder routes, auth flows, and fields the agents actually call.