I have opened a succeeded run, then opened the live site, more times than I want to count. The board is green. Release notes already went out. Support is already in a ticket. The pipeline did what we asked: it finished jobs. It did not prove the product.

That gap is not a YAML typo. The badge answered a different question than the one the user is asking.

What “Succeeded” actually means

A green job means the delivery system completed the steps you wrote. It does not mean:

• the same bits that passed staging are what production is running
• the process stayed up
• a customer can complete a request
• yesterday’s revision can be put back without a rebuild

Treat “Succeeded” as “the script finished.” Product health is a separate fact. If you debug YAML first, you often spend an hour on the wrong file.

Three identities that must match

Before anyone rewrites a pipeline, write one sentence:

This artifact is on this target, the process is running, this user path fails at this hop.

If you cannot write that sentence, you are still locating the release.

1. The artifact

Staging built once. Production rebuilt from the same commit. Restore, package feeds, base images, and even the agent image can differ between those two compiles. You did not promote a revision. You compiled a cousin and called it a release.

Name the artifact ID or image digest that went out — not “the latest run.” Compare it to what staging ran. If they differ, stop talking about “the same release.”

2. The target

Slot, namespace, App Service, VM, traffic manager, ingress host. Staging success plus production traffic on the old bits is a common mystery. Swap skipped. Hostname still on the previous slot. Environment name in YAML is a label, not a control.

If production is only a stage display name — no Azure DevOps Environment, no approval, no lock, no check on the service connection — anyone who can queue the pipeline can reach production.

3. The user path

A /health 200 from inside the job is not a customer completing a request. After deploy, hit a real URL with the same kind of identity a user would use: login, one write, one read. If that job is not allowed to fail the release, you will keep shipping green badges onto broken apps.

Dependencies sit here too: database firewall, Key Vault, DNS, private endpoint, an API you do not own. The pipeline cannot see those unless you added a smoke check that can.

What I change first

The rest can wait a week. These four changes stop the most common green-but-broken night:

1. Build once. Store the artifact. Every later stage consumes that revision.
2. Put production behind an Environment with an approval and a lock.
3. Add one smoke job against a real URL, allowed to fail the release.
4. Add a rollback job that redeploys the previous artifact without rebuilding.

If rollback needs a person who is asleep, you do not have rollback. You have hope.

Secrets belong in a store with a trail, not in YAML, chat, or a variable group that everyone on the project can read. Rotation that is painful is rotation that does not happen. That is a separate week of work. Do not let it block the four items above.

Terraform is the same pattern

People argue about HCL when the argument is in the wrong file. The .tf files are intent. State is what Terraform believes exists. A plan that looks clean on a laptop can still be a bad night in CI if the backend key is shared across environments or state lived on one machine.

Same rule as artifacts: one identity, one place, promote it. Do not rebuild a cousin and call it the same environment.

A Monday check

Pick the last production run. Ask, in order:

1. What artifact ID or image digest went out?
2. Which Environment and service connection deployed it?
3. What smoke ran against a real URL?
4. How would we put yesterday’s artifact back, using this same pipeline, without rebuilding?

If any answer is “we think so,” stop treating the pipeline as a release system. It is a script with a green badge.

If you only fix one thing this week: build once, store the artifact, promote that same revision. Everything else in CI/CD gets easier after that identity exists.

I write about production delivery problems at ItsCloudHub.