Test Strategy
What Is Canary Deployment? A Plain-English Guide
Canary deployment sends a release to a small group first, so problems show up before everyone sees them. Here's how it works and when to use it.
Austin Meyer · · 7 min read
In this post
Canary deployment means releasing a new version of your software to a small share of users or servers before everyone else gets it. That way, problems show up on a few people instead of all of them. If something's broken, you find out while it's small and stop the rollout before most users notice. You'll also hear it called a canary release, canary testing, or a canary launch. Same idea, different names.
What Is Canary Deployment?
If your engineers say they'll canary deploy a release, this is what they mean. The new version goes to a small group first: a slice of your users, a few servers, one region, or your own employees. Everyone else stays on the current version while you watch the new one. If it holds up, you widen the rollout in stages until everyone has it. If it doesn't, the small group goes back to the old version.
Say you're shipping a rebuilt checkout page. You send a small share of traffic to the new page and leave everyone else on the old one. Then you compare the two: error rates, page speed, and how many shoppers finish an order. If the new page holds up, you widen the share until it's the only page left.
Danilo Sato's canary release article on Martin Fowler's site describes the same technique.
Why Is It Called Canary Deployment?
Canary deployment is named after the canaries coal miners carried underground to warn them about toxic gas. Canaries react to small amounts of carbon monoxide faster than people do. So a struggling canary was the miners' cue to get the eff out before the gas got to them.
The OG canaries were the real thing: miners used them from the late 1800s. The Science Museum Group credits the idea to physiologist John Scott Haldane. After investigating an 1896 explosion at Tylorstown Colliery, he proposed small, sensitive animals as early warning detectors. The museum holds a cage built to revive a canary: a sealable box with an oxygen tank, dated 1896 to 1986.
It's a grim job for a bird. The software version is kinder. Your first small group runs into any trouble before everyone else does, and then you roll back. No oxygen tank required.
How Canary Deployment Works, Step by Step
- Pick the change and who sees it first: a random share of users, one region, your own employees, or a set of servers.
- Decide what "healthy" looks like. Pick a few signals, like error rate, latency, crash rate, and support contacts, and write down where you'll stop.
- Send a small share of traffic, or a set of devices, to the new version.
- Compare the two versions side by side. Judge the new one against the old one running right now, not last week. Metrics drift with the time of day and the traffic.
- Widen in stages. If the signals hold, add more users and check again. Early stages lean on the clearest warning signs, like crashes and failed requests. Later stages build the confidence to judge subtler ones, like latency.
- Finish or roll back. If the signals stay healthy, everyone gets the new version. If they turn, pause the rollout and send users back to the old version, which never stopped running.
If you want the engineering depth, Google's SRE Workbook has a full chapter on canarying releases. And if you're on the product or QA side, ask engineering two things: which signals they're watching, and what makes them stop.
What a Rollout Looks Like
Here's one illustration, not a standard. Say you start at 1% of users and hold there for an hour, watching crashes and failed requests. If those stay clean, you move to 10%, then 25%, then 50%, then everyone, checking latency and support contacts along the way. If crashes spike at 10%, you roll back, and 90% of users never saw the problem.
Your real schedule depends on your traffic, your risk, and how fast problems show up. Treat these numbers as a sketch. Run one canary at a time, so you know which change caused what.
Canary vs. Blue-Green, Rolling, and Feature Flags
| Method | How it works | Best when |
|---|---|---|
| Canary | Sends the new version to a small share first, then widens in stages | You want evidence from real use before everyone is exposed |
| Blue-green | Two near-identical production environments, with traffic switched all at once | You want a clean cutover and a fast way back, and can run two environments |
| Rolling | Old instances replaced by new ones in batches | You need zero downtime and no second full environment |
| Feature flags | A switch that shows a feature to some users or all | You want to ship code now and choose who sees it later |
The line between a canary and a rolling update is intent. A rolling update keeps going as long as new instances start cleanly. A canary stops at a small share so you can judge the change before widening. Flags can pick your canary group. Our post on feature flagging in continuous customer testing shows how teams use them to test with real customers.
Benefits and Limits
What you get
- More confidence in each release, because a few real users have already run it.
- Major problems show up at small scale, affecting a few users instead of all of them.
- Less bad press. A problem that reaches a small group is far less likely to become a news story.
- A way back. The old version is still running, so a rollback means sending users back to it.
Where it falls short
You need enough traffic to get a signal. Rare problems hide in a small group, and a product with few users has little traffic to read.
You need good monitoring. If you can't compare error rates, latency, and crashes between the two versions, a canary is just a slow release.
The group has to look like your users. Employees on fast office networks tell you little about customers on older devices.
Both versions run at once, so shared data has to work with both. Sato's article recommends a parallel change to keep the database compatible with each.
And a canary can tell you that a change broke something. It can't tell you why people like or dislike it.
What Canary Does Not Tell You (and What It Does)
A canary watches machines and numbers. It'll tell you errors climbed or an app crashed more often. It won't tell you people found the new layout confusing, hunted for a button that moved, or understood the change and still didn't want it.
That part takes real testers: people from your target market who use the product in their own homes, offices, and devices, then tell you what they found. Beta testing puts a product in front of real users in real environments before release. Field testing watches how people use a finished product on their own. Delta Testing, the method we created, turns that work into loops you can plan, run, and measure, release after release.
If you ship updates to devices, a staged firmware rollout is a canary: the update reaches part of the fleet first. One open-source device-update framework, for example, starts each group only when enough of the previous group installed successfully. It halts the whole rollout past a set failure limit.
Numbers tell you whether the update installed and whether the device kept running. To learn whether people like what it changed, you need people living with it.
Tools That Support Canary Releases
We don't build deployment tooling, so this is a map of categories, not a buyer's guide. Four kinds of tool support canary deployments:
- Feature flag systems choose who sees a change and can switch it off without a new release.
- Traffic routing and load balancers split requests between the old and new versions by weight.
- Service meshes shift traffic between versions, service to service, in set percentages.
- Continuous delivery platforms run the stages for you and can compare the canary with the current version.
FAQ
Is canary deployment the same as canary testing?
In everyday use, yes. People say canary deployment, canary release, canary testing, and canary launch for the same practice. Google's SRE team defines canarying as a partial, time-limited deployment of a change plus an evaluation of it. So the word covers both the rollout and the judging.
One wrinkle: some monitoring tools use "canary" for a scheduled script that acts like a customer and checks a live site or API. That's synthetic monitoring, a different job.
How big should a canary group be?
There's no standard size, so I won't hand you a magic number. Make it big enough to be representative and to give you a signal you can read, and small enough that a failure stays contained. Traffic volume, how uniform your requests are, and how fast problems show up all move the answer. Google's SRE Workbook says a canary doesn't need to be very large. Start with the smallest group that can show the problems you're watching for, then widen it.
What is the difference between canary deployment and beta testing?
A canary release checks how the system holds up when a slice of real users gets a new version. Beta testing puts a pre-release product in front of chosen real users and asks what they think. A canary gives you metrics, and beta testing gives you feedback, so they answer different questions. The limited-release phase also adds users gradually, which can blur the line. Our post on what comes after beta testing explains where it fits.
A canary tells you whether a release broke. Real testers tell you whether it works for people. See how teams get both before launch. Request a demo.
