Progressive delivery on Kubernetes with Argo Rollouts

Most teams meet the limits of a plain Kubernetes Deployment at the worst possible moment: mid-rollout, watching a new version march out to every pod while a graph slowly turns the wrong colour. A Deployment rolls pods out carefully enough — it respects readiness, it surges and drains in order — but it has one stubborn blind spot. Once the new ReplicaSet is Ready, it keeps going unless something stops it. There is no built-in way to say “send 10% of traffic here, wait, check an SLO, then continue”, or “only promote this preview stack if the metrics hold up”.

That missing control loop is exactly what Argo Rollouts adds. It keeps the Deployment mental model you already know and layers canary, blue-green, analysis, and automated rollback on top. For platform teams, that quietly turns rollout safety from a runbook someone has to remember into a controller that does it the same way every time. As of Argo Rollouts v1.9.0, that controller can drive percentage-based canaries, blue-green cutovers, and metric-gated promotion without asking you to rip out the rest of your Kubernetes delivery stack.

The gap in a normal Deployment

A Deployment is genuinely good at one thing: swapping one ReplicaSet for another while keeping enough healthy pods around to serve traffic. Readiness probes, maxUnavailable, and maxSurge all pull their weight here. What none of them give you is a first-class sense of whether the new version is actually any good.

That distinction matters most when “pods are Ready” and “this version is safe” quietly stop meaning the same thing. A service can start up cleanly and still come apart under real load. Error rate can sit flat through the first few requests and then climb once traffic arrives in earnest. Latency can drift at the 95th percentile while the average stays there looking reassuring. And a bad release often hurts just one path through the app — which is cold comfort when a straight rollout has already reached every user before anyone has the evidence to hit stop.

Argo Rollouts lives precisely in that gap. It is not a replacement for the Kubernetes scheduler, and it is not a GitOps controller — it does not want either job. It is a rollout controller that takes over the update strategy for a single workload and adds the parts a Deployment never had: promotion steps, pause points, analysis runs, and traffic-shaping integrations.

The mental model: Deployment behaviour, but with a promotion engine

The easiest way to hold Argo Rollouts in your head is “a Deployment with a state machine bolted on”. The Rollout resource still owns ReplicaSets and still reacts to pod template changes, but on top of that it keeps track of where a release currently sits in its promotion flow.

At a high level, the loop runs like this:

1. A new pod template lands in Git and is applied to the cluster.
2. The Rollout controller creates a new ReplicaSet.
3. The strategy decides what happens next: shift some traffic, create a preview stack, pause, or run analysis.
4. If the checks pass, the controller promotes the release.
5. If the checks fail, the controller aborts and keeps the stable version serving traffic.

That separation is really the whole idea. Kubernetes carries on running the pods, exactly as it did before. Argo Rollouts is the part that decides whether a new version has earned more traffic yet.

The pieces that matter

As of v1.9.0, the CRDs worth genuinely understanding are Rollout, AnalysisTemplate, ClusterAnalysisTemplate, AnalysisRun, and Experiment. That list looks longer than it needs to be, though — in day-to-day work most teams spend nearly all their time in the first three.

Rollout

The Rollout (apiVersion: argoproj.io/v1alpha1, kind: Rollout) sits at the centre of everything. It is deliberately built to be a drop-in replacement for a Deployment, which is why migrating an existing workload is mostly a matter of changing the apiVersion, the kind, and the strategy — the pod template underneath can stay much as it was.

A Rollout can either carry its own pod template or point at an existing Deployment through workloadRef. That second option is the gentle on-ramp: it lets you bring a live workload under Rollouts gradually, rather than rewriting the whole object in one nervous commit.

AnalysisTemplate and AnalysisRun

AnalysisTemplate describes the check; AnalysisRun is that check actually running during a rollout. The template might ask Prometheus for a success rate, a latency figure, or some service-specific business metric that only your team would think to measure. The run takes the answer and decides whether the rollout carries on, pauses, or aborts.

This is the part that makes Argo Rollouts something more than “rolling updates, but slower”. The controller is not simply counting down a timer between steps — it is waiting for evidence before it commits.

Canary vs blue-green

Argo Rollouts supports both, and it helps to remember that they are answers to different questions.

• Canary is about gradual exposure. Raise the traffic a step at a time, watch what happens, then decide whether to keep going.
• Blue-green is about a clean cutover. Stand the new version up beside the old one, satisfy yourself it works, then flip the active service across.

Neither is universally the right answer. Canary tends to be the better default for customer-facing services, where the whole point is for a bad version to touch as little traffic as possible before you catch it. Blue-green comes into its own when you want an obvious stable-versus-preview split and the reassurance of a fast switch back.

A concrete canary with analysis

The heart of a real Argo Rollouts setup is rarely the sheer volume of YAML — it is the promotion logic hiding inside it. The example below is deliberately small: a canary that shifts traffic in stages and refuses to promote unless a Prometheus-backed success-rate check agrees.

First, install the controller:

kubectl create namespace argo-rollouts
kubectl apply -n argo-rollouts -f https://github.com/argoproj/argo-rollouts/releases/latest/download/install.yaml

Then define the analysis template:

apiVersion: argoproj.io/v1alpha1
kind: AnalysisTemplate
metadata:
  name: checkout-success-rate
spec:
  args:
    - name: service-name
  metrics:
    - name: success-rate
      interval: 5m
      successCondition: result[0] >= 0.99
      failureLimit: 3
      provider:
        prometheus:
          address: http://prometheus.monitoring.svc.cluster.local:9090
          query: |
            sum(
              rate(http_requests_total{service="{{args.service-name}}",status!~"5.."}[5m])
            ) /
            sum(
              rate(http_requests_total{service="{{args.service-name}}"}[5m])
            )

Then wire it into the rollout:

apiVersion: argoproj.io/v1alpha1
kind: Rollout
metadata:
  name: checkout
spec:
  replicas: 8
  revisionHistoryLimit: 2
  selector:
    matchLabels:
      app: checkout
  template:
    metadata:
      labels:
        app: checkout
    spec:
      containers:
        - name: checkout
          image: ghcr.io/example/checkout:v2
          ports:
            - containerPort: 8080
  strategy:
    canary:
      steps:
        - setWeight: 10
        - pause:
            duration: 5m
        - analysis:
            templates:
              - templateName: checkout-success-rate
            args:
              - name: service-name
                value: checkout
        - setWeight: 30
        - pause:
            duration: 10m
        - setWeight: 100

The interesting part here is not the syntax — it is the sequence.

At 10% traffic, the rollout deliberately pauses long enough to gather some real request data. The AnalysisRun then measures the service against a threshold you wrote down in advance, and if that metric fails three times it aborts, rather than quietly waving the release on to 30% and then to everyone.

There is one subtlety worth internalising early. Without a traffic-routing integration, setWeight is an approximation drawn from ReplicaSet sizes rather than exact request-level shaping. At low replica counts that is often good enough, but it is not the same thing as a true 10% split. When you genuinely need that precision, wire Rollouts up to a traffic provider and give it stableService, canaryService, and trafficRouting. That is the line between “best effort, by pod count” and “deliberate, by request path”.

Where blue-green is simpler

Blue-green is the model most teams find easiest to explain out loud, mostly because it has fewer moving parts. There is an active stack, a preview stack, and a single moment of promotion between them.

In Rollouts, that shape becomes an activeService carrying live traffic and, optionally, a previewService pointed at the new ReplicaSet before it goes live. The part that makes it genuinely interesting is what you can hang either side of the cutover with prePromotionAnalysis and postPromotionAnalysis:

• prePromotionAnalysis has to be satisfied that the preview version is good enough before the switch happens at all.
• postPromotionAnalysis keeps watching once traffic has moved across, and can send it back to the old version if the new one starts to degrade.

The field I would commit to memory here is scaleDownDelaySeconds. It defaults to 30 seconds, giving the service selectors and traffic infrastructure time to converge before the old ReplicaSet is scaled away. It sounds like a trivial setting, but it is exactly the sort of thing that separates a rollout which “worked” as far as the controller was concerned from one that quietly dropped traffic down in the dataplane.

How this fits with GitOps

Argo Rollouts and Argo CD get along well, but it is worth being clear that they are doing two different jobs.

Argo CD reconciles the desired state in Git into the cluster. Argo Rollouts decides how a workload that has just changed should be promoted, once that change has landed. The boundary between them is clean, and that cleanness is the point.

In practice it means you do not have to smuggle rollout logic into your CI pipelines or bend your GitOps repo layout around it. Git still holds the desired pod template, Argo CD still applies it, and Rollouts takes over only for the runtime question of whether the new version deserves to keep advancing.

It also means rollback here is operational rather than declarative — a distinction that trips people up more often than you would expect. When a rollout aborts, Argo Rollouts does not go and rewrite Git on your behalf. It scales the release back to the stable version in the cluster, but the repository still holds the bad image tag until someone, or another controller, reverts it. That gap has teeth: if Argo CD is running with auto-sync and self-heal, it can read the aborted rollout as simply out of sync and try to re-apply the very version that just failed. Teams new to the model tend to walk straight past that boundary, and only notice it the first time the two controllers start pulling against each other.

Production reality: the sharp edges

Argo Rollouts demos beautifully and is surprisingly easy to misuse once real traffic is involved. The failure modes that actually bite are rarely exotic — they are the boring, avoidable ones.

Canary percentages are only exact if your traffic layer is exact

A setWeight: 10 step reads as though it means precisely ten percent, but without traffic routing it is really just a pod-count approximation. If your service runs seven replicas, there is no way for the controller to carve traffic into a clean tenth. When exact exposure genuinely matters, reach for a supported traffic manager rather than trusting the arithmetic.

Traffic-routed canaries can overprovision

The moment you add trafficRouting, the replica arithmetic shifts under you. maxSurge and maxUnavailable are no longer the whole story, particularly once setCanaryScale is in the mix, and it is easy to find yourself running more stable and canary pods at once than you had budgeted for during a promotion. That is usually the right trade for safety — just make sure it is a trade you planned for, rather than one you discover on the cluster.

Blue-green is simple, but not cheap

Running blue and green side by side doubles parts of your footprint by design — that is the whole mechanism, not a bug. If you lean on preview environments heavily, keep them short-lived. Rollouts is built for promotion windows measured in minutes, not for quietly maintaining a second long-lived environment that drifts further from the first with every passing day.

HPA needs to target the Rollout

If the workload is autoscaled, point the HPA at the Rollout itself — not the Deployment you migrated from, and certainly not an individual ReplicaSet. Get this wrong and you end up refereeing two controllers that hold different opinions about who owns the replica count, usually at the least convenient moment.

When Argo Rollouts is the right call

Reach for Argo Rollouts when the cost of a bad release is high enough that “the pods became Ready” simply is not a strong enough gate to stand behind. In practice that means customer-facing APIs, services with SLOs you actually measure, and teams who already have decent metrics but are still making the promotion call by hand, one deploy at a time.

What it is not is a decoration. Adopting it to make a deployment diagram look more sophisticated is a poor reason on its own. If a service is low-risk, its traffic layer is simple, and rollback is already fast, a plain Deployment may well remain the honest choice.

The real win was never that Rollouts hands Kubernetes yet another CRD. It is that it gives your release process a proper control loop: expose a little, measure what happens, then decide. That single step — pausing to look before committing — is the one a plain Deployment has never been able to take on its own.