Zero-Downtime Deployments: A Practical Guide

When you're running systems that process financial transactions or healthcare data around the clock, "we'll deploy during the maintenance window" isn't always an option. You need to deploy new code without interrupting the running system.

I've implemented zero-downtime deployment strategies across multiple production systems. Here's what actually works.

The Rolling Deploy

The simplest approach: run multiple instances of your service behind a load balancer. Deploy new code to one instance at a time. The load balancer routes traffic to healthy instances while each instance restarts.

Requirements:

• At least two instances of every service
• Health check endpoints that the load balancer monitors
• Graceful shutdown that finishes in-flight requests before terminating

This handles 90% of deployments. The remaining 10% — the ones involving database changes — are where things get interesting.

Database Migrations Without Downtime

Database migrations are the hardest part of zero-downtime deployment. You can't just run ALTER TABLE on a production database that's actively serving requests.

The pattern I use: expand and contract.

Expand phase (deploy first):

1. Add the new column/table alongside the old one
2. Deploy code that writes to both old and new
3. Backfill existing data into the new structure
4. Deploy code that reads from the new structure

Contract phase (deploy second):

1. Deploy code that only writes to the new structure
2. Drop the old column/table

This is more work than a single migration. It's also the only way to change your schema without downtime.

Message Queue Considerations

If you're using message queues (and you should be), deployments need to account for in-flight messages.

Consumer graceful shutdown: When a consumer receives SIGTERM, it should stop accepting new messages, finish processing current ones, and then exit. Don't lose messages mid-processing.

Schema evolution in messages: When you change a message format, make the change backward-compatible. Add new fields, don't rename or remove existing ones. Consumers should ignore fields they don't recognize.

Deploy consumers before producers. If a new feature adds fields to a message, deploy the consumer that understands the new fields before deploying the producer that sends them.

Feature Flags

For complex deployments, feature flags are invaluable. Deploy the code, then gradually enable the new behavior:

1. Enable for internal users first
2. Enable for a small percentage of production traffic
3. Monitor error rates, latency, and business metrics
4. Gradually increase to 100%
5. Remove the flag once fully rolled out

The key discipline: remove flags after rollout. Feature flags that linger become a maintenance burden and a source of bugs.

Rollback Strategy

Every deployment needs a rollback plan. For code changes, this is straightforward — deploy the previous version. For database migrations, this is why the expand-and-contract pattern matters: during the expand phase, the old code still works, so rolling back is just a code deployment.

Test your rollback process. Not in theory — actually do it. In a non-production environment, deploy your change, then roll it back. Verify that everything returns to the previous state cleanly.

The Boring Truth

Zero-downtime deployments aren't technically exciting. They're a series of careful, boring decisions: health checks, graceful shutdowns, backward-compatible changes, phased rollouts.

But "boring" in infrastructure is a compliment. It means predictable. It means your team can deploy with confidence at 2 PM on a Tuesday instead of scheduling a midnight maintenance window. And that confidence compounds into faster iteration, less stress, and better software.