Your Deploys Weren't Built for This Stage: Fixing CI/CD as Your Team Grows

The symptoms

Releases get batched into big, nervous deploys because each one is painful. A red build blocks everyone, and nobody is sure whose change broke it. Only one or two people really know how to release. Staging is broken or out of date more often than not, and urgent fixes skip the pipeline "just this once".

None of this means the team is doing something wrong. The process was designed for a smaller team, and nobody owned redesigning it as the team grew.

Make the pipeline the only way to production

If people can deploy from their laptops or patch servers by hand, every other improvement leaks. Give production credentials to the pipeline rather than to people, and make the pipeline the fastest way to ship, including for hotfixes. When the safe path is also the quick path, people take it.

Keep the main branch releasable

Long-lived branches turn every merge into a small project. Short-lived branches merged into main at least daily, with feature flags to separate deploying code from releasing a feature, keep changes small and conflicts rare. If your branching strategy needs a diagram to explain, it's probably costing more than it protects.

Fast feedback first, slow checks later

Order the pipeline so the cheap checks (build, lint, unit tests) fail within minutes, run independent jobs in parallel, and cache dependencies. Slower end-to-end suites can run against a deployed test environment instead of holding up every commit.

Flaky tests deserve special attention: once people learn to hit "retry", they stop trusting red builds. Quarantine flaky tests, give each one an owner, and fix or delete it.

Environments you can rebuild from code

When environments are built by hand, staging drifts away from production and nobody wants to touch either. Defining them with infrastructure-as-code (Terraform, for example) makes staging match production, makes changes reviewable like any other code, and makes a new environment a routine job rather than a week of work. Once that's in place, short-lived preview environments per branch become an option worth weighing.

Small, boring releases

Deploy small changes often, with health checks and an automated way back: a rollback, or a flag you can switch off. To know whether things are improving, track the four DORA measures over time: how often you deploy, how long a change takes to reach production, how often a deploy causes a problem, and how long it takes to recover. Trends matter more than any single number.

Where to start

Don't try to fix everything at once. Pick the problem that costs the team the most this month, whether that's slow builds, a fragile staging environment or a release only one person can run, and fix that properly. Then pick the next one. If you'd like an outside view of where to start, that's exactly what my 2-week review covers.