Concepts

CI/CD

Continuous Integration and Continuous Delivery/Deployment — practices that automate code integration, testing, and delivery to production. Foundation of modern software engineering.

evergreen#devops#automation#testing#dx

CI/CD are two complementary practices that automate the path from code commit to production. Together they eliminate "integration hell" and enable frequent releases with confidence.

Continuous Integration (CI)

Integrate code frequently (at least daily) to a shared branch, with automatic verification.

Principles

  1. Single repository — all code in one place
  2. Frequent commits — integrate small changes, don't accumulate
  3. Automated build — every commit triggers build + tests
  4. Fast tests — feedback in minutes, not hours
  5. Fix broken builds immediately — top priority

Typical CI pipeline

commit → build → lint → unit tests → integration tests → artifact
# GitHub Actions example
jobs:
  ci:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: 20
          cache: 'pnpm'
      - run: pnpm install --frozen-lockfile
      - run: pnpm lint
      - run: pnpm test
      - run: pnpm build

Continuous Delivery vs Continuous Deployment

Two distinct concepts often confused:

Continuous Delivery

Code can go to production at any time — deploy is manual but the process is automated.

CI → auto staging → manual approval → production

Continuous Deployment

Code goes to production automatically if it passes all checks — no human intervention.

CI → auto staging → auto production
AspectDeliveryDeployment
Deploy to productionManual (button)Automatic
Typical frequencyDaily/weeklyMultiple times per day
RequiresReliable testsTests + feature flags + monitoring
RiskLowerRequires maturity

Complete pipeline

┌─────────────────────────────────────────────────────────────────┐
│                        CI/CD Pipeline                           │
├─────────┬─────────┬─────────┬─────────┬─────────┬──────────────┤
│  Build  │  Test   │  Scan   │ Package │ Deploy  │   Monitor    │
├─────────┼─────────┼─────────┼─────────┼─────────┼──────────────┤
│ compile │ unit    │ SAST    │ docker  │ staging │ logs         │
│ lint    │ integ   │ DAST    │ helm    │ canary  │ metrics      │
│ deps    │ e2e     │ secrets │ artifact│ prod    │ alerts       │
└─────────┴─────────┴─────────┴─────────┴─────────┴──────────────┘

Stages explained

  1. Build — compile, verify syntax, resolve dependencies
  2. Test — unit, integration, e2e (testing pyramid)
  3. Scan — static security (SAST), dynamic (DAST), secrets
  4. Package — create deployable artifact (Docker image, binary, bundle)
  5. Deploy — staging, canary, production
  6. Monitor — post-deploy observability

Deployment strategies

Blue-Green

Two identical environments. Switch traffic instantly.

         ┌─────────┐
Users ───┤ Router  ├──► Blue (v1) ← active
         └────┬────┘
              └──────► Green (v2) ← standby/new

Rollback: switch router back to Blue.

Canary

Send small percentage of traffic to new version.

Users ──┬── 95% ──► v1 (stable)
        └── 5%  ──► v2 (canary)

Gradually increase if metrics are good.

Rolling

Update instances one by one.

[v1] [v1] [v1] [v1]  →  [v2] [v1] [v1] [v1]  →  [v2] [v2] [v2] [v2]

Key metrics (DORA)

The four DevOps Research and Assessment metrics:

MetricEliteHighMediumLow
Deployment frequencyOn-demand (multiple/day)Daily-weeklyWeekly-monthlyMonthly+
Lead time for changes< 1 hour1 day - 1 week1 week - 1 month1 month+
Change failure rate0-15%16-30%31-45%46%+
Time to restore< 1 hour< 1 day< 1 week1 week+

Tools by category

CategoryTools
CI/CD PlatformsGitHub Actions, GitLab CI, Jenkins, CircleCI, Travis CI
Artifact RegistryDocker Hub, GitHub Packages, AWS ECR, Google Artifact Registry
InfrastructureTerraform, Pulumi, AWS CDK, CloudFormation
KubernetesArgoCD, Flux, Helm, Kustomize
TestingJest, Playwright, Cypress, k6
SecuritySnyk, Trivy, SonarQube, OWASP ZAP
MonitoringDatadog, Grafana, Prometheus, New Relic

Anti-patterns

  • Slow tests — if CI takes 30+ minutes, developers avoid integrating
  • Flaky tests — randomly failing tests destroy confidence
  • Manual deploy — if it requires manual steps, work piles up
  • No rollback — every deploy must be revertible in minutes
  • Secrets in code — use secret managers, never hardcode
  • Ignoring alerts — alert fatigue leads to ignoring real problems

Why it matters

CI/CD is the most important velocity multiplier in software engineering. Without CI, bugs accumulate and merges are painful. Without CD, approved code waits days or weeks to reach production. DORA metrics consistently demonstrate that teams with mature CI/CD deliver faster and with greater stability.

References

  • Continuous Delivery — Jez Humble & David Farley, 2010. The book that defined the practice.
  • Accelerate — Nicole Forsgren, Jez Humble & Gene Kim, 2018. Scientific research on DORA metrics and high performance.
  • The DevOps Handbook — Gene Kim et al., 2021. Practical implementation guide.
  • Google SRE Book — Google, 2016. Site Reliability Engineering practices including CI/CD.
  • Martin Fowler on CI — Martin Fowler, 2006. Seminal article that popularized CI.
Concepts