CI/CD - Continuous Integration and Continuous Deployment

Introduction

Continuous Integration and Continuous Deployment (CI/CD) represent a set of practices that enable development teams to deliver code changes more frequently and reliably. CI/CD bridges the gap between development and operations teams by automating the build, test, and deployment processes. This automation reduces manual errors, accelerates time-to-market, and increases confidence in the software being delivered.

The core benefits of CI/CD include:

Faster feedback loops: Developers discover issues within minutes rather than days
Reduced risk: Small, frequent deployments are easier to troubleshoot than large releases
Improved quality: Automated testing catches bugs before they reach production
Increased productivity: Automation frees developers to focus on writing code
Better collaboration: Standardized processes improve team alignment

This chapter explores CI/CD fundamentals, implementation strategies, best practices, and common pitfalls to avoid.

Understanding CI/CD

CI/CD is often mentioned as a single concept, but it actually consists of three distinct practices that work together:

Continuous Integration (CI)

Continuous Integration is the practice of automatically building and testing code changes as they are committed to version control. The goal is to detect integration issues early and provide rapid feedback to developers.

Key principles:

Commit code frequently (at least daily)
Every commit triggers an automated build
Automated tests run with every build
Build failures are fixed immediately
The main branch is always in a deployable state

Continuous Delivery (CD)

Continuous Delivery extends CI by ensuring that code is always in a releasable state. While deployments to production still require manual approval, the deployment process itself is automated and can happen at any time.

Key principles:

Automated deployment to staging/pre-production environments
Manual approval gate before production deployment
Deployment process is repeatable and reliable
Configuration is version-controlled and environment-specific

Continuous Deployment

Continuous Deployment takes Continuous Delivery one step further by automatically deploying every change that passes all stages of the production pipeline. There is no manual approval gate.

Key principles:

Every change that passes automated tests goes to production
Requires high confidence in automated testing
Extensive monitoring and quick rollback capabilities
Feature flags to control feature visibility

CI/CD Pipeline Architecture

A typical CI/CD pipeline consists of several stages that code changes must pass through before reaching production:

Pipeline Stages Explained

1. Source Stage

The pipeline begins when a developer commits code to version control (Git). Modern CI/CD systems support various triggers:

Push to specific branches (main, develop, feature/*)
Pull request creation or update
Tag creation for releases
Scheduled runs (nightly builds)

2. Build Stage

The build stage compiles source code, resolves dependencies, and creates executable artifacts.

Example (Node.js project):

# .github/workflows/ci.yml
name: CI Pipeline

on:
  push:
    branches: [main, develop]
  pull_request:
    branches: [main]

jobs:
  build:
    runs-on: ubuntu-latest

    steps:
      - name: Checkout code
        uses: actions/checkout@v3

      - name: Setup Node.js
        uses: actions/setup-node@v3
        with:
          node-version: '18'
          cache: 'npm'

      - name: Install dependencies
        run: npm ci

      - name: Build application
        run: npm run build

      - name: Upload build artifacts
        uses: actions/upload-artifact@v3
        with:
          name: build-artifacts
          path: dist/

3. Test Stage

Multiple layers of testing ensure code quality and correctness:

test:
  needs: build
  runs-on: ubuntu-latest

  steps:
    - name: Checkout code
      uses: actions/checkout@v3

    - name: Setup Node.js
      uses: actions/setup-node@v3
      with:
        node-version: '18'

    - name: Install dependencies
      run: npm ci

    - name: Run unit tests
      run: npm test -- --coverage

    - name: Run integration tests
      run: npm run test:integration

    - name: Upload coverage reports
      uses: codecov/codecov-action@v3

4. Quality Gates

Quality gates enforce standards before code can proceed:

Code Coverage: Minimum percentage of code covered by tests (e.g., 80%)
Code Quality: Static analysis scores (SonarQube, ESLint)
Security Vulnerabilities: No critical or high severity issues
Performance Benchmarks: No significant performance regressions

5. Deployment Stages

Deployments typically follow a progression through environments:

CI/CD Best Practices

1. Keep Builds Fast

Slow builds discourage frequent commits and delay feedback. Optimize your pipeline:

// Good: Run tests in parallel
describe('User Service Tests', () => {
  // Use beforeAll for expensive setup
  beforeAll(async () => {
    await setupTestDatabase();
  });

  // Tests run independently
  it('creates user', async () => {
    // Test implementation
  });

  it('updates user', async () => {
    // Test implementation
  });
});

Optimization strategies:

Run tests in parallel
Use build caching aggressively
Split long-running test suites
Use incremental builds when possible
Limit unnecessary dependencies

Target build times:

Unit tests: < 5 minutes
Integration tests: < 10 minutes
Full pipeline: < 20 minutes

2. Fail Fast

Configure your pipeline to fail as soon as a problem is detected:

jobs:
  quick-checks:
    runs-on: ubuntu-latest
    steps:
      - name: Lint code
        run: npm run lint

      - name: Type check
        run: npm run type-check

      - name: Run unit tests
        run: npm test

  # Only run expensive tests if quick checks pass
  integration-tests:
    needs: quick-checks
    runs-on: ubuntu-latest
    steps:
      - name: Run integration tests
        run: npm run test:integration

3. Make Pipelines Reproducible

Ensure builds produce consistent results regardless of when or where they run:

# Good: Pin specific versions
FROM node:18.17.0-alpine

# Use lock files for deterministic dependencies
COPY package-lock.json .
RUN npm ci --only=production

# Bad: Using latest versions
# FROM node:latest
# RUN npm install

Reproducibility checklist:

✅ Pin dependency versions in lock files
✅ Use specific versions of base images
✅ Version control all configuration
✅ Use environment variables for configuration
✅ Avoid depending on external resources without caching

4. Implement Proper Secret Management

Never commit secrets to version control. Use dedicated secret management:

# GitHub Actions example
deploy:
  runs-on: ubuntu-latest
  steps:
    - name: Deploy to AWS
      env:
        AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
        AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
      run: |
        aws s3 sync ./build s3://my-bucket/

Secret management best practices:

Use platform-provided secret stores (GitHub Secrets, AWS Secrets Manager)
Rotate secrets regularly
Use IAM roles and service accounts when possible
Audit secret access
Never log secrets in build output

5. Monitor and Alert

Implement comprehensive monitoring for your CI/CD pipeline:

# Send notifications on failure
notify:
  runs-on: ubuntu-latest
  if: failure()
  steps:
    - name: Send Slack notification
      uses: 8398a7/action-slack@v3
      with:
        status: ${{ job.status }}
        text: 'Pipeline failed for ${{ github.repository }}'
        webhook_url: ${{ secrets.SLACK_WEBHOOK }}

What to monitor:

Build success/failure rates
Build duration trends
Deployment frequency
Mean time to recovery (MTTR)
Test flakiness
Pipeline costs

6. Implement Progressive Deployment Strategies

Deploy changes gradually to minimize risk:

Blue-Green Deployment

Canary Deployment

Feature Flags

// Good: Use feature flags for gradual rollout
import { isFeatureEnabled } from './featureFlags';

function renderDashboard(user) {
  if (isFeatureEnabled('new-dashboard', user)) {
    return <NewDashboard user={user} />;
  }
  return <LegacyDashboard user={user} />;
}

// Configuration-based rollout
const featureFlags = {
  'new-dashboard': {
    enabled: true,
    rolloutPercentage: 10, // 10% of users
    allowlist: ['beta-tester@company.com'],
  },
};

See the Feature Flags chapter for more details.

7. Automate Database Migrations

Include database changes in your deployment pipeline:

// Good: Version-controlled migration
// migrations/20240101_add_user_preferences.js
module.exports = {
  async up(db) {
    await db.schema.createTable('user_preferences', (table) => {
      table.increments('id').primary();
      table.integer('user_id').notNullable().references('users.id');
      table.jsonb('preferences').defaultTo('{}');
      table.timestamps(true, true);
    });
  },

  async down(db) {
    await db.schema.dropTable('user_preferences');
  },
};

Migration best practices:

Always write reversible migrations (up and down)
Test migrations on production-like data
Run migrations before deploying application code
Monitor migration performance
Have a rollback plan

8. Implement Automated Rollbacks

Configure automatic rollbacks when deployments fail health checks:

deploy:
  runs-on: ubuntu-latest
  steps:
    - name: Deploy new version
      id: deploy
      run: ./deploy.sh v2.0.0

    - name: Wait for health checks
      run: |
        for i in {1..30}; do
          if curl -f https://api.example.com/health; then
            echo "Health check passed"
            exit 0
          fi
          sleep 10
        done
        echo "Health check failed"
        exit 1

    - name: Rollback on failure
      if: failure()
      run: ./deploy.sh v1.9.0

CI/CD Tools and Platforms

Popular CI/CD Platforms

Platform	Best For	Pricing Model
GitHub Actions	GitHub repositories, simple workflows	Free tier + usage-based
GitLab CI/CD	Integrated DevOps platform	Free tier + subscription
Jenkins	Self-hosted, highly customizable	Free (self-hosted)
CircleCI	Fast builds, Docker support	Free tier + usage-based
Travis CI	Open source projects	Free for OSS
Azure DevOps	Microsoft ecosystem	Free tier + usage-based
AWS CodePipeline	AWS infrastructure	Usage-based
Bitbucket Pipelines	Atlassian ecosystem	Free tier + usage-based

Choosing the Right Platform

Consider these factors when selecting a CI/CD platform:

Integration with your version control: Native integration (GitHub Actions with GitHub) provides the best experience
Runner options: Cloud-hosted vs. self-hosted runners
Pricing: Free tier limits, cost at scale
Features: Built-in secrets management, caching, matrix builds
Performance: Build speed, concurrent jobs
Ecosystem: Available actions/plugins, community support

Common CI/CD Pitfalls

1. Testing in Production

// Bad: Relying on production for testing
function deployToProduction() {
  // No staging environment testing
  deploy('production');
  // Hope everything works!
}

// Good: Test in staging first
async function deployWithVerification() {
  await deploy('staging');
  await runSmokeTests('staging');
  await runE2ETests('staging');

  if (allTestsPassed) {
    await deploy('production');
    await monitorHealthMetrics();
  }
}

2. Flaky Tests

Flaky tests that pass and fail intermittently erode trust in the CI/CD pipeline:

// Bad: Test depends on timing
it('loads data', async () => {
  fetchData();
  await new Promise((resolve) => setTimeout(resolve, 1000)); // Hope 1s is enough
  expect(data).toBeDefined();
});

// Good: Wait for actual condition
it('loads data', async () => {
  const data = await fetchData();
  expect(data).toBeDefined();
});

Addressing flaky tests:

Identify and quarantine flaky tests
Fix or delete unreliable tests
Use retry logic sparingly (only for known external issues)
Make tests deterministic and independent

3. Long-Running Builds

# Bad: Sequential execution
jobs:
  build:
    runs-on: ubuntu-latest
    steps:
      - run: npm run lint        # 2 min
      - run: npm test            # 10 min
      - run: npm run test:e2e    # 20 min

# Good: Parallel execution
jobs:
  lint:
    runs-on: ubuntu-latest
    steps:
      - run: npm run lint        # 2 min

  test:
    runs-on: ubuntu-latest
    steps:
      - run: npm test            # 10 min

  test-e2e:
    runs-on: ubuntu-latest
    steps:
      - run: npm run test:e2e    # 20 min

4. Inadequate Rollback Procedures

// Bad: No rollback capability
function deploy(version) {
  deleteOldVersion();
  deployNewVersion(version);
  // If deployment fails, we're in trouble
}

// Good: Keep previous version for quick rollback
function deploy(version) {
  const previousVersion = getCurrentVersion();

  try {
    deployNewVersion(version);
    await verifyHealthChecks();
    markAsStable(version);
  } catch (error) {
    console.error('Deployment failed, rolling back');
    rollbackToVersion(previousVersion);
    throw error;
  }
}

5. Ignoring Security Scanning

# Good: Include security scanning in pipeline
security:
  runs-on: ubuntu-latest
  steps:
    - name: Checkout code
      uses: actions/checkout@v3

    - name: Run dependency vulnerability scan
      run: npm audit --audit-level=moderate

    - name: Run SAST scanner
      uses: github/codeql-action/analyze@v2

    - name: Scan Docker images
      run: docker scan myapp:latest

6. Not Version Controlling Pipeline Configuration

# Good: Keep pipeline configuration in version control
git add .github/workflows/ci.yml
git add .gitlab-ci.yml
git add Jenkinsfile

# Version control everything needed to build and deploy
git add Dockerfile
git add docker-compose.yml
git add kubernetes/

CI/CD Metrics and KPIs

Track these metrics to measure the effectiveness of your CI/CD pipeline:

Deployment Frequency

How often you deploy to production. High-performing teams deploy multiple times per day.

// Track deployment frequency
const deployments = await getDeployments({
  startDate: '2024-01-01',
  endDate: '2024-01-31',
  environment: 'production',
});

const deploymentsPerDay = deployments.length / 31;
console.log(`Deployment frequency: ${deploymentsPerDay.toFixed(2)} per day`);

Lead Time for Changes

Time from code commit to production deployment. Measures agility.

const leadTime = productionDeploymentTime - commitTime;
// Target: < 24 hours for high performers

Mean Time to Recovery (MTTR)

How quickly you can recover from a failed deployment.

const mttr = recoveryTime - incidentDetectionTime;
// Target: < 1 hour for high performers

Change Failure Rate

Percentage of deployments that cause a failure in production.

const changeFailureRate = (failedDeployments / totalDeployments) * 100;
// Target: < 15% for high performers

Real-World CI/CD Pipeline Example

Here's a complete GitHub Actions workflow for a Node.js application:

name: Complete CI/CD Pipeline

on:
  push:
    branches: [main, develop]
  pull_request:
    branches: [main]

env:
  NODE_VERSION: '18'
  REGISTRY: ghcr.io
  IMAGE_NAME: ${{ github.repository }}

jobs:
  # Stage 1: Quick validation
  validate:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3

      - name: Setup Node.js
        uses: actions/setup-node@v3
        with:
          node-version: ${{ env.NODE_VERSION }}
          cache: 'npm'

      - name: Install dependencies
        run: npm ci

      - name: Lint code
        run: npm run lint

      - name: Type check
        run: npm run type-check

      - name: Check formatting
        run: npm run format:check

  # Stage 2: Build and test
  test:
    needs: validate
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3

      - name: Setup Node.js
        uses: actions/setup-node@v3
        with:
          node-version: ${{ env.NODE_VERSION }}
          cache: 'npm'

      - name: Install dependencies
        run: npm ci

      - name: Run unit tests
        run: npm test -- --coverage

      - name: Upload coverage
        uses: codecov/codecov-action@v3
        with:
          files: ./coverage/coverage-final.json

  # Stage 3: Integration tests
  integration-test:
    needs: validate
    runs-on: ubuntu-latest
    services:
      postgres:
        image: postgres:15
        env:
          POSTGRES_PASSWORD: postgres
        options: >-
          --health-cmd pg_isready --health-interval 10s --health-timeout 5s --health-retries 5
    steps:
      - uses: actions/checkout@v3

      - name: Setup Node.js
        uses: actions/setup-node@v3
        with:
          node-version: ${{ env.NODE_VERSION }}
          cache: 'npm'

      - name: Install dependencies
        run: npm ci

      - name: Run database migrations
        env:
          DATABASE_URL: postgresql://postgres:postgres@localhost:5432/test
        run: npm run migrate

      - name: Run integration tests
        env:
          DATABASE_URL: postgresql://postgres:postgres@localhost:5432/test
        run: npm run test:integration

  # Stage 4: Security scanning
  security:
    needs: validate
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3

      - name: Run dependency audit
        run: npm audit --audit-level=moderate

      - name: Run Snyk security scan
        uses: snyk/actions/node@master
        env:
          SNYK_TOKEN: ${{ secrets.SNYK_TOKEN }}

  # Stage 5: Build Docker image
  build:
    needs: [test, integration-test, security]
    runs-on: ubuntu-latest
    permissions:
      contents: read
      packages: write
    steps:
      - uses: actions/checkout@v3

      - name: Log in to Container Registry
        uses: docker/login-action@v2
        with:
          registry: ${{ env.REGISTRY }}
          username: ${{ github.actor }}
          password: ${{ secrets.GITHUB_TOKEN }}

      - name: Extract metadata
        id: meta
        uses: docker/metadata-action@v4
        with:
          images: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}

      - name: Build and push Docker image
        uses: docker/build-push-action@v4
        with:
          context: .
          push: true
          tags: ${{ steps.meta.outputs.tags }}
          labels: ${{ steps.meta.outputs.labels }}

  # Stage 6: Deploy to staging
  deploy-staging:
    needs: build
    if: github.ref == 'refs/heads/develop'
    runs-on: ubuntu-latest
    environment:
      name: staging
      url: https://staging.example.com
    steps:
      - name: Deploy to staging
        run: |
          echo "Deploying to staging environment"
          # Deployment commands here

      - name: Run smoke tests
        run: |
          curl -f https://staging.example.com/health || exit 1

  # Stage 7: Deploy to production
  deploy-production:
    needs: build
    if: github.ref == 'refs/heads/main'
    runs-on: ubuntu-latest
    environment:
      name: production
      url: https://example.com
    steps:
      - name: Deploy to production
        run: |
          echo "Deploying to production environment"
          # Deployment commands here

      - name: Wait for health checks
        run: |
          for i in {1..30}; do
            if curl -f https://example.com/health; then
              echo "Health check passed"
              exit 0
            fi
            sleep 10
          done
          exit 1

      - name: Notify team
        if: always()
        uses: 8398a7/action-slack@v3
        with:
          status: ${{ job.status }}
          text: 'Production deployment ${{ job.status }}'
          webhook_url: ${{ secrets.SLACK_WEBHOOK }}

Getting Started with CI/CD

If you're new to CI/CD, follow this roadmap:

Phase 1: Basic CI (Week 1-2)

Set up version control (Git)
Choose a CI platform (GitHub Actions recommended for beginners)
Create a basic build pipeline
Add linting and unit tests
Configure build status badges

Phase 2: Enhanced Testing (Week 3-4)

Add integration tests
Implement code coverage tracking
Add security scanning
Set up quality gates
Configure notifications

Phase 3: Continuous Delivery (Week 5-8)

Create staging environment
Automate deployment to staging
Add smoke tests after deployment
Implement manual approval for production
Document deployment procedures

Phase 4: Continuous Deployment (Week 9-12)

Implement feature flags
Add comprehensive monitoring
Set up automatic rollbacks
Remove manual approval gates
Implement canary or blue-green deployments

Conclusion

CI/CD is not just about automation—it's about creating a culture of continuous improvement and rapid feedback. A well-designed CI/CD pipeline enables teams to deliver value faster, with higher quality and lower risk.

Key Takeaways

✅ Start simple: Begin with basic CI, then gradually add more sophisticated practices ✅ Fail fast: Configure pipelines to catch issues early and provide rapid feedback ✅ Automate everything: From builds to tests to deployments—automation reduces errors ✅ Monitor continuously: Track metrics and pipeline performance to identify improvements ✅ Iterate and improve: Regularly review and optimize your CI/CD processes ✅ Security first: Integrate security scanning throughout your pipeline ✅ Plan for failure: Implement robust rollback and recovery procedures

Remember: The goal of CI/CD is to reduce the time between writing code and delivering value to users, while maintaining high quality and minimizing risk. Start with the basics, measure your progress, and continuously improve.

Testing: Comprehensive guide to testing strategies
Branch-Based Testing: Testing in ephemeral environments
Branching Strategies: Git workflow best practices
Feature Flags: Controlling feature rollout
Code Reviews: Ensuring code quality before merging

Introduction​

Understanding CI/CD​

Continuous Integration (CI)​

Continuous Delivery (CD)​

Continuous Deployment​

CI/CD Pipeline Architecture​

Pipeline Stages Explained​

1. Source Stage​

2. Build Stage​

3. Test Stage​

4. Quality Gates​

5. Deployment Stages​

CI/CD Best Practices​

1. Keep Builds Fast​

2. Fail Fast​

3. Make Pipelines Reproducible​

4. Implement Proper Secret Management​

5. Monitor and Alert​

6. Implement Progressive Deployment Strategies​

Blue-Green Deployment​

Canary Deployment​

Feature Flags​

7. Automate Database Migrations​

8. Implement Automated Rollbacks​

CI/CD Tools and Platforms​

Popular CI/CD Platforms​

Choosing the Right Platform​

Common CI/CD Pitfalls​

1. Testing in Production​

2. Flaky Tests​

3. Long-Running Builds​

4. Inadequate Rollback Procedures​

5. Ignoring Security Scanning​

6. Not Version Controlling Pipeline Configuration​

CI/CD Metrics and KPIs​

Deployment Frequency​

Lead Time for Changes​

Mean Time to Recovery (MTTR)​

Change Failure Rate​

Real-World CI/CD Pipeline Example​

Getting Started with CI/CD​

Phase 1: Basic CI (Week 1-2)​

Phase 2: Enhanced Testing (Week 3-4)​

Phase 3: Continuous Delivery (Week 5-8)​

Phase 4: Continuous Deployment (Week 9-12)​

Conclusion​

Key Takeaways​

Related Topics​

Further Reading​

Introduction

Understanding CI/CD

Continuous Integration (CI)

Continuous Delivery (CD)

Continuous Deployment

CI/CD Pipeline Architecture

Pipeline Stages Explained

1. Source Stage

2. Build Stage

3. Test Stage

4. Quality Gates

5. Deployment Stages

CI/CD Best Practices

1. Keep Builds Fast

2. Fail Fast

3. Make Pipelines Reproducible

4. Implement Proper Secret Management

5. Monitor and Alert

6. Implement Progressive Deployment Strategies

Blue-Green Deployment

Canary Deployment

Feature Flags

7. Automate Database Migrations

8. Implement Automated Rollbacks

CI/CD Tools and Platforms

Popular CI/CD Platforms

Choosing the Right Platform

Common CI/CD Pitfalls

1. Testing in Production

2. Flaky Tests

3. Long-Running Builds

4. Inadequate Rollback Procedures

5. Ignoring Security Scanning

6. Not Version Controlling Pipeline Configuration

CI/CD Metrics and KPIs

Deployment Frequency

Lead Time for Changes

Mean Time to Recovery (MTTR)

Change Failure Rate

Real-World CI/CD Pipeline Example

Getting Started with CI/CD

Phase 1: Basic CI (Week 1-2)

Phase 2: Enhanced Testing (Week 3-4)

Phase 3: Continuous Delivery (Week 5-8)

Phase 4: Continuous Deployment (Week 9-12)

Conclusion

Key Takeaways

Related Topics

Further Reading