Code Coverage Metrics: Why 100% Coverage Can Still Give You False Confidence

"We have 90% code coverage!" is often declared with pride, but what does it actually mean? High coverage doesn't guarantee quality tests, and chasing coverage percentages can lead to poor testing practices. This comprehensive guide helps you understand code coverage strategically and use it to improve your testing effectiveness.

Understanding Code Coverage Types

Code coverage measures which parts of your codebase are executed during testing. However, there are multiple ways to measure this.

The Four Dimensions of Coverage

graph TD
    A[Code Coverage] --> B[Statement Coverage]
    A --> C[Branch Coverage]
    A --> D[Function Coverage]
    A --> E[Line Coverage]

    B --> F[Which statements executed?]
    C --> G[Which decision paths taken?]
    D --> H[Which functions called?]
    E --> I[Which lines touched?]

    style A fill:#a78bfa
    style B fill:#6bcf7f
    style C fill:#ffd93d
    style D fill:#ff9a76
    style E fill:#4d96ff

Coverage Type	What It Measures	Example	Limitation
Line Coverage	% of lines executed	85/100 lines = 85%	Doesn't verify behavior
Statement Coverage	% of statements executed	All assignments run	Multiple statements per line
Branch Coverage	% of decision paths taken	if/else both tested	Doesn't test combinations
Function Coverage	% of functions called	45/50 functions = 90%	Function called ≠ properly tested

Why Line Coverage Alone Is Misleading

// Example: 100% line coverage but poor testing

function processPayment(amount: number, user: User): PaymentResult {
  if (amount <= 0) throw new Error('Invalid amount');
  if (!user.verified) throw new Error('User not verified');

  const fee = amount * 0.03;
  const total = amount + fee;

  return {
    success: true,
    amount: total,
    transactionId: generateId(),
  };
}

// ❌ BAD TEST: 100% line coverage but incomplete
test('processes payment', () => {
  const result = processPayment(100, { verified: true });
  expect(result.success).toBe(true);
});
// Coverage: 100% lines, but:
// - Doesn't test error conditions
// - Doesn't verify calculation accuracy
// - Doesn't test edge cases

// ✅ GOOD TESTS: Meaningful coverage
describe('processPayment', () => {
  test('calculates correct total with fee', () => {
    const result = processPayment(100, { verified: true });

    expect(result.success).toBe(true);
    expect(result.amount).toBe(103); // 100 + 3% fee
    expect(result.transactionId).toBeDefined();
  });

  test('rejects negative amounts', () => {
    expect(() => processPayment(-50, { verified: true })).toThrow('Invalid amount');
  });

  test('rejects zero amount', () => {
    expect(() => processPayment(0, { verified: true })).toThrow('Invalid amount');
  });

  test('rejects unverified users', () => {
    expect(() => processPayment(100, { verified: false })).toThrow('User not verified');
  });

  test('handles large amounts correctly', () => {
    const result = processPayment(10000, { verified: true });
    expect(result.amount).toBe(10300);
  });
});

Setting Up Comprehensive Coverage Reporting

Configuration for JavaScript/TypeScript Projects

// vitest.config.ts
import { defineConfig } from 'vitest/config';

export default defineConfig({
  test: {
    coverage: {
      provider: 'v8', // or 'istanbul'

      // Report types
      reporter: ['text', 'json', 'html', 'lcov'],

      // Coverage thresholds
      lines: 80,
      branches: 75,
      functions: 80,
      statements: 80,

      // Include/exclude patterns
      include: ['src/**/*.{js,ts}'],
      exclude: [
        'node_modules/',
        'dist/',
        '**/*.test.ts',
        '**/*.spec.ts',
        '**/types.ts',
        '**/index.ts', // Re-exports only
      ],

      // Per-file thresholds (stricter for critical code)
      perFile: true,

      // Fail if any threshold not met
      thresholdAutoUpdate: false,
    },
  },
});

Playwright Test Coverage

// playwright.config.ts
import { defineConfig } from '@playwright/test';

export default defineConfig({
  use: {
    // Collect coverage during tests
    collectCoverage: true,
  },

  webServer: {
    command: 'npm run dev',
    port: 3000,
    env: {
      // Enable coverage in dev server
      NODE_V8_COVERAGE: 'coverage/tmp',
    },
  },
});

// tests/example.spec.ts
import { test, expect } from '@playwright/test';

test.describe('Homepage', () => {
  test('loads and displays content', async ({ page }) => {
    await page.goto('/');

    // Collect coverage
    await page.coverage.startJSCoverage();

    // Interact with page
    await page.click('[data-testid="hero-cta"]');
    await expect(page).toHaveURL(/.*signup/);

    // Stop and get coverage
    const jsCoverage = await page.coverage.stopJSCoverage();

    // Log covered functions
    for (const entry of jsCoverage) {
      console.log(`${entry.url}: ${entry.text.length} bytes`);
    }
  });
});

Visualizing Coverage Reports

# Generate HTML report
npm run test:coverage

# Open in browser
open coverage/index.html

# CI: Upload to Codecov
bash <(curl -s https://codecov.io/bash)

# .github/workflows/coverage.yml
name: Code Coverage

on:
  push:
    branches: [main]
  pull_request:

jobs:
  coverage:
    runs-on: ubuntu-latest

    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: '22'

      - run: npm ci
      - run: npm run test:coverage

      # Upload to Codecov
      - name: Upload coverage to Codecov
        uses: codecov/codecov-action@v4
        with:
          files: ./coverage/lcov.info
          flags: unittests
          name: codecov-umbrella

      # Comment on PR with coverage
      - name: Coverage Comment
        uses: romeovs/lcov-reporter-action@v0.3.1
        with:
          lcov-file: ./coverage/lcov.info
          github-token: ${{ secrets.GITHUB_TOKEN }}

      # Enforce thresholds
      - name: Check coverage thresholds
        run: |
          node scripts/check-coverage-thresholds.js

Strategic Coverage Goals by Code Type

Different parts of your codebase require different coverage levels.

Coverage Target Matrix

Code Type	Target Coverage	Priority	Rationale
Business Logic	90-100%	Critical	Core functionality, high bug impact
API Routes	85-95%	High	User-facing, integration points
Utils/Helpers	80-90%	High	Reused across application
UI Components	70-80%	Medium	Visual testing more important
Config/Setup	40-60%	Low	Mostly declarative
Type Definitions	0%	N/A	No runtime code

Implementing Differential Coverage

// scripts/check-coverage-thresholds.js
const fs = require('fs');
const path = require('path');

// Read coverage report
const coverage = JSON.parse(fs.readFileSync('./coverage/coverage-final.json', 'utf8'));

// Define thresholds by path pattern
const thresholds = [
  { pattern: /src\/lib\/business-logic/, lines: 90, branches: 85 },
  { pattern: /src\/api/, lines: 85, branches: 80 },
  { pattern: /src\/utils/, lines: 80, branches: 75 },
  { pattern: /src\/components/, lines: 70, branches: 65 },
  { pattern: /src\/config/, lines: 50, branches: 40 },
];

let failures = [];

for (const [filePath, fileData] of Object.entries(coverage)) {
  // Find matching threshold
  const threshold = thresholds.find((t) => t.pattern.test(filePath));
  if (!threshold) continue;

  const lineCoverage = (fileData.lines.hit / fileData.lines.found) * 100;
  const branchCoverage = (fileData.branches.hit / fileData.branches.found) * 100;

  if (lineCoverage < threshold.lines) {
    failures.push({
      file: filePath,
      metric: 'lines',
      actual: lineCoverage.toFixed(1),
      required: threshold.lines,
    });
  }

  if (branchCoverage < threshold.branches) {
    failures.push({
      file: filePath,
      metric: 'branches',
      actual: branchCoverage.toFixed(1),
      required: threshold.branches,
    });
  }
}

if (failures.length > 0) {
  console.error('\n❌ Coverage thresholds not met:\n');

  for (const failure of failures) {
    console.error(`${failure.file}`);
    console.error(`  ${failure.metric}: ${failure.actual}% (required: ${failure.required}%)\n`);
  }

  process.exit(1);
}

console.log('✅ All coverage thresholds met!');

Understanding Branch Coverage Deeply

Branch coverage is often more valuable than line coverage because it ensures you test different execution paths.

Example: Why Branch Coverage Matters

// Function with multiple branches
function calculateDiscount(price: number, userType: 'new' | 'regular' | 'premium', itemCount: number): number {
  let discount = 0;

  // User type discounts
  if (userType === 'new') {
    discount = 0.1; // 10% for new users
  } else if (userType === 'premium') {
    discount = 0.2; // 20% for premium
  }

  // Bulk discounts
  if (itemCount >= 10) {
    discount += 0.05; // Additional 5% for bulk
  }

  // Cap discount at 30%
  if (discount > 0.3) {
    discount = 0.3;
  }

  return price * (1 - discount);
}

// ❌ BAD: 100% line coverage, poor branch coverage
test('calculates discount', () => {
  const result = calculateDiscount(100, 'new', 1);
  expect(result).toBe(90);
});
// Covers: new user, < 10 items, discount < 30%
// Missing: regular, premium, bulk, capped discount

// ✅ GOOD: Full branch coverage
describe('calculateDiscount', () => {
  describe('user type discounts', () => {
    test('applies 10% for new users', () => {
      expect(calculateDiscount(100, 'new', 1)).toBe(90);
    });

    test('applies 0% for regular users', () => {
      expect(calculateDiscount(100, 'regular', 1)).toBe(100);
    });

    test('applies 20% for premium users', () => {
      expect(calculateDiscount(100, 'premium', 1)).toBe(80);
    });
  });

  describe('bulk discounts', () => {
    test('adds 5% for 10+ items', () => {
      expect(calculateDiscount(100, 'regular', 10)).toBe(95);
    });

    test('no bulk discount for < 10 items', () => {
      expect(calculateDiscount(100, 'regular', 9)).toBe(100);
    });
  });

  describe('discount capping', () => {
    test('caps combined discounts at 30%', () => {
      // Premium (20%) + Bulk (5%) + more would exceed 30%
      expect(calculateDiscount(100, 'premium', 10)).toBe(70);
    });
  });

  describe('combined scenarios', () => {
    test('new user with bulk order', () => {
      // 10% new + 5% bulk = 15%
      expect(calculateDiscount(100, 'new', 10)).toBe(85);
    });

    test('premium user with bulk order', () => {
      // 20% premium + 5% bulk = 25%
      expect(calculateDiscount(100, 'premium', 10)).toBe(75);
    });
  });
});

Identifying Meaningful Coverage Gaps

Not all uncovered code is equally important. Focus on high-risk, uncovered paths.

Coverage Gap Analysis Tool

// scripts/analyze-coverage-gaps.ts
import fs from 'fs';
import { parse } from '@typescript-eslint/parser';

interface CoverageGap {
  file: string;
  line: number;
  type: 'error-handling' | 'edge-case' | 'unhappy-path' | 'normal';
  severity: 'critical' | 'high' | 'medium' | 'low';
  code: string;
}

function analyzeCoverageGaps(coverageData: any): CoverageGap[] {
  const gaps: CoverageGap[] = [];

  for (const [filePath, fileData] of Object.entries(coverageData)) {
    const sourceCode = fs.readFileSync(filePath, 'utf8');
    const lines = sourceCode.split('\n');

    // Find uncovered lines
    const uncoveredLines = fileData.s // statement coverage
      .map((count: number, index: number) => ({ count, index }))
      .filter(({ count }: any) => count === 0)
      .map(({ index }: any) => fileData.statementMap[index].start.line);

    for (const lineNumber of uncoveredLines) {
      const code = lines[lineNumber - 1].trim();

      // Classify the gap
      let type: CoverageGap['type'] = 'normal';
      let severity: CoverageGap['severity'] = 'medium';

      // Error handling
      if (code.includes('throw') || code.includes('catch') || code.includes('error')) {
        type = 'error-handling';
        severity = 'high';
      }

      // Edge cases
      if (
        code.includes('=== 0') ||
        code.includes('=== null') ||
        code.includes('=== undefined') ||
        code.includes('isEmpty')
      ) {
        type = 'edge-case';
        severity = 'high';
      }

      // Unhappy paths
      if (code.includes('if (!') || code.includes('if (!') || (code.includes('!==') && !code.includes('undefined'))) {
        type = 'unhappy-path';
        severity = 'medium';
      }

      // Critical paths (in business logic)
      if (filePath.includes('business-logic') || filePath.includes('payment') || filePath.includes('auth')) {
        severity = 'critical';
      }

      gaps.push({
        file: filePath,
        line: lineNumber,
        type,
        severity,
        code,
      });
    }
  }

  // Sort by severity
  const severityOrder = { critical: 0, high: 1, medium: 2, low: 3 };
  return gaps.sort((a, b) => severityOrder[a.severity] - severityOrder[b.severity]);
}

// Usage
const coverage = JSON.parse(fs.readFileSync('./coverage/coverage-final.json', 'utf8'));
const gaps = analyzeCoverageGaps(coverage);

console.log('📊 Coverage Gap Analysis\n');

const criticalGaps = gaps.filter((g) => g.severity === 'critical');
console.log(`🔴 Critical: ${criticalGaps.length}`);

for (const gap of criticalGaps) {
  console.log(`\n${gap.file}:${gap.line}`);
  console.log(`  Type: ${gap.type}`);
  console.log(`  Code: ${gap.code}`);
}

Coverage Trends Dashboard

// scripts/track-coverage-trends.ts
interface CoverageTrend {
  date: string;
  commit: string;
  overall: number;
  branches: number;
  statements: number;
  functions: number;
}

function trackCoverage(newData: CoverageTrend): void {
  const historyFile = './coverage-history.json';

  let history: CoverageTrend[] = [];
  if (fs.existsSync(historyFile)) {
    history = JSON.parse(fs.readFileSync(historyFile, 'utf8'));
  }

  history.push(newData);

  // Keep last 100 entries
  if (history.length > 100) {
    history = history.slice(-100);
  }

  fs.writeFileSync(historyFile, JSON.stringify(history, null, 2));

  // Analyze trends
  if (history.length >= 5) {
    const recent = history.slice(-5);
    const avgRecent = recent.reduce((sum, d) => sum + d.overall, 0) / recent.length;

    const older = history.slice(-10, -5);
    const avgOlder = older.reduce((sum, d) => sum + d.overall, 0) / older.length;

    const trend = avgRecent - avgOlder;

    if (trend < -2) {
      console.warn('⚠️  Coverage is declining!');
      console.warn(`  5-commit average: ${avgRecent.toFixed(1)}%`);
      console.warn(`  Previous average: ${avgOlder.toFixed(1)}%`);
    } else if (trend > 2) {
      console.log('✅ Coverage is improving!');
      console.log(`  5-commit average: ${avgRecent.toFixed(1)}%`);
      console.log(`  Previous average: ${avgOlder.toFixed(1)}%`);
    }
  }
}

Coverage Anti-Patterns to Avoid

Anti-Pattern 1: Testing for Coverage's Sake

// ❌ BAD: Tests that boost coverage but add no value
test('getters return values', () => {
  const user = new User('test@example.com', 'John');

  expect(user.getEmail()).toBe('test@example.com');
  expect(user.getName()).toBe('John');
});

// Simple getters don't need tests - they add no business logic

// ✅ GOOD: Test actual behavior
test('user validation rejects invalid emails', () => {
  expect(() => new User('invalid-email', 'John')).toThrow('Invalid email format');
});

Anti-Pattern 2: Ignoring Important Gaps for Easy Coverage

// ❌ BAD: Test easy paths, ignore error cases
describe('UserService', () => {
  test('creates user successfully', () => {
    const user = userService.create({ email: 'test@example.com' });
    expect(user.id).toBeDefined();
  });

  // Coverage: 60% - but missing all error handling!
});

// ✅ GOOD: Test error paths too
describe('UserService', () => {
  test('creates user successfully', () => {
    const user = userService.create({ email: 'test@example.com' });
    expect(user.id).toBeDefined();
  });

  test('rejects invalid email', () => {
    expect(() => userService.create({ email: 'invalid' })).toThrow('Invalid email');
  });

  test('rejects duplicate email', async () => {
    await userService.create({ email: 'test@example.com' });

    await expect(userService.create({ email: 'test@example.com' })).rejects.toThrow('Email already exists');
  });

  test('handles database errors gracefully', async () => {
    // Mock database failure
    jest.spyOn(database, 'insert').mockRejectedValue(new Error('DB Error'));

    await expect(userService.create({ email: 'test@example.com' })).rejects.toThrow('Failed to create user');
  });
});

Anti-Pattern 3: 100% Coverage as a Mandate

// Not all code needs 100% coverage

// Example: Logging code
function logUserAction(action: string, userId: string): void {
  if (process.env.NODE_ENV === 'development') {
    console.log(`[DEV] User ${userId}: ${action}`);
  }

  logger.info('user_action', { action, userId });
}

// Testing the console.log provides little value
// Testing logger.info is sufficient

// ✅ Reasonable test
test('logs user action', () => {
  const mockLogger = jest.spyOn(logger, 'info');

  logUserAction('login', 'user-123');

  expect(mockLogger).toHaveBeenCalledWith('user_action', {
    action: 'login',
    userId: 'user-123',
  });
});

// Don't waste time getting console.log to 100%

Mutation Testing: Coverage Quality Check

Coverage tells you what code was executed, but mutation testing tells you if your tests actually catch bugs.

Introduction to Mutation Testing

# Install Stryker Mutator
npm install --save-dev @stryker-mutator/core @stryker-mutator/typescript-checker

# Configure
npx stryker init

// stryker.config.json
{
  "mutator": "typescript",
  "packageManager": "npm",
  "reporters": ["html", "clear-text", "progress"],
  "testRunner": "vitest",
  "coverageAnalysis": "perTest",
  "mutate": [
    "src/**/*.ts",
    "!src/**/*.test.ts",
    "!src/**/*.spec.ts"
  ],
  "thresholds": {
    "high": 80,
    "low": 60,
    "break": 50
  }
}

Understanding Mutation Scores

// Original code
function isAdult(age: number): boolean {
  return age >= 18;
}

// Test with 100% coverage
test('isAdult checks age', () => {
  expect(isAdult(18)).toBe(true);
  expect(isAdult(17)).toBe(false);
});

// Mutation testing creates variants:
// Mutant 1: age > 18 (boundary changed)
// Mutant 2: age <= 18 (operator flipped)
// Mutant 3: age >= 19 (constant changed)

// If tests don't catch these mutants, coverage is misleading!

// ✅ Better test that kills mutants
test('isAdult age boundary', () => {
  expect(isAdult(17)).toBe(false);
  expect(isAdult(18)).toBe(true); // Catches >= vs >
  expect(isAdult(19)).toBe(true);
});

Coverage in Code Review

// .github/workflows/coverage-comment.yml
name: Coverage Comment

on:
  pull_request:

jobs:
  coverage:
    runs-on: ubuntu-latest

    steps:
      - uses: actions/checkout@v4
        with:
          fetch-depth: 0  # Need full history

      # Get base branch coverage
      - name: Get base coverage
        run: |
          git checkout ${{ github.base_ref }}
          npm ci
          npm run test:coverage
          mv coverage/coverage-summary.json base-coverage.json

      # Get PR coverage
      - name: Get PR coverage
        run: |
          git checkout ${{ github.head_ref }}
          npm ci
          npm run test:coverage
          mv coverage/coverage-summary.json pr-coverage.json

      # Compare and comment
      - name: Compare coverage
        uses: actions/github-script@v7
        with:
          script: |
            const fs = require('fs');
            const base = JSON.parse(fs.readFileSync('base-coverage.json'));
            const pr = JSON.parse(fs.readFileSync('pr-coverage.json'));

            const basePct = base.total.lines.pct;
            const prPct = pr.total.lines.pct;
            const diff = prPct - basePct;

            const emoji = diff >= 0 ? '✅' : '⚠️';
            const sign = diff >= 0 ? '+' : '';

            const comment = `
            ## ${emoji} Coverage Report

            **Overall Coverage:** ${prPct.toFixed(2)}% (${sign}${diff.toFixed(2)}%)

            | Metric | Base | PR | Change |
            |--------|------|----|----- --|
            | Lines | ${basePct}% | ${prPct}% | ${sign}${diff.toFixed(2)}% |
            | Branches | ${base.total.branches.pct}% | ${pr.total.branches.pct}% | ${sign}${(pr.total.branches.pct - base.total.branches.pct).toFixed(2)}% |
            | Functions | ${base.total.functions.pct}% | ${pr.total.functions.pct}% | ${sign}${(pr.total.functions.pct - base.total.functions.pct).toFixed(2)}% |
            `;

            github.rest.issues.createComment({
              issue_number: context.issue.number,
              owner: context.repo.owner,
              repo: context.repo.repo,
              body: comment
            });

Conclusion: Coverage as a Tool, Not a Goal

Code coverage is a valuable metric when used strategically:

1. Use Multiple Metrics: Line, branch, function coverage together
2. Set Contextual Targets: Different thresholds for different code types
3. Focus on Gaps: Identify and prioritize meaningful uncovered paths
4. Avoid Vanity Metrics: 100% coverage doesn't guarantee quality
5. Combine with Mutation Testing: Verify tests actually catch bugs
6. Track Trends: Monitor coverage over time, not just point-in-time percentages

Remember: The goal isn't 100% coverage—it's confidence that your code works correctly.

Improve Your Testing with ScanlyApp

ScanlyApp provides advanced coverage reporting and analysis tools, helping you identify meaningful gaps and improve test quality beyond simple percentages.

Start Your Free Trial and level up your testing strategy today.

Related articles: Also see mutation testing to validate the quality of the coverage you have, design patterns that improve coverage without test redundancy, and how coverage metrics feed into the QA velocity picture.