DOMAIN:TESTING — MUTATION_TESTING

OWNER: koen
ALSO_USED_BY: marije, judith (score review), jasper (gap analysis input)
UPDATED: 2026-03-24
SCOPE: mutation testing for all GE client projects using Stryker

---

MUTATION_TESTING_OVERVIEW

Mutation testing answers the question: "Would my tests catch a bug?"
It introduces small changes (MUTANTS) into the source code and checks if tests DETECT them.
If a mutant survives (tests still pass), your tests have a GAP.

WHY_MUTATION_TESTING:
- Coverage tells you what CODE runs. Mutation testing tells you what code is TESTED.
- 100% line coverage with 40% mutation score = test theater
- Mutation score is the most honest measure of test quality
- It finds tests that always pass regardless of code correctness

HOW_IT_WORKS:
1. Stryker parses source code
2. Creates MUTANTS by applying mutation operators (change > to >=, remove a line, etc.)
3. Runs test suite against EACH mutant
4. IF tests fail → mutant is KILLED (good — your tests caught the bug)
5. IF tests pass → mutant SURVIVED (bad — your tests missed the bug)
6. Mutation score = killed / (killed + survived) * 100%

---

STRYKER_CONFIGURATION

BASIC_CONFIG

// stryker.config.mjs
/** @type {import('@stryker-mutator/api/core').PartialStrykerOptions} */
const config = {
  mutate: [
    'src/**/*.ts',
    '!src/**/*.test.ts',
    '!src/**/*.spec.ts',
    '!src/**/*.d.ts',
    '!src/**/types.ts',
    '!src/**/index.ts',
  ],
  testRunner: 'vitest',
  reporters: ['html', 'clear-text', 'progress', 'json'],
  htmlReporter: {
    fileName: 'reports/mutation/index.html',
  },
  jsonReporter: {
    fileName: 'reports/mutation/report.json',
  },
  thresholds: {
    high: 80,
    low: 60,
    break: 60,  // CI fails below this
  },
  concurrency: 4,
  timeoutMS: 30000,
  tempDirName: '.stryker-tmp',
  cleanTempDir: 'always',
};

export default config;

CONFIG_FIELDS_EXPLAINED

mutate: glob patterns for files TO mutate — exclude tests, types, barrel files
testRunner: 'vitest' for GE projects (requires @stryker-mutator/vitest-runner)
reporters: 'html' for local review, 'json' for CI integration, 'clear-text' for terminal
thresholds.high: mutation score above this shows as GREEN (target: 80%)
thresholds.low: score below this shows as RED (warning: 60%)
thresholds.break: CI FAILS if score is below this (enforced: 60%)
concurrency: parallel test processes — set to CPU core count
timeoutMS: kill mutant test run if slower than this — prevents infinite loops

REQUIRED_PACKAGES

{
  "devDependencies": {
    "@stryker-mutator/core": "^8.x",
    "@stryker-mutator/vitest-runner": "^8.x",
    "@stryker-mutator/typescript-checker": "^8.x"
  }
}

RULE: always install @stryker-mutator/typescript-checker — it eliminates type-invalid mutants that waste time
RULE: pin to same major version across all Stryker packages

---

MUTANT_TYPES

ARITHMETIC_MUTATIONS

Original: a + b → Mutant: a - b, a * b, a / b, a % b
CATCHES: tests that don't verify mathematical correctness

CONDITIONAL_MUTATIONS

Original: a > b → Mutant: a >= b, a < b, a <= b, a === b
CATCHES: off-by-one errors, boundary conditions

LOGICAL_MUTATIONS

Original: a && b → Mutant: a || b
Original: !a → Mutant: a
CATCHES: tests that don't verify logical combinations

STRING_MUTATIONS

Original: "hello" → Mutant: ""
CATCHES: tests that don't validate string content

ARRAY_MUTATIONS

Original: array.push(item) → Mutant: removed
Original: array.filter(fn) → Mutant: array
CATCHES: tests that don't verify collection modifications

BLOCK_STATEMENT_MUTATIONS

Original: entire block { ... } → Mutant: {}
CATCHES: tests that don't verify side effects happen

EQUALITY_MUTATIONS

Original: === → Mutant: !==
Original: !== → Mutant: ===
CATCHES: tests that don't verify equality checks

UNARY_MUTATIONS

Original: ++x → Mutant: --x
Original: -x → Mutant: x
CATCHES: tests that don't verify increment/decrement behavior

OPTIONAL_CHAINING_MUTATIONS

Original: obj?.prop → Mutant: obj.prop
CATCHES: tests that don't verify null-safety

BOOLEAN_LITERAL_MUTATIONS

Original: true → Mutant: false
CATCHES: tests that don't verify boolean logic

---

INTERPRETING_RESULTS

MUTATION_SCORE

SCORE = killed / (killed + survived) * 100%

INTERPRETATION:
- 90-100%: Excellent — tests catch almost all possible bugs
- 80-89%: Good — standard target for new code
- 60-79%: Acceptable — target for existing code, improve over time
- 40-59%: Poor — significant test gaps, prioritize fixing
- Below 40%: Critical — tests are providing false confidence

SURVIVING_MUTANTS

When a mutant survives, it means: "This bug could exist and your tests wouldn't catch it."

RESPONSE_TO_SURVIVING_MUTANTS:
1. REVIEW the mutant — is it a real possible bug?
2. IF YES: write a test that kills it
3. IF NO: the mutant is an EQUIVALENT mutant (semantically identical to original)
4. Document equivalent mutants so they don't waste time in future runs

EQUIVALENT_MUTANTS

Some mutants produce code that behaves IDENTICALLY to the original.
These are FALSE NEGATIVES — they inflate the surviving count.

EXAMPLE:

// Original
function isPositive(n: number): boolean {
  return n > 0;
}

// Mutant: n >= 0 — this is NOT equivalent (0 is treated differently)
// Mutant: n > -1 — this IS equivalent for integers (but not floats!)

RULE: before writing a new test to kill a mutant, verify it's not equivalent
RULE: if you find many equivalent mutants in one file, consider adding a Stryker comment to ignore them:

// Stryker disable next-line all: equivalent mutant — logging order doesn't affect behavior
logger.info(`Processing ${item.id}`);

---

BASELINE_THRESHOLDS

NEW_CODE (mandatory)

MINIMUM_MUTATION_SCORE: 80%
CRITICAL_PATHS (auth, payment, data validation): 90%
RULE: new code that doesn't meet threshold = PR blocked

EXISTING_CODE (improve over time)

MINIMUM_MUTATION_SCORE: 60%
RULE: existing code below 60% = warning, not blocking
RULE: every sprint should improve mutation score by at least 2% on focused areas
RULE: never LOWER mutation score of existing code

PER_FILE_THRESHOLDS

// stryker.config.mjs — per-directory overrides
const config = {
  // ... base config ...
  mutate: ['src/**/*.ts', '!src/**/*.test.ts'],
  thresholds: { high: 80, low: 60, break: 60 },
};

CRITICAL_DIRECTORIES (apply 90% threshold manually in review):
- src/lib/auth/ — authentication and authorization
- src/lib/payment/ — payment processing
- src/lib/validation/ — input validation
- src/middleware/ — security middleware

---

INCREMENTAL_MUTATION_TESTING

Full mutation testing is SLOW. Incremental mode tests only CHANGED code.

CONFIGURATION

// stryker.config.mjs
const config = {
  // ... other config ...
  incremental: true,
  incrementalFile: '.stryker-incremental.json',
};

HOW_INCREMENTAL_WORKS

1. First run: full mutation testing, stores results in .stryker-incremental.json
2. Subsequent runs: only mutates files that CHANGED since last run
3. Reuses cached results for unchanged files
4. Dramatic speed improvement — 10x faster in typical PR

CI_STRATEGY

# CI pipeline
mutation-test:
  script:
    - npx stryker run --incremental
  artifacts:
    paths:
      - .stryker-incremental.json
      - reports/mutation/
  cache:
    key: stryker-cache
    paths:
      - .stryker-incremental.json

RULE: commit .stryker-incremental.json for accurate incremental runs
RULE: full mutation run (non-incremental) weekly on main branch
RULE: PR mutation testing is ALWAYS incremental — speed matters for developer flow

---

PERFORMANCE_OPTIMIZATION

PROBLEM

Mutation testing generates HUNDREDS of mutants. Each runs the full test suite.
Without optimization, a project with 1000 mutants and 5-second test suite = 83 minutes.

SOLUTIONS

1. INCREMENTAL MODE: test only changed files (10x faster)
2. CONCURRENCY: concurrency: 4 runs mutants in parallel
3. TYPESCRIPT_CHECKER: eliminates type-invalid mutants before running tests
4. TIMEOUT: timeoutMS: 30000 kills slow mutants (infinite loops)
5. TARGETED MUTATE: only mutate the files you care about

// For a specific PR — mutate only changed files
const config = {
  mutate: [
    'src/lib/user-service.ts',
    'src/lib/auth.ts',
  ],
};

1. IGNORE_MUTATIONS: skip mutations that don't provide value

const config = {
  ignoreMutations: [
    'StringLiteral',  // Skip string mutations (often log messages)
  ],
};

RULE: never skip ConditionalExpression or EqualityOperator mutations — they find real bugs
RULE: StringLiteral and ObjectLiteral can be skipped for non-critical code
RULE: never set timeoutMS below 10000 — some tests legitimately take time

---

KOEN_WORKFLOW

Koen runs mutation testing AFTER Marije/Judith write post-impl tests.

STEP 1: Run incremental mutation testing on changed files

npx stryker run --incremental

STEP 2: Review surviving mutants
- Open HTML report at reports/mutation/index.html
- Focus on files with score below threshold

STEP 3: Classify surviving mutants
- REAL_GAP: test is missing — report to Marije/Judith
- EQUIVALENT: mutant behavior is identical — document and skip
- LOW_VALUE: mutation in logging/comments — consider ignoring

STEP 4: Report findings
- List of files below threshold
- Specific surviving mutants that represent real gaps
- Recommended tests to write (for Marije/Judith to implement)

STEP 5: Feed results to Jasper
- Jasper uses mutation data in reconciliation analysis
- Surviving mutants in TDD tests indicate spec gaps
- Surviving mutants in post-impl tests indicate coverage gaps

RULE: Koen does NOT write tests — Koen identifies gaps, Marije/Judith write the tests
RULE: Koen does NOT modify source code — mutation testing is read-only analysis
RULE: Koen runs mutation testing on BOTH TDD and post-impl test suites separately

---

STRYKER_CLI_COMMANDS

# Full mutation testing
npx stryker run

# Incremental mode
npx stryker run --incremental

# Specific files only
npx stryker run --mutate "src/lib/auth.ts"

# With specific concurrency
npx stryker run --concurrency 8

# Clear incremental cache (force full run)
rm .stryker-incremental.json && npx stryker run --incremental

# Dry run (show what would be mutated, don't run)
npx stryker run --dryRunOnly

---

CROSS_REFERENCES

VITEST: domains/testing/vitest-patterns.md — the test runner Stryker executes
TDD: domains/testing/tdd-methodology.md — pre-impl tests that Stryker validates
RECONCILIATION: domains/testing/test-reconciliation.md — Jasper uses mutation data
PITFALLS: domains/testing/pitfalls.md — mutation testing anti-patterns