Testing and software quality

Status: Complete. Last reviewed 2026-08-28.

Testing is evidence that selected risks are controlled under stated conditions. A large suite can still miss the production database, concurrency, serialization, deployment, or dependency behaviors that matter. Choose test boundaries from failure cost and uncertainty, then keep feedback fast enough that engineers use it.

Test observable contracts at the cheapest credible boundary

A unit test exercises a small policy or transformation with controlled collaborators. An integration test crosses a real boundary such as a database, filesystem, queue serializer, or HTTP adapter. A feature/service test drives an application boundary with much of the stack. An end-to-end test crosses deployed components and a real client/browser. Labels vary; state what is real.

Prefer the lowest boundary that can fail for the risk. Pure pricing rules benefit from table/property-style unit tests. SQL constraints, isolation, query mappings, and migrations need the production database engine. Authentication, middleware, validation, and serialization need the framework boundary. A small set of critical journeys may justify browser or environment-level tests.

The test pyramid is a feedback heuristic, not a quota. A service whose risk lies in database transactions may reasonably have many integration tests. Avoid reproducing framework implementation in mocks just to create more “unit” tests. Conversely, do not send every edge case through a slow end-to-end stack when a focused policy test proves it more clearly.

Test outcomes and invariants rather than private call sequences. Good assertions distinguish the failure: persisted state, emitted contract, authorization denial, ledger balance, retry scheduling, or absence of an unintended effect. Snapshotting a huge response can hide meaningful changes in noise; assert the guaranteed shape and use schema/contract checks where breadth matters.

Doubles have different jobs

A stub returns controlled input to the subject.
A fake is a working simplified implementation, such as an in-memory adapter.
A spy records interactions for later assertions.
A mock has preprogrammed interaction expectations.

Use doubles at volatile boundaries you own. Wrap a payment SDK in an application interface, unit-test policy against a controlled fake/stub, and separately integration/contract-test the adapter. Mocking vendor internals couples tests to an implementation you do not control and can prove a fictional protocol.

Interaction assertions are appropriate when the interaction is the outcome—publishing exactly one command with a stable idempotency key, for example. They are brittle when they encode incidental call order between internal methods. A fake database rarely reproduces transactions, constraints, query planning, locking, null/type semantics, or generated IDs; it cannot validate those risks.

Framework fakes deliberately replace drivers. A queue fake can prove dispatch intent but not payload serialization, worker boot, visibility timeout, acknowledgement, or actual transport configuration. Pair it with focused real-driver or worker tests where those failures are costly.

Design deterministic data and time

Factories should name meaningful states—expired reservation, suspended tenant, partially refunded payment—rather than merely fill required columns. Builders/default factories reduce noise, but hidden global defaults can make tests pass for the wrong reason. Keep identity, time, and authorization relationships visible when they drive behavior.

Control time through an injected clock or framework facility. Assert instants/ranges instead of sleeping. Random/property tests should report and persist a seed and shrink a failure to a minimal example. Generated IDs can be supplied through a boundary when exact correlation is part of the outcome.

Database cleanup must match the test. Transaction rollback is fast but can hide after-commit hooks and code running on other connections/processes. Truncation/rebuild is slower but crosses commits. Parallel tests require isolated databases/schemas/tenants and collision-free external resources; a shared static path or queue creates suite-only failures.

Flakiness is a defect in the test or system assumptions, not background weather. Quarantine can keep the main signal usable only with an owner and deadline. Repeated retries conceal failure probability and stretch CI. Diagnose ordering, leaked globals, time, async completion, ports/files, non-isolated data, and dependency instability.

Test failure timelines and concurrency

Happy-path mocks do not establish retry safety. For a job crossing a database and external provider, inject failure before/after local commit, before/after external success, before acknowledgement, and on redelivery. Assert one logical effect, durable unknown states, and reconciliation—not merely that an exception was thrown.

Concurrency tests need controlled interleavings. Use barriers/latches or two connections to ensure both transactions read before either writes, then assert the database invariant. Running a race thousands of times without control produces weak non-reproduction evidence. Execute against the deployed engine/isolation; SQLite is valuable for some speed but not proof of PostgreSQL/MySQL behavior.

Contract tests verify an interaction boundary. Provider-side schemas/examples can check a client adapter; consumer-driven contracts can reveal assumptions before producer deployment. They do not replace sandbox/live smoke tests for authentication, quotas, TLS, or undocumented provider behavior. Version fixtures and replay old queue/API payloads during compatibility changes.

Static analysis and type evidence

PHPStan or Psalm reasons about declared and inferred types, control flow, generics, shapes, and unreachable paths without executing code. Native types provide runtime constraints; PHPDoc adds shapes/generics/refinements; runtime parsing validates external data. These layers complement each other.

Adopt a sustainable strictness, baseline existing findings, fail new violations, and reduce the baseline by owner/area. A baseline is migration debt, not proof the ignored errors are safe. Avoid suppressions without the narrow reason. Framework extensions/stubs must reflect runtime behavior or analysis can be confidently wrong.

Static analysis does not execute SQL, prove database isolation, validate configuration, detect every dynamic call, or confirm external schemas. Tests cover representative executions; analysis covers many possible paths under its model. Code review and production observability cover intent and reality neither tool can fully know.

Coverage and mutation testing

Line/branch coverage reports what executed, not what was asserted or which inputs matter. Use it to find surprising unexecuted risk, not as a quality score. A trivial assertion can cover every line; unreachable defensive code can lower a target without reducing confidence.

Mutation testing changes operators, conditions, returns, or calls and checks whether tests fail. Surviving mutants can reveal missing assertions, equivalent mutations, unreachable code, or low-value code. Run it on important deterministic units or changed code first; full-suite mutation can be expensive and noisy. Review survivors rather than chasing a percentage blindly.

Property-based tests generate inputs to assert invariants such as round trips, conservation, monotonicity, or equivalence. They are especially useful for parsers, money calculations, state machines, and serialization. A property must be meaningful; generating more examples of the wrong rule adds false confidence.

CI as staged risk control

Fast presubmit gates commonly include formatting/lint, static analysis, unit tests, and focused integration tests. Heavier database matrices, mutation tests, browser tests, dependency scans, image builds, and deployment smoke tests can run in parallel, on affected areas, or after merge according to risk and cost. Required checks should be deterministic and owned.

Cache dependencies and test artifacts carefully: a cache key must include lockfiles, runtime, extensions, configuration, and relevant tool versions. Pin actions/images and minimize CI credentials. Treat pull-request code as untrusted where forks can reach secrets. Publish test reports and failure artifacts without leaking environment values.

A red gate can block unsafe delivery only if teams can diagnose it quickly. Track duration, queue time, failure/flaky rate, and time to repair. Optimize the critical path with parallelism and test selection, but retain periodic full validation to catch bad dependency graphs. Emergency bypasses need authorization, audit, compensating verification, and follow-up—not a hidden rerun-until-green culture.

Production is additional evidence

Preproduction cannot reproduce every dataset, load, dependency, clock, or topology. Use progressive delivery, health checks, canaries, feature flags, and rollback/roll-forward with observable business and system metrics. Smoke tests after deployment should be safe and identify their data. Synthetic checks and real-user/trace evidence validate contracts that tests approximate.

Production monitoring does not excuse missing tests, and tests do not excuse missing monitoring. Incidents should produce the smallest durable regression evidence at the right layer plus improvements to detection or recovery when prevention is impractical.

Current and legacy context

Current: PHPUnit, Pest, PHPStan/Psalm, and Infection are evolving tools; pin compatible versions and use their current official documentation.
Common: Mixed PHPUnit/Pest syntax, static-analysis baselines, framework feature tests, and containerized real-database integration are normal.
Legacy: Large mocked unit suites and coverage thresholds may encode real organizational constraints. Ratchet toward risk-focused evidence instead of rewriting syntax without improving confidence.