Contract Mocking: Structure From the Spec, Values From the Model

A hand-written mock starts accurate and decays. Then your tests assert your code works against a contract that no longer exists, and the failure moves to production.

Mocks that drift are worse than no mocks

A hand-written mock of an upstream service starts accurate and decays. The upstream adds a required field, changes a status code, renames something in a minor release — and your tests keep passing against a mock that describes last quarter's API.

The tests are then actively harmful: they assert your code works against a contract that no longer exists, and they fail in production instead.

The fix is to generate mocks from the contract rather than writing them, so the mock cannot describe an API the spec does not.

Generate the skeleton, do not write it

An OpenAPI document already contains everything a basic mock needs: paths, methods, response schemas, status codes, and often examples. Standard tooling turns that into a running server with no model involved at all.

# A mock server that always matches the spec
npx @stoplight/prism-cli mock openapi.yaml --port 4010

# Validate that YOUR requests conform, too
npx @stoplight/prism-cli proxy openapi.yaml https://api.upstream.com \
  --errors                 # fail loudly on any request/response mismatch

Start there. Schema-driven mocking solves the drift problem outright, and anything a model adds is on top of a base that is correct by construction.

Where a model does add something

Spec-generated responses are structurally valid and semantically useless. Every string is "string", every integer is 0, every date is the same. That is fine for checking your parser and useless for checking your logic.

{
  "id": "string",
  "customer_name": "string",
  "total_cents": 0,
  "currency": "string",
  "status": "pending",
  "created_at": "2019-08-24T14:15:22Z"
}

{
  "id": "ord_7fa2c918",
  "customer_name": "Ana María Ruiz-Ferreira",
  "total_cents": 1299500,
  "currency": "COP",
  "status": "partially_refunded",
  "created_at": "2026-02-29T23:59:59.999Z"
}

The second one exercises things the first does not: a non-ASCII name with a hyphen, a currency with no minor units where cents arithmetic is wrong, a status people forget to handle, and a timestamp on a leap day at the boundary of a second. Generating that variety is a genuinely good use of a model.

Given this OpenAPI schema for `Order`, produce 12 fixtures that
are all VALID under the schema but stress different cases:

  - minimum and maximum for every constrained numeric field
  - empty and maximum-length for every string
  - every enum value, including ones our code probably ignores
  - unicode, RTL text, and names with apostrophes or hyphens
  - a currency with no minor units (JPY, COP)
  - timezone boundaries: DST transition, leap day, 23:59:59.999
  - optional fields present and absent, in different combinations

Output JSON only. Every fixture must validate against the schema —
I will check.

The last line matters, and so does actually doing it. Validate every generated fixture against the schema before it enters your suite; a fixture that does not conform is testing a case that cannot happen, which wastes time and creates false confidence.

import json, jsonschema, sys

schema = json.load(open("schemas/order.json"))
fixtures = json.load(open("fixtures/orders.json"))

bad = 0
for i, f in enumerate(fixtures):
    try:
        jsonschema.validate(f, schema)
    except jsonschema.ValidationError as e:
        print(f"fixture {i}: {e.message}")
        bad += 1
sys.exit(1 if bad else 0)

Error responses are the half everyone skips

Most mock setups cover the happy path and a 404. Production failures come from the rest.

Rows two and six are the ones that produce the most production incidents, because both look like success until something reads a field. A gateway returning an HTML error page with a 500 breaks JSON parsing in a way that surfaces three frames away from the cause.

Recording rather than inventing

A third option sits between spec-generated and model-generated: record real responses and replay them. For an upstream you can call in staging, this produces fixtures with genuine data shapes nobody would think to invent.

import vcr

@vcr.use_cassette(
    "fixtures/cassettes/orders_list.yaml",
    record_mode="once",
    filter_headers=["authorization", "x-api-key"],
    filter_query_parameters=["token"],
)
def test_lists_orders():
    orders = client.list_orders(status="shipped")
    assert len(orders) == 3

The filter_headers argument is not optional. A recorded cassette committed with a live bearer token in it is a credential in git history, and this is a common way it happens — the recording captures whatever the real request contained.

Recorded fixtures also go stale, so re-record on a schedule and diff. A cassette that changes when you re-record is telling you the upstream changed, which is the same signal the conformance check gives you and arrives through a different route.

Keep the mock honest against the real thing

Generated-from-spec mocks solve drift between the mock and the spec. They do not solve drift between the spec and the actual service, which is a separate and common failure.

#!/bin/bash
# Does the real service still match the spec we mock against?
set -e
npx @stoplight/prism-cli proxy openapi.yaml "$UPSTREAM_URL" \
  --errors --port 4010 &
PRISM=$!
sleep 3

# Replay a small set of real request shapes through the proxy.
# Any spec mismatch makes prism return an error.
./scripts/replay-recorded-requests.sh http://localhost:4010

kill $PRISM

Run it nightly against staging. When the upstream changes without telling you — which is the normal case, not the exceptional one — you find out from a failing job rather than from an incident.

What to hold to

1. The spec is the source of truth. Mocks are generated from it, never alongside it.
2. A model generates fixture values, never fixture structure.
3. Every generated fixture is schema-validated before it is committed.
4. Error and edge cases get as much coverage as the happy path.
5. A scheduled job checks the real service still matches the spec.

Point two is the whole discipline in one line. Structure comes from the contract, values come from wherever produces interesting ones, and confusing the two is how mocks start lying.