The Contract Matrix: SQLite Meets Billing

"Works on SQLite" ships broken on Supabase

Picture a webhook handler tested on Mongo that hits Supabase in production and dies on a row-level policy nobody's test exercised.

The SQLite suite is green. The Mongo suite is green. Supabase, the engine customers actually use, has never run the billing path once.

Each engine passed its own tests. The combination of this provider, that engine and a money flow went unverified.

The matrix had holes exactly where revenue flows. All green dashboards. Production red.

Test every action, every engine and every money flow. Combined, not sampled.

Sampled or matrix

Intersections teach three lessons.

Bugs live in combinations. SQLite-correct plus Supabase-strict equals denied writes that neither suite catches alone. The failure exists only at the intersection. Production runs intersections exclusively.

Providers differ where it matters. Same actions, different payloads, different failure modes. Signature formats. Retry shapes. Refund flows. Uniform actions do not imply uniform behavior.

A scheduled full run beats a per-push partial. Breadth costs minutes CI cannot spare on every push. Run the quick checks on every push and the full set on a schedule.

What to do

Contract-check every action on every engine.

The kit includes a contract check. It confirms each database provides the full set of actions the app calls, and that the SQLite schema has the columns the app relies on. It also checks that every payment provider implements the shared interface.

Run it with the kit's test command. Divergence fails loudly at the action level, instead of whispering in production.

Test each payment provider's reality, not just the shared interface.

For each provider, drill four flows.

A bad signature is rejected.

A replayed event is deduped.

A refund removes access.

The portal round-trips.

Use Stripe's test tools, LemonSqueezy's test mode and Dodo's sandbox. All free.

Split the cadence.

Quick checks on every push. The full set, every engine by every provider, on a schedule such as nightly.

Quick feedback gates merges.

Slow breadth guards the quarter.

Never gate merges on the slow run, because developers route around slow gates.

Never skip the scheduled run either, because gaps compound silently.

Sampled thinking or matrix thinking

The honest limit

SQLite is the engine tested end to end. After a switch to Supabase or MongoDB, run your own smoke test of sign-in, checkout and the dashboard against that engine.

Maintenance costs real effort.

Contract checks per action.

Drills per flow.

Scheduled CI minutes.

A flaky-test duty.

BoringPush publishes no incident counts from this practice. The claim is narrower: untested combinations are where production bugs live.

Know this first

SQLite is the engine tested end to end. Full runs across every engine and provider are yours to schedule.

Maintenance takes real effort.

Why BoringPush is the way to do this

Tested where money moves.

BoringPush ships a contract check for the database actions and a test suite that runs with one command. Billing flows are drilled in test mode.

Your own smoke test closes the gap.

FAQ

Is this overkill for solo projects?

At multi-engine scale it is the highest-return testing you can do. A solo founder cannot staff incident response, so prevention is the only affordable strategy. Minutes of scheduled testing beat one cross-engine midnight.

How do test databases stay isolated?

Use throwaway instances per engine, seeded the same way and torn down after each run. Isolation should be by construction, not convention. Shared test databases leak state between runs and make failures untrustworthy.

Should contract tests live beside providers or centrally?

Write them beside each provider so a change to an action updates its contract in the same PR. Run them from one command so the whole set is always covered. Locality keeps contracts current, and central running keeps them complete.

How are flaky tests handled?

Move a flaky test to a quarantine lane with a tracked issue and a two-week fix-or-delete deadline. Never skip and forget, because a skipped test is a deleted test wearing patience. Flakiness often signals real nondeterminism worth fixing.

Do billing drills need real money?

Never. Use test mode everywhere, plus one live dollar per provider per year as a ceremony, refunded immediately. Test-mode drills are engineering, and the live dollar proves the human path.

When does the matrix grow?

When you add an engine or a provider, run the checks before you adopt it, then add it to the scheduled run. Growth should be a checklist event, not a project.

Keep reading

• requirePaidUser(): One Boolean, Zero Paywall Holes — the guard this matrix proves
• Switch Databases With One Line (Zero Rewrites) — the flip the matrix de-risks
• One Auth Flow, Three Databases, Zero Forced Logouts — users surviving both