Fixture-first CLI
Run deterministic checks against saved baseline and candidate outputs without API keys, model calls, or external dependencies.
For AI product builders and prompt consultants
Prompt Regression Test Harness
Compare baseline and candidate prompt outputs, score the differences with local checks, and generate an HTML report before prompt changes reach users or clients.
What you get
A small prompt testing harness for reviewable release checks.
Use it when a prompt, example, model setting, or system instruction changes and you need a concrete before-and-after report instead of gut feel.
Reviewable reports
Generate HTML and JSON reports with pass rate, regression count, check notes, and side-by-side output comparison.
Practical checks
Use required terms, blocked terms, regex patterns, word limits, critical gates, and rubric criteria.
Workflow
Turn prompt review into a repeatable release step.
-
01
Save the current trusted prompt output as the baseline.
-
02
Run the changed prompt and save the candidate output.
-
03
Score the candidate with local checks and critical gates.
-
04
Review the HTML report before shipping the prompt change.
Deliberately local
No live model adapter required.
This first version is fixture-based so prompt teams can agree on expected behavior without adding API cost, credentials, or network variance. It is one QA layer, not a guarantee that every AI output is correct or safe.
One-time purchase
Get Prompt Regression Test Harness for $39.
Download the CLI, sample suites, JSON schema, documentation, changelog, and license.
Open Gumroad checkout