Product Talk · Teresa Torres
articleplaybookGet My Eval Test Harness and Run Your First Experiment
2 September 2026Product
Source excerpt
About Product TalkMy in-depth AI evals guide explains what evals are and how to build them. This post gives you the tool I use to run them. Here's the practical problem. To get a baseline, you have to run every input through your prompt, save each output, run every
This is a limited feed-provided excerpt, not the full original work.
Apollo layer
What a founder can learn
Founder takeaway
Before changing an AI prompt, create a repeatable eval harness: run a fixed set of inputs through the current prompt, save every output, and use that baseline to assess experiments.
Why it matters
Without a consistent baseline, teams can mistake a few promising outputs for genuine product improvement. Structured comparisons make prompt decisions more reliable and reproducible.
Relevant guides
Put it to work
Choose a representative input set, record the current prompt’s outputs, then change one variable and compare the new results against that baseline.
Private notes