Create an account to save sources and durable progress.
Essays - Benedict Evans · Benedict Evans
articleinsightSource excerpt
About Essays - Benedict EvansEvery week there’s a better AI model that gives better answers. But a lot of questions don’t have better answers, only ‘right’ answers, and these models can’t do that. So what does ‘better’ mean, how do we manage these things, and should we change what we expect from computers?
This is a limited feed-provided excerpt, not the full original work.
Apollo layer
Define AI quality by the job being done, not by general model capability. Separate tasks where incremental improvement is valuable from tasks that require a verifiably correct answer, then design evaluation and human oversight accordingly.
A broadly better model may still be unsuitable for workflows where one wrong answer creates outsized risk. Task-specific standards help founders set honest expectations, choose models rationally, and build safeguards where correctness matters.
List your product’s top AI-assisted tasks. For each, ask: Is a plausible improvement useful, or must the output be correct? What test, constraint, or review step matches that standard?
Private notes
© Apollo Advisors° ABN 34 346 108 139