Every LLM feature demo looks the same: a founder shares their screen, types a question, and something impressively fluent comes back. The room nods. The feature ships. Then the ninety-day clock starts.

By day ninety, in most teams we meet, the prompt has grown to eleven paragraphs nobody fully understands, costs have tripled because retrieval pulls twice the context it needs, and the one engineer who understood the eval spreadsheet has moved teams. This essay is the playbook we now run on every engagement to make day ninety look like day three.

Evals before features

We do not write a single line of product code until there is an eval set — usually 50 to 150 real examples with agreed-correct answers, drawn from the client’s actual data. It feels slow for the first week. It is the fastest thing we do all quarter, because every prompt change, model swap and retrieval tweak afterwards gets scored in minutes instead of argued about in meetings.

The eval set also forces the most valuable conversation of the project: what does "good" actually mean? Teams that cannot agree on that in a spreadsheet will not discover agreement in production.

Retrieval is the product

Nine times out of ten, when output quality disappoints, the model is fine and the context is wrong. We spend far more engineering time on chunking, filtering and ranking than on prompt wording. A smaller model with excellent retrieval reliably beats a frontier model fed a junk drawer.

Our rule: the prompt gets attention last. It is the cheapest thing to change and the least likely to be the real problem.

Observability from day one

Every request logs its prompt version, retrieved context, token counts, latency and cost — from the first deploy, not after the first incident. When quality drifts (and it will, silently, when a model version rolls over), you want a dashboard that shows exactly which eval cases regressed and when.

None of this is glamorous. That is rather the point: LLM features that survive are built like production software, because that is what they are.