Choosing models: a decision tree for people with budgets

Model choice gets argued like religion. It's actually arithmetic. My decision tree, after a year of shipping AI features on real constraints:

1. Find the capability floor, empirically

Take your ten hardest real inputs — not toy examples, real ones from logs — and run them through the cheapest models first. The smallest model that clears the bar on all ten is your candidate. Everything bigger is charity to the vendor.

2. Price the feature honestly

Cost per call × expected calls per user × users. I've watched features die not because the model was weak but because the economics were: a "free" tool that costs its maker four cents per message doesn't survive a good launch week.

3. Feel the latency

Users tolerate slow answers for hard questions and not for easy ones. Route: cheap fast model for the easy path, escalate only when confidence is low or the input is complex. Most requests never need the flagship.

4. Then, and only then, compare vendors

Switching costs are real but smaller than people fear — if you kept prompts, outputs, and evaluation in files rather than in a vendor's dashboard, migration is an afternoon.

The uncomfortable summary: most teams use models ten times more capable than their task requires, and their moat was never the model anyway.