指南
How to Choose an LLM for Your App
A practical, field-tested checklist for picking an LLM — covering quality, cost, latency, lock-in, regions, compliance and licensing.
Picking an LLM is rarely a single decision; it's a sequence of trade-offs. This guide walks through the questions production teams actually argue about — quality, cost at scale, latency, vendor lock-in, region availability, compliance, and license — and the order in which to settle them. The goal isn't to crown one winner, it's to help you stop oscillating between two-or-three reasonable choices.
Step 1 — Define the workload, not the model
Before comparing models, write down the workload in concrete terms: what does the input look like (length, modality, language), what does the output need to be (free text, JSON, tool calls), and what's the volume per day? A single short reply at low volume tolerates almost any model; a 200K-token RAG answer at 10K req/day rules out 80% of the catalogue.
Two requirements decide most of the search space: the maximum input length you must support and whether the model has to call tools. Write those down first.
Step 2 — Eliminate by hard constraints
Apply each of these as a true/false filter — anything that fails is out, no matter how strong it scores elsewhere:
- Modality — does it accept the input you actually have? (text, image, audio)
- Tool calling / structured output — required for agents and reliable JSON pipelines.
- Context window — must comfortably fit your largest expected prompt + reply.
- Region / data residency — EU, US-only, on-prem? This usually halves the candidate list.
- License — Llama's 700M-MAU clause and similar acceptable-use clauses can disqualify whole families for some products.
Step 3 — Estimate cost honestly
Don't price-shop on headline rates alone. Real production cost is dominated by three factors most comparison tables ignore:
- Input/output mix — output tokens cost 3-5× more than input. A model with cheap input but expensive output can lose to a 'more expensive' model with a flatter ratio.
- Prompt caching — for system prompts you reuse, Anthropic and OpenAI's cache_read rates can drop input cost 5-10×. Unlock this only if your provider supports it.
- Reasoning tokens — thinking models (o-series, Claude Extended Thinking, Gemini Thinking, DeepSeek R1) bill internal reasoning at output rates. Real cost is typically 2-3× the visible answer length.
Step 4 — Test quality on your actual inputs
Public benchmarks are weak signal at this stage — they correlate roughly with real-world quality but never settle disputes. Build a 50-200 example eval set from real production traffic (PII-redacted), then sample-rank with the top 3-4 candidates. The cheapest model that meets your bar always wins.
If you don't have enough traffic to build an eval set, two cheap shortcuts: (a) ask each candidate to grade the others' outputs side-by-side, and (b) instrument a 1% production canary on the candidate alongside your incumbent.
Step 5 — Diversify against lock-in
If your business depends on the LLM call, a single-vendor failure mode is operationally cheap to ignore right up until it isn't. Three diversification patterns worth setting up early:
- Two providers, one model — many open-weight models are hosted by 5+ providers. Keep two configured and fail over on errors.
- Two models, one provider — most providers offer a 'flagship + mini' pair. Auto-downgrade on rate limits.
- Two vendors, equivalent models — when SLAs really matter, run an alternative vendor warm. Picking similar-capability models in our /alternatives pages is a good starting point.
Common mistakes to avoid
After reviewing many production picks, these are the recurring traps:
- Chasing benchmarks — benchmarks measure narrow capabilities; your workload is a different distribution.
- Optimising for headline price too early — the cheapest model that fails your eval is the most expensive one to ship.
- Overweighting context size — 1M context is rarely better than RAG over a 200K window in real apps.
- Ignoring latency — interactive UIs need TTFT (time-to-first-token) under 1s. Some 'cheap' models break this badly.