Where AI Stands Today
What's stable and usable now, what's still changing fast, and what demos well but isn't reliable in production.
Today's models already do real work in language, code, images, and some tool use. But their capability profile is bumpy — and that's the first thing to know before building a product. stanford-ai-index
Symptoms you'll recognize:
- In a demo it completes the whole task automatically; on your real data it keeps getting stuck.
- The same model writes complex code but miscounts the rows in a table.
- The team argues about whether AI can replace a role without defining the task, quality bar, or cost of error.
Jagged capability
We judge a person's ability on a smooth curve: if they can solve olympiad math, basic arithmetic must be no problem. That inference works for humans because our abilities build on each other. It doesn't hold for models. Their capability comes from the statistical distribution of training data — the more corpus and feedback in one area, the stronger that area. That has almost nothing to do with human notions of "easy" and "hard." Hence the jagged profile: competitive math solved beautifully, table-row counting wrong; full modules written, small text in a screenshot unread. The industry calls this jagged intelligence.
It's not a contradiction, and it's not the model being sloppy. It means one thing: every capability has to be verified on its own; you can't infer it from the neighboring one.
A second gap matters just as much: doing it once and doing it reliably ten thousand times are different things. A demo only needs to run one successful path; a product faces all the failure paths. Between them sit task decomposition, eval, permissions, fallbacks, and human review — that stretch of work turns a model into a product.
Three tiers of maturity
Instead of asking "is AI good enough," put what you want to do into this table.
| Maturity | Typical tasks | How to use it |
|---|---|---|
| Already reliable | Rewriting, summarization, translation, extraction, code completion, drafts | Use as a copilot; outputs are quick to check and cheap to retry |
| Usable with guardrails | Enterprise knowledge Q&A, document analysis, controlled tool use, short agent flows | Provide authoritative sources, clear tools, fixed steps; add automated eval and human fallback |
| Still needs caution | Long unattended runs, high-risk approvals, medical/legal conclusions, open-ended web browsing | Strictly limit permissions and scope; key actions need human confirmation; don't treat output as fact |
Note the last row's point. "Use an LLM for customer support" has no tier by itself — answering product questions is tier one, auto-issuing refunds is tier three. What decides the tier is the consequence of failure, not the technology.
The tier people overrate most
Agents are the most overrated tier of the three.
It's not the model suddenly gaining agency. It's the model doing the same thing in a loop: pick a tool, read the result, decide the next step. So its success rate is multiplicative — a 90% per-step accuracy leaves only about a third after ten steps.
That's why the gap between agent demos and production is the largest. A demo shows one successful path; your product faces a reality where every step can fail and errors accumulate. The longer the chain, the bigger the gap.
The practical move isn't waiting for a stronger model; it's shortening the chain. Split long tasks into segments, and give each segment an automatically verifiable checkpoint — fail one, and stop to bring in a human.
Four questions before you build
When you're unsure whether to build a feature, or how far to push it, ask these four.
- Can the answer be verified? With a database, rules, or a person who can check, automation is appropriate. Unverifiable conclusions with serious consequences shouldn't be handed to users directly.
- Is failure reversible? A draft that goes wrong can be redone; auto-payments, data deletion, and external publishing need gates. Where it's reversible, let go; where it isn't, require a human checkpoint.
- Is the task boundary clear? Input format, available tools, and what counts as done — the clearer, the higher the success rate. "Run my company" isn't an executable task; "tag this batch of tickets into these five categories" is.
- Does the math work? Count model cost, latency, human review time, and failure cost together. Technically possible doesn't mean more cost-effective than the current process — and this is the question people skip most. claude-pricing
The safest starting point: tasks that are high-frequency, time-consuming, verifiable, and reversible. Then grant autonomy in order — propose first, draft next, execute last. Autonomy is not a switch; it's handed over segment by segment.
Baseline note: as of August 2026. "Jagged intelligence" is the industry's common phrase, not a vendor trademark; the three tiers are this article's own synthesis. Capabilities and prices change quickly — check the vendor's current official docs before deciding. stanford-ai-index-2026