In October, more than one in four requests on VercelâÂÂs AI model router went to decision models, not LLMs â an enormous share of work for a kind of computerized intelligence that was invented only a month earlier.
Decision models are LLMs that donâÂÂt produce output tokens; instead, theyâÂÂre tuned to use their general intelligence to quickly and cheaply decide between preset options, while registering its level of confidence in the choice. TheyâÂÂre useful in production environments, where fast decisions can be more important than the reasoning provided by a slower and more expensive frontier model.
TypeSafe became a $7.5 billion company after releasing its Jev family of models, which continues to dominate usage (at least where we can see metrics from services like OpenRouter, which handled more than 1.5 billion queries to TypeSafeâÂÂs models last week).
However, itâÂÂs easy to make a decision model, and dozens were released in the wake of Jev. One Amazon engineer made one in a weekend, and soon every hyperscaler had its own option. On Tuesday, Microsoft CEO Satya Nadella said that his company is rolling out its decision model, Microsoft-Decision-1.1, for wide use on Azure immediately.
The action is happening on both ends of the ecosystem: Sqwish, a UK-based AI startup behind a decision model called kettei fat, announced a $3 million pre-seed round today. Cofounders Ushnish Sengupta and Federica Freddi, who met as Cambridge grad students, had been developing small LLMs when Jev arrived on the scene, and they realized their architectures could be repurposed as a powerful decision model.
Data appears to be the key to a successful model in this category â TypeSafe CEO Diogo Almedia told TechCrunch that the hardest part of building Jev was finely calibrating its judgement using statistically-adjusted synthetic data. Sqiwsh, too, has been working with synthetic data, but its unique offering is built-in post-training tools that use a customerâÂÂs own data to improve the modelâÂÂs judgement.
âÂÂWhat makes us not a commodity, I think, is the fact that we are offering our customers the same powerâ¦they can keep uploading their production data, and just the same way that we trained our models, their models can also keep getting better at the tasks that they care about,â CEO Ushnish Sengupta told TechCrunch.
Attempting to rank decision models is difficult; SqwishâÂÂs âÂÂkettei fatâ model has performed well on some benchmarks, but most builders of the technology say that the real proof is in the pudding: itâÂÂs generally simple for developers to sub in one model for another and see how it does in a specific context.
âÂÂThe benchmark is more of an invitation to try the model out, and then the continual tuning is the reason why why they should stick to us,â Sengupta says.
MicrosoftâÂÂs Decision 1.1 model is also the result of the company carefully balancing across cost, latency, intelligence and calibration. Achint Srivastava, the engineering VP who led the project, agreed that too much focus on benchmarks is a mistake.
âÂÂWhat we see is that âÂÂbenchmaxedâ models fall flat under real enterprise usage very quickly,â Srivastava wrote in an email. âÂÂOur testing underscores just how challenging it is to deliver accuracy, reliability, speed and cost efficiency in a single model. Microsoft-Decision-1.1 and Jev are among the few that perform strongly across all four dimensions, even when evaluated on âÂÂheld outâ or unseen benchmarks reflective of real use cases.âÂÂ
For startups like Sqwish, competition with hyperscalers and frontier labs is an opportunity. Freddi predicts that future systems will see so-called System One intelligence, like decision models, working in concert with System Two intelligence, like frontier LLMs.
âÂÂEven in that future, System One is maybe a $10 billion, $50 billion [opportunity], and OpenAI is doing this IPO with you know $1 trillion, $2 trillion market value in mind,â she told TechCrunch. âÂÂThis is all we care about; whereas for them, this is just a side project.âÂÂ
When you purchase through links in our articles, we may earn a small commission. This doesnâÂÂt affect our editorial independence.
