Salesforce and Nvidia’s new reasoning model is everything the AI labs should fear

A new AI model called Koa is one of the biggest announcements from Salesforce this week at its giant Dreamforce tech conference. Koa is the company’s first reasoning model, built on Nvidia’s open-weight Nemotron model. The two companies worked together to post-train Koa to excel at sales, marketing, and customer-support-related tasks.

Koa is a shining example of how the enterprise world’s needs for AI are diverging from what the frontier labs are offering. Proprietary AI labs would rather have enterprises uploading files, code, prompts, and feedback directly into their models and agents, and spending millions to do so.

But with the model, Salesforce is offering its enterprise customers:

• an open-weight alternative to closed frontier models
• a model trained to do specific work tasks (rather than to solve impossible math problems)
• one that has not ingested any actual customer data and therefore cannot leak it to others
• a model that helps reduce AI spending, since it uses fewer tokens to do the same work
• one that can be automatically routed through an AI “gateway,” depending on the need
• and a model that follows all of a customer’s data requirements and security embedded within Salesforce.

Koa will be provided as an alternative to the other models Salesforce offers in its Agentforce platform, where its customers build agents to handle rote tasks like answering customer service questions or scheduling appointments.

“We’ve built many small task-specific language models, which are part of Agentforce’s portfolio,” Jayesh Govindarajan, EVP of Salesforce AI, told TechCrunch. “But reasoning has always been something that we’ve relied on the frontier model providers for. Until now.”

Before Koa, if an agent needed to reason through a long-running or multi-step task, those prompts would be routed to a frontier model like Claude or ChatGPT through Agentforce’s AI gateway (the system that decides which model handles which request).

“One of the reasons we hadn’t done this before, train our own enterprise-grade frontier model — we always wanted to — but the challenge has always been the lack of a pre-trained base model to start with. Until Nemotron came along, there was no sovereign American pre-trained model that was available, one, and two, that was state of the art, and, three, that had clear data provenance. We have no idea what Qwen trains on,” Govindarajan said, referring to the popular Chinese open-weight model produced by Alibaba.

Post-training a model like this means taking it from a general-purpose system to one well-versed in sales and customer support knowledge, and to do that, Salesforce and Nvidia did not use any actual data from Salesforce’s customers. Instead, they crafted synthetic data that mimicked customers’ patterns.

“We actually simulated a customer service environment with a persona customer service professional, including irate customers that call into the customer service center, all the way to a sales professional who’s trying to close a deal,” Govindarajan described.

Koa is meant to be better at the work tasks Salesforce customers want an agent to do — and cheaper, in terms of tokens burned — than sending those same tasks to Claude or ChatGPT.

With Nemotron, “we have a unique architecture for inference to be token efficient,” Kari Ann Briski, Nvidia’s VP of Generative AI Software for Enterprise, told TechCrunch. “It’s kind of the trifecta of things that you need to have: sovereign AI, time to first token, efficient reasoning, for the tokenomics of it all.”

However, Salesforce isn’t exactly abandoning Anthropic or OpenAI. It just announced a partnership with Anthropic called ClaudeForce that allows companies to use Claude as their AI interface, while their data remains in Salesforce’s system of records, secured by its infrastructure.

Topics

When you purchase through links in our articles, we may earn a small commission. This doesn’t affect our editorial independence.