Former Plaud exec raises $26M for AI app that listens to you in the background

One of the core promises of AI is that the more it knows about a person, the better it will be able to assist them with tasks and make appropriate suggestions, which, if you’ll believe the tech companies, requires apps and devices to listen, watch or record users’ activity all the time and gather all possible context.

There’s obviously privacy and data security concerns with anything that’s collecting data about you continuously. But startups are building such apps undeterred anyway, arming themselves with the belief that the market for personalized assistance is big enough to swallow such concerns.

Former Plaud exec Peter Mo is building Ambi to do something similar. The app aims to provide real-time intelligence and insights to people by listening to them all the time.

The company said on Monday that it has raised $26 million across multiple funding rounds led by Luminous Ventures, which has backed companies like Meituan and Unitree Robotics, and Original Design Manufacturer (ODM) Huaqin.

Ambi is also releasing apps for iPhones, Macs and Apple Watch. These apps act like meeting note-takers, but users can choose to have them listening so they can capture context all day.

Meeting note-taking companies have been working to automate tasks that follow once calls are done, based on the summaries and action items they gather during the conversation, and using multitudes of connectors to software systems. Ambi wants to take this a step further by having its agent start the work during the meeting itself.

The company’s promise is that its AI agent can proactively give you suggestions or surface relevant knowledge while you are in a meeting, based on your calendar, meetings, knowledge base, and web data. Users can also start a task on one device and continue it on another.

“The way we interact with intelligence is limited. Often the intelligence is trapped on the screen, and if you use hardware like Plaud, you need to wait for recording to transfer to the phone, upload to the cloud, and get intelligence after processing. That’s why we are building Ambi to use AI anytime and anywhere,” the company’s co-founder and CEO, Mo, told TechCrunch.

The company has ambitions to capture more information. Its upcoming desktop app will be able to look at the work that users do and store important parts in its memory to give more personalized suggestions. Apps like Littlebird or Minimi already do something similar.

The startup is also planning to launch its own hardware device, which will have a screen, a speaker, a camera, and microphones to capture context and display information.

Question of privacy

Ambi’s launch comes amidst growing concerns around AI devices that don’t have enough indicators to suggest they are capturing data on people around the user. It is not clear if Ambi’s apps or its upcoming device will have enough indicators to alert third-parties that their conversations are being recorded if the user doesn’t disclose their use of Ambi’s tech.

Some meeting note-takers have even been sued because their tech recorded people’s conversation and data without their consent. Plus, the recent release of Apple’s Voice Intelligence feature, which transcribes speech continuously, has raised privacy concerns.

Mo said that users can turn off ambient data capture on Ambi’s apps and devices at any time, and the feature is off by default. The startup said it doesn’t store any recordings of users and encrypts the data, but it didn’t specify what kind of encryption it uses. Despite these measures, the onus of declaring ambient capture is still on the users. He classified this as an industry-wide problem.

“We have to inform our users that they have to tell others about ambient capture. We, as an industry, haven’t solved this problem [of disclosure]. We have tried to put the information about disclosure in every place possible, but if users try to hide the capture, we can’t just solve the problem,” he said.

Ambi currently has over 50 staff across offices in San Francisco, Beijing, Shanghai and Shenzhen. The company is hiring for its go-to-market function and AI engineers for working on speech recognition models.

When you purchase through links in our articles, we may earn a small commission. This doesn’t affect our editorial independence.