What have we learned from one year of conducting AI-Powered User Interviews
Natan Voitenkov
Aug 26, 2024
•
7
min read
The original post has been written in Hebrew for the Startup for Startup community by Monday and can be found here. Below you can find an English translation, enjoy the read.
For those who don’t know me, I’m Natan, co-founder and CEO of Genway AI. Long before (and hopefully long after) Genway AI, I am a sailor, triathlete, and scuba diving enthusiast. Today, I’m excited to share our insights from a year of conducting AI-Powered User Interviews — meaning using an AI Agent to perform Product Feedback interviews based on a specific interview objective.
Over a year ago, Gal Dayan (CTO) and Omri Ben-Shoham (VP of R&D), and I decided to leave our high-tech jobs and jump on this crazy rollercoaster called a Startup. Why specifically focus on user interviews using an AI Agent? Because we identified a gap — in recent decades, we’ve seen a lot of innovation in product development (with tools like Figma, Miro, AWS, Atlassian, and recently OpenAI), but not much in the way of understanding customers or users, especially at scale. What started as a simple idea quickly became an MVP in Google Slides, and eventually, an end-to-end platform that’s now being used in the production environments of enterprise companies.
Genway AI Founders: Omri Ben Shoham (VP R&D), Natan Voitenkov (CEO)and Gal Dayan (CTO) - left to right
TL;DR
AI breaks down barriers for in-depth conversations on sensitive or complex topics
AI enables Customer Obsession at scale
There's a strong need to use AI for customer understanding, given the complexity of products we're building
AI is available anytime, in any language, and for any need
Deep concern about disruption that could leave workers without their jobs
Insight #1 - Artificial Intelligence Breaks Down Barriers for In-Depth Conversations on Sensitive or Complex Topics
When customers engage in conversations with product developers, they are aware of two things:
First, these are the people who decided what to build and how; they have a vested interest.
Second, they are human—and therefore may feel hurt, offended, or embarrassed during the conversation, which can make the interaction difficult.
Additionally, there are a wide range of topics that are particularly uncomfortable to discuss, especially with those involved in product development. For example, subjects like health, finances, and dating are sensitive areas, as are unpleasant scenarios like interviewing customers who have abandoned your product (churn analysis). Unlike a human, who has emotions, worldviews, and the capacity for self-criticism, users perceive an AI Agent as an “it” (rather than “he” or “she”). Because of this, they don’t hesitate to share their true opinions—no matter how harsh or complex they may be.
Insight #2 - Artificial Intelligence Enables Customer Obsession at Scale
If you’re reading this, you likely consider yourself customer-obsessed. The theoretical importance of customer focus is clear to every founder and product person. But in the pre-AI era, how was this obsession truly manifested? If you’re only talking to a very limited percentage of your customers, and even then, it’s only happening once or twice in the customer’s lifetime—where’s the obsession? Until now, the distance from the customer remained large due to operational limitations. AI allows for direct, immediate, and in-depth contact with every customer, enabling hyper-personalization of the product, experience, and service. Recently, an American enterprise company used our services to interview over 250 customers in just two days—a project that would have taken weeks, if not months, without AI support. It can be said that for the first time since Jeff Bezos spread the message of customer focus more than 25 years ago, we can truly achieve it by leveraging AI Agents.
Insight #3 - There’s a Strong Need to Use Artificial Intelligence for Customer Understanding, Given the Complexity of the Products We’re Building
The paradigms of customer understanding and user research were established decades ago and haven’t fundamentally changed since. However, what has changed dramatically is the complexity of the products we’re building and the technologies that power them. Additionally, we now recognize that users are divided into segments with fundamentally different needs that must be understood to serve each segment effectively. Those who still rely on a series of interviews with five users before a product launch, or a single survey after launch, are likely missing critical insights for product success. And those who depend solely on analytics, focusing on behavioral data and metrics, miss the context and reasons behind those behaviors. AI Agents bridge the gap between qualitative and quantitative customer understanding, providing deep and comprehensive insights simultaneously.
Insight #4 - AI is Available Anytime, in Any Language, and for Any Need
The connection between customers/users and technological teams is often limited by time constraints (and language barriers, if you’re serving an international community). Additionally, different teams have varying user understanding needs—such as a Customer Success Manager versus a Product Manager versus a UX Researcher. With an AI Agent, availability is constant, conversations can be conducted in any language, and the nature of the conversation can be tailored to the specific needs of each team. Because of this constant availability, it’s possible to engage with customers throughout their entire journey with your product—from the onboarding stage, through the initial use of new features, and even (if necessary) to automatically and immediately understand churn.
Insight #5 - Deep Concern About Disruption That Could Leave Workers Without Their Jobs
In today’s established companies, there are several teams that can be classified as Insight Generating Functions. These teams are responsible for gaining a deep understanding of customers/users in all aspects, including market research, user experience research, passive feedback collection, and analytical analysis of user behavior and experiment results. These teams are now questioning what their role will be in the era of AI Agents and are concerned about the potential implications. At the same time, they are eager to use advanced tools to increase their impact and better serve their customer base. The reality, as always, likely lies somewhere in the middle. The role of these teams will undoubtedly change and evolve, but it won’t disappear. Instead of focusing on conducting interviews, these teams will concentrate on proper planning, in-depth analysis, and collaborating with product, design, and R&D teams to effectively "activate" the insights.
A Few Closing Thoughts
Conducting user interviews with an AI Agent is not suitable for small companies with a limited number of end users.
It’s important to set clear expectations with users and be transparent about using an AI Agent for the interview.
Using an AI Agent for conducting user interviews is not meant to replace 1:1 conversations with customers, but rather to expand your reach as an organization that aspires to be truly customer-centric.
More insights
Generate insights with depth and scale using AI Interviewers