AI, ROI, and the Future of UX Research: Insights from Thomas Stokes
How are UX teams really adopting AI? What does it mean to quantify research impact when traditional metrics fall short? And where is our field heading in this era of rapid change?
I recently sat down with Thomas Stokes, co-founder of Drillbit Labs, to dive deep into these questions that are keeping so many of us up at night.
Thomas brings a unique perspective—he's not just practicing UX research, he's researching the field itself through what he calls "meta research."
What emerged from our conversation was both reassuring and challenging. The uncertainty we're all feeling? It's real, it's documented, and we're navigating it together.
The AI Adoption Gap We're Not Talking About
Thomas shared some fascinating findings from his analysis of job postings in the UX field. Here's what caught my attention:
While AI mentions in job descriptions have significantly increased year over year, there's a telling pattern in how it shows up.
"More often than not, it's saying you're going to work on an AI feature set or experience. It was less common—I think it was 4 times less common—that would say we expect you to be an expert in using AI in UX workflows."
This reveals something interesting about where we are right now.
Companies are hiring for AI product work while apparently assuming researchers already know how to integrate AI into their workflows. But that assumption might be problematic.
The reality? Many teams lack established practices for AI integration, with most adoption happening at the individual rather than team level.
It's this Wild West approach that led Thomas and his partner Lawton to develop the PAIRED framework.
A Framework for Intentional AI Integration
The PAIRED framework stands for:
Principled
Accountable
Initiated
Reviewed
Enabled
Documented
It's designed to help teams move beyond just "slapping AI onto something" to actually thinking through why and how they want to use these tools.
"We don't want to just use some new AI just because it's a fun different thing. Let's really be intentional about the way that we use this in our work."
The Power of Antagonistic Prompting
One technique that's been particularly valuable in their work is what Thomas calls "antagonistic prompting."
Instead of getting the typical sycophantic responses from AI, he prompts it to be an "articulate antagonist" that challenges his thinking.
"It's almost like a tough academic reviewer," I noted during our conversation.
"Once you've addressed all of its points, then you can be more confident that you've really thought through the decisions you're making."
The ROI Question That Won't Go Away
We spent significant time discussing something many researchers struggle with: quantifying impact.
Especially when you're doing strategic research or when you don't have access to clean product metrics, how do you demonstrate value?
Thomas had a refreshing take on this.
ROI calculations aren't financial reports that will be audited. They're estimates based on reasonable assumptions—and that's understood and accepted in the business world.
"If you're probably the person that's nervous about being inaccurate, in fact, you're probably going to be pretty good at this. Because you're going to document your assumptions, and you're going to make very reasonable estimates."
The Simple ROI Formula
The formula itself is straightforward:
(Return - Investment) / Investment × 100
The art is in the assumptions and estimates for both the return and the total investment across teams.
Why This Matters More Than Ever
What I found most compelling about Thomas's approach is how he frames ROI as a bridge between what researchers care about (user-centered design) and what business stakeholders care about (financial outcomes).
"Good UX is good for business," he said.
"What you're doing is just showing how your user centeredness and their finance-centered thinking actually go hand in hand."
Navigating Career Uncertainty with Stable Foundations
When we turned to the future of UX research careers, Thomas offered something I found both realistic and reassuring.
Instead of making bold predictions about what the field will look like, he started with what we can say for certain:
"People interact with technology. We can agree to that, right? Okay, people will probably continue to interact with technology. Companies will then also continue to make different technologies that people interact with. So someone at those companies has to understand how to make the technology work for the people that use it."
His conclusion? "Something UX-like will be a thing. Just it might change a little bit, and we might have different titles."
A Four-Pronged Approach to Career Development
Thomas shared his three-pronged approach:
Be clear about what actually interests you
Be a keen observer of what skills are in demand
Identify people whose work output you admire and figure out why
But given the current uncertainty, I suggested we need a fourth prong:
4. Developing your ability to cope with uncertainty itself.
Because if there's one thing we can be certain about, it's that change will continue.
The Meta-Lesson About Our Field
What struck me most about our conversation was how Thomas approaches uncertainty.
Instead of panicking about what we don't know, he focuses on building understanding through careful observation and analysis.
Whether it's:
Studying job posting trends
Developing frameworks for AI integration
Calculating ROI
The approach is consistently methodical and evidence-based.
That feels like exactly the mindset our field needs right now.
Not blind optimism or paralyzing fear, but curious, systematic thinking about how to navigate change while staying grounded in what actually matters: making technology work better for people.
The tools might change, the titles might evolve, but that core purpose? That's not going anywhere.
References
The PAIRED Framework - Thomas Stokes and Lawton Pybus
How AI is Changing User Research Jobs - Thomas Stokes
Use of AI in UX - Dr. Maria Panagiotidi
Bad Research Doesn't Stink - Carl Pearson, PhD
Research Impact Framework - Carl Pearson, PhD
Calculate ROI for UX Research - Thomas Stokes
Return to Generalist - NN/g
🎧 Listen to the full conversation with Thomas Stokes on Deep Thoughts with Michelle Handy.
📚 Want to dive deeper? Check out Thomas's work at Drillbit Labs and his newsletter Depth by Drillbit Labs.
What questions about AI, impact measurement, or career development are on your mind? I'd love to hear your thoughts—because these conversations are how we navigate uncertainty together.