CASE STUDY · HUMAN VS. SYNTHETIC TESTERS
Can Synthetic Testers Think Like Real Shoppers?
We gave 25 real shoppers and 25 Synthetic Testers the same challenge: find and choose the best cordless vacuum on Amazon for under $300, Dyson or Shark. Then we compared how they researched, reasoned and chose.
HUMAN VS. AI TESTER
Human shopper
Real participant
Dyson
Cordless Stick Vacuum V8
$189
Shark
Cordless Stick Vacuum Pro
$249
Is the Pro worth $60 more? Not sure...
AI tester
Synthetic persona
Dyson
Cordless Stick Vacuum V8
$189
PRICEShark
Cordless Stick Vacuum Pro
$249
COMPARING 2 ITEMS
Comparing feature claims across items
- 25 human shoppers
- 25 Synthetic Testers
- Same tasks. $300 budget.
The Results
Different Shoppers.
Remarkably Similar Results.
Humans and Synthetic Testers shopped separately. Their choices, priorities and conclusions ended up closely aligned.
Chose the same brand
21 of 25 Synthetic Testers picked the same brand as their matched human shopper.
Major findings surfaced
Of the human study’s major findings, counting full and partial matches.
Shared the same priorities
Synthetic Testers reproduced most of their human counterpart’s top purchase factors.
Same winning brand
Shark won both studies: 56% of humans and 64% of Synthetic Testers.
The Experiment
One Shopping Challenge. Two Kinds of Shoppers.
Every session happened live on Amazon.com, not in a survey or a simulation.
Participants searched, opened product pages, read reviews and added items to their cart while thinking out loud. The same AI moderator guided both groups, so every human and Synthetic Tester got identical instructions.
Find a Dyson cordless vacuum for $300 or less.- Find a Shark cordless vacuum for $300 or less.
- Compare products, add both to cart, and choose one.
- Explain the decision. No order is placed.
How Synthetic Testers Are Built
We Didn’t Invent the Personas. We Learned From Real Humans.
TheySaid’s AI Research Assistant studied the human sessions to understand each shopper’s motivations, beliefs and product knowledge, then used those insights to shape the Synthetic Testers.
Run human research
25 real shoppers complete the challenge while their sessions and answers are recorded.
Analyze behavior
AI reviews the recordings, transcripts, answers and purchase decisions.
Build AI personas
Observed motivations, worldview and knowledge become the inputs for each Synthetic Tester.
Run AI testing
25 Synthetic Testers complete the same Amazon task with the same moderator.
Compare results
Decision factors, shopping behavior, brand choices and key findings, side by side.
The Three Building Blocks of Every Synthetic Tester
Motivation
What they’re trying to get done and what matters most to them.
Worldview
The attitudes, biases and trust that shape how they judge products and brands.
Knowledge
What they already know about your category before they start.
Both Studies Crowned the Same Winner.
56% of human shoppers chose Shark, and so did 64% of Synthetic Testers. The two groups were just two shoppers apart, and their average confidence in the choice was almost identical.
Human shoppers
56% of humans preferred Shark over Dyson
Synthetic testers
64% of Synthetic Testers preferred Shark over Dyson
70% of Human Insights. Found by Synthetic Testers.
Analyzed as two separate studies, humans and Synthetic Testers surfaced the same core reasons, concerns and trade-offs.
Major human findings covered
What both studies discovered
Price is the #1 driver
For humans and Synthetic Testers alike.
Reviews build confidence
Ratings and review volume act as social proof.
Pet hair decides purchases
Anti-tangle claims got close scrutiny.
Brand trust breaks ties
Dyson’s premium image vs. Shark’s value reputation.
One bad review can flip a sale
Negative reviews changed minds in both groups.
FINDING 03 · BEHAVIORAL DIFFERENCES
Humans shopped like test participants. Synthetic Testers shopped like buyers.
Human shoppers knew they were completing a user test, so many approached the experience like a task to finish. Synthetic Testers behaved more like serious buyers: they spent more time researching, compared more options, read more reviews, and gathered more evidence before making a decision.
HUMAN SHOPPERS
Practical, Personal, and Selective
Humans often decided once they had enough information, bringing personal habits, taste and emotion into the choice.
- Usually considered 1–2 products per brand
- Often went by the star rating
- Stopped at a “good enough” option
- Weighed suction, design and past brand experience
SYNTHETIC TESTERS
Thorough, Analytical and Persistent
Synthetic Testers gathered more evidence before deciding, reading closely and comparing carefully.
- Often compared 2–4 products per brand
- Read review summaries and rating breakdowns
- Asked for hard specs like runtime, suction and warranty
- Kept exploring alternatives before committing
“Synthetic Testers got much of the big picture right. Humans added real-world nuance.”
Download the Full Case StudyFINDING 04 · WHERE AI GOES FURTHER
Synthetic Testers Don’t Skim.
Real shoppers often glance at the stars and stop at “good enough.” Synthetic Testers read every review breakdown, compare more products and ask the questions your product page should already answer.
More products compared
Synthetic Testers often evaluated 2–4 options per brand.
Most humans looked at 1–2.
Deeper review reading
They went past the star rating into review summaries and rating breakdowns.
A new insight humans missed
They kept asking for battery runtime by power mode, real suction numbers and warranty length. That’s a clear signal shoppers want hard proof, not vague claims.
WATCH THE SESSIONS
Don't Just Read the Results.
Watch the Decisions.
Same task. Same website. One human, one Synthetic Tester. See what they looked at, what they said and why they chose what they chose.
How Human Shoppers Decided
Gut reactions, past experiences and the moment research feels good enough.
How Synthetic Testers Decided
Side-by-side comparisons, deep review reading and pointed questions about the specs.
ABOUT THE RESEARCH
Methodology
How the study was run, and how to read the results.
This study compared 25 US shoppers with 25 Synthetic Testers, each matched to one of the human participants. Everyone completed the same Dyson-versus-Shark cordless vacuum task live on Amazon, with a $300 budget and the same AI moderator.
We compared the two groups on top purchase factors, shopping behavior, final brand choices and major findings, both pair by pair and as two complete studies.
Good to know
Amazon personalizes listings, prices and offers, so shoppers didn’t always see identical pages. A few sessions were affected by technical issues. The results show strong directional agreement in this scenario, and Synthetic Testers work best alongside human research, not as a replacement for it.
SEE IT BY YOURSELF
No Data Will Convince You Like Trying It Yourself.
Run real users and Synthetic Testers side by side on your next project, and compare the results.
