Can people actually tell the difference between human-generated content and AI? We recently put respondents in our proprietary insights panel to the test to answer this question.
The short answer? Not as reliably as you might think.
In a recent survey of ~1,200 people, we found only 47% of respondents correctly identified which of two images was created using AI. (Though if you’re trying to spot machine-made imagery, ask your younger team members—respondents aged 18–34 were significantly better at distinguishing AI visuals.)
Written text proved to be easier to decipher: 65% of respondents correctly identified a human-written paragraph over an AI drafted one. But that means approximately 1 in 3 of respondents were not able to distinguish between something written by a person vs. by a machine.
Many other recent studies have delivered similar findings: artificial fluency has reached a point where synthetic content easily passes as authentic. At first glance, this makes a tempting case for automation in the world of market research: If the final output looks the same, why have actual people to write things like surveys or reports? But this thinking conceals a dangerous trap.
As we’ve noted before, the act of writing is where the thinking actually happens. Just because an LLM can generate a polished-looking survey or grammatically flawless report doesn’t mean the output is any good. When market research relies too heavily on AI without human involvement, it yields the ultimate strategic hazard: hollow insights and strategy.
How AI Fluency Creates “Hollow Insights”
Across every industry, brand and marketing teams are facing internal pressure to move faster, compress budgets, and adopt AI tools. At the same time, many research software platforms have begun touting AI-powered capabilities. The result: Responsibilities that used to be “owned” by humans, such as drafting research instruments or analyzing data, are being outsourced to AI tools.
It sounds efficient on paper. But as Wharton professor and AI thought leader Ethan Mollick points out, “AI plausibility leads to human gullibility.” Because machine-generated outputs sound articulate and well-structured, research teams easily fall into the trap of “falling asleep at the wheel.”
Hollow insights and strategy are the direct result of that asleep-at-the-wheel approach. This thinking treats tasks such as survey design or data analysis as mechanical chores that can be outsourced without any negative consequences. But this is a flawed assumption.
Let’s start with research instrument design. When you rely on AI to draft a survey or a qualitative research discussion guide, it will default to an average of what already exists. It will produce safe, arguably obvious questions that will lead to safe, obvious data. It will stay closely within the guardrails of however far you’ve gotten in your thinking up until the point of drafting a prompt, but it won’t go beyond that.
Now compare that to how we approach research instrument design at Campos. A person takes the lead in drafting, forcing themselves to think deeply about our objectives, the client’s industry, the people we’ll be surveying. This process of writing, as painful as it can feel sometimes, forces that person to think deeply enough to spot gaps in logic, challenge assumptions, and ask different and better questions. Then, they present the survey to a team of other people who pressure test their logic and revise the survey further. The result is something none of us could have come up with had we not sat with our objectives, experience, and creativity and done the hard work of writing.
The same rule applies to analysis and reporting. AI can summarize massive datasets, but the output is just a polished data dump. The act of doing that analysis and then writing that report is where the real thinking comes in. Which subgroups of respondents should I be focused on in my analysis and why? Where can I combine data points for more meaningful insights? What data should we set aside as noise and what do I need to elevate as a central finding? And how can I deliver all of this in a way that is clear and compelling enough to hold the attention of the C-suite while making the case for what should come next based on this data?
If you outsource all of this to AI, you end up with research instruments that ask generic questions and reports that summarize everything while explaining nothing. Your deliverables may sound authoritative on the surface, but they’ll lack the insight and strategy required to make high-stakes business decisions with true confidence.
To bring this to life, here are a few real-world examples of how over-reliance on AI has led to hollow insights or worse–the disintegration of the human perspective altogether:
Example 1: Hallucinated Content
According to Legal Dive, nearly three quarters of lawyers are either already using generative AI for their work or plan on using it. Similar to what we see in the market research world, there are now a number of industry-specific AI tools geared toward lawyers. Unfortunately, recent research by Stanford University found that legal AI tools don’t tell the truth, the whole truth, and nothing but the truth.
When testing two popular legal AI tools, Stanford’s study found that Lexis+ AI and Ask Practical Law AI systems produced incorrect information more than 17% of the time, and Westlaw’s AI-Assisted Research hallucinated more than 34% of the time. From questions to understanding the law to interpreting changes to the law, the study showed that these hallucinations range from being simply misgrounded to entirely incorrect, threatening lawyers’ ability to comply with ethical and professional standards of their roles.
Fake citations and other hallucinations have been caught in legal filings around the country (see here for some examples in PA), as the industry scrambles to course correct and emphasize the importance of humans in writing legal documents and fact-checking the work of any AI-produced content.
The same risks exist in market research. While AI can be incredibly helpful in the world of secondary research, for example, it’s imperative to ask AI tools to cite their sources and then carefully vet those sources for reliability and accuracy. Whether you’re in the legal world, the market research industry, or another field, AI will hallucinate to fill gaps in information in order to complete a task.
Example 2: Biased Information from AI Writing Assistants
“Smart replies” in email and text messages are one of the first places where people experiment with AI. This introduction naturally makes people more comfortable with relying on AI writing assistants in other scenarios. But over-reliance on AI for writing doesn’t just carry the risk of producing lower quality deliverables, as noted above. It can also bias your point of view.
Science Advances recently conducted a study with over 2,000 respondents to investigate the way in which AI writing assistants affect users’ attitudes. Researchers exposed participants who were “writing about important societal issues to an AI writing assistant that provided biased autocomplete suggestions.”
The research showed that people’s attitudes were indeed influenced by biased AI autocomplete suggestions, ultimately shifting them in favor of what the AI bot suggested. What’s more, the majority of respondents who were exposed to biased AI suggestions agreed that the AI suggestions “were reasonable and balanced,” indicating a lack of awareness of the biased responses in the first place: “only 19% of participants in experiment 1 and 14% of participants in experiment 2 disagreed with the statement that the suggestions were balanced.” All to say, AI models are not only commonly sharing misinformation, but actively changing the human perspective, often without people even realizing it’s happening.
A Human-Led Approach to Using AI
At Campos, we are surely not anti-AI. We leverage specific, secure AI tools frequently, but in select use cases, and we take a mandatory human-led approach to using AI.
So what does a human-led approach to using AI in strategic market research look like?
- Brainstorming: When other team members are not available to brainstorm, AI tools can be used as an effective jumping-off point for more rigorous thinking.
- Research instrument design: Humans do the intellectual heavy lifting (see above), but AI tools can be used as a pressure tester. Just as we ask our colleagues to poke holes in our drafts and provide constructive feedback, we can ask AI tools to do the same. The key here is we’re not only using AI as our pressure tester. We’re using this as a complement to our human-first approach.
- Data analysis: Again, humans do the deep, strategic work to get us from a data dump to a strategic point of view on what the most meaningful findings are, why they matter, and what should come next. But AI can be very helpful when it comes to querying unstructured datasets, allowing us to make human-led analyses more efficient.
AI can sharpen and strengthen our work, but humans must own context, vetting, and strategic thinking.
As market research leader and founder Abigail Stuart noted in a recent GreenBook article, “AI only matters in market research when it improves decisions, shapes investment, and increases the odds of market success.” Speeding up market research means nothing if the resulting strategy points your team in the wrong direction. If a finding doesn’t hold up under rigorous strategic debate, it doesn’t make it into our clients’ hands.
Ensure Your Strategy Is Built with Human-Led Rigor
The recently published 2026 GRIT Report found that only 44% of brand-side researchers are confident that their organizations are minimizing the risks of AI misuse. And it’s no wonder, amidst ever-increasing pressures to automate and deliver more insights faster and cheaper.
While much attention is paid to the obvious risks of AI misuse, such as hallucinated data, the risk of hollow insights and strategy are just as great. Artificial intelligence can summarize what people say, but it takes human curiosity, critical analysis, and collaboration to decode what they actually mean—and what your business should do about it.
If you are evaluating research partners and want to ensure your investments yield actionable strategies rather than hollow insights, let’s connect.
