In experiments involving 4,500 participants at the University of Pennsylvania’s Wharton School, people who used LLMs to research everyday topics demonstrated weaker understanding of those topics afterward and produced less original insights than people who looked up the same topics using Google. The research is resonant with similar studies: Students who use AI tools to complete assignments tend to do better on homework—but worse on tests. “They’re getting the right answers, but they’re not learning.” The findings suggest believing information came from an LLM makes people learn less. "They think the system is smarter than them, so they stop trying. That’s a motivational issue, not just a cognitive one.”ChatGPT and other “large language models” promise to make learning easier than ever. But new research suggests that lessons learned so easily are less likely to stick.
LLMs are a form of generative artificial intelligence that communicate like humans, using language. But in a series of experiments involving more than 4,500 participants at the University of Pennsylvania’s Wharton School, people who used LLMs to research everyday topics demonstrated weaker understanding of those topics afterward and produced less original insights than people who looked up the same topics using Google.
The findings raise concerns about how people search and learn, says Wharton marketing professor Shiri Melumad, first author of the research. “It is like the Google Effect on steroids,” she says, in a nod to earlier research suggesting people tend to remember less when information is easy to look up. With LLMs, she says, “We’re shifting even further away from active learning.”
Four experiments
Across four experiments, Melumad and co-author Jin Ho Yun randomly assigned participants to use either Google Search or an LLM to research an ordinary topic and then to write advice based on what they learned.
The first experiment asked more than 1,100 participants to use either Google or ChatGPT to research how to plant a vegetable garden. Compared with the ChatGPT users, Google users spent more time searching, reported greater effort, and wrote longer, more-detailed responses. Natural language analysis showed their advice also contained more unique phrasing and factual references.
To rule out the possibility that differences in the information itself—not just the way it was presented—was driving the results, a second experiment showed nearly 2,000 participants the same seven gardening tips, either as a single AI-style summary or split across six mock webpages, the way google search results are typically displayed. The Google users again engaged more deeply, retained more and wrote more thoughtful and original advice than the LLM users.
Two other experiments produced similar results.
A motivation problem
Daniel Oppenheimer, a professor of psychology and decision sciences at Carnegie Mellon University, says the research is resonant of what he sees in similar studies he does in his lab: Students who use AI tools to complete assignments tend to do better on homework—but worse on tests. “They’re getting the right answers, but they’re not learning,” he says.
Oppenheimer says the findings suggest that simply believing information came from an LLM makes people learn less. “It is like they think the system is smarter than them, so they stop trying,” he says. “That’s a motivational issue, not just a cognitive one.”
Oppenheimer cautions against rejecting AI altogether, however. He has seen GPT help students learn when they use it the right way—say, by critiquing a draft produced by an LLM or asking it probing questions. “AI doesn’t have to make us passive. But right now, that’s how people are using it,” he says.
Melumad, too, has concerns about future effects of AI, particularly in educational settings or professions that depend on critical thinking. But, like Oppenheimer, she sees the opportunities that AI offers as well.
“Younger people are increasingly turning to LLMs first,” Melumad says. “But if we don’t teach them how to synthesize and interpret information themselves, we risk deskilling the ability to learn deeply at all.”
4 comments:
Creative Digital Marketing is the driving force behind successful brand differentiation in a crowded online space. It combines innovative ideas, compelling visuals, storytelling, and strategic use of digital channels to captivate audiences and spark engagement
top-10-mobile-app-development-companies-in-perth-2025
From custom mobile platforms to enterprise-grade applications, Australia remains a hotspot for transformative app development. Collaborating with the best mobile app developers 2025 ensures a competitive edge, robust performance, and long-term scalability.
Medical App Developers Brisbane - Build secure healthcare apps with appointment scheduling, prescription management, and AI diagnostics. Our solutions improve patient care while ensuring data privacy. Start your project now! App Developers Brisbane
Supply Chain Visibility Platforms
End-to-end supply chain apps with real-time tracking, predictive analytics, and API integrations for carriers and shippers. Let’s discuss your needs!
A complete guide to the software development cost in 2025
Post a Comment