This study from Wharton's Generative AI Labs examines whether assigning expert personas to AI models improves their performance on difficult objective multiple-choice questions. The researchers tested six models (GPT-4o, GPT-4o-mini, o3-mini, o4-mini, Gemini 2.0 Flash, Gemini 2.5 Flash) on two demanding benchmarks: GPQA Diamond (198 doctoral-level questions) and MMLU-Pro (300 professional-level questions).
The protocol compares three conditions: a baseline with no persona, expert personas (expert in physics, mathematics, economics, biology, chemistry, engineering, law, history), and "low-knowledge" personas (Layperson, Young Child, Toddler — "a 4-year-old who believes the moon is made of cheese"). Each model-prompt pair is evaluated over 25 independent responses per question (4,950 runs per pair on GPQA, 7,500 on MMLU-Pro), with 95% confidence intervals.
The results are essentially null: most persona conditions produce performance statistically indistinguishable from the baseline. On GPQA Diamond, no expert or low-knowledge persona reliably improves performance; the sole exception is a small gain from the "Young Child" prompt on Gemini 2.5 Flash (RD = 0.098). On MMLU-Pro, no expert persona delivers a statistically significant improvement for 5 of the 6 models, and nine significant negative differences are observed. Low-knowledge personas often degrade accuracy: the "Toddler" persona reduces performance in 4 of 6 models and proves significantly worse than "Layperson" in 5 of 6 models.
The notable exception is Gemini 2.0 Flash, which shows modest positive differences with all five expert personas on MMLU-Pro, particularly in engineering and chemistry. Additionally, aligning the expert persona with the question's domain provides no consistent benefit. The researchers identify failure modes: the Gemini Flash models sometimes refuse to answer when assigned an out-of-domain expert persona, and overly narrow role instructions lead the models to underuse their actual knowledge.
The practical implications are significant: the widespread practice of persona prompting is likely ineffective for improving factual accuracy. Organizations will derive more value from task-specific instructions, and should test multiple prompt variants for their concrete problems. Personas may nonetheless retain other uses, such as modulating tone or presentation style. The study's limitations (a limited number of models and personas, academic benchmarks) open avenues for future research.