In 2025, researchers Hamsa Bastani and colleagues ran a field experiment with nearly a thousand high school math students. Some had access to off the shelf GPT-4 as a tutor while practicing. Others used a custom GPT tutor designed with pedagogical guardrails. On subsequent exams, where students no longer had AI access, the GPT-4 group showed a performance decline. The custom tutor group did not.

While the study focused on math instruction, it highlights a broader lesson that may apply to dialogue education as well: pedagogical design often matters more than the underlying model. AI is rapidly moving into dialogue education, where students are already using chatbots to rehearse arguments, work through disagreement, and practice difficult conversations. Our recent white paper, Can AI Teach Dialogue Skills to College Students?, reaches a conclusion that echoes Bastani's: the AI itself isn't the variable that matters most. The deliberate pedagogical design behind it is.

Why generic AI misses the mark in dialogue education

Generic AI tools are built to be helpful, agreeable, and responsive. Those are useful traits in most contexts, and they're exactly the wrong traits for teaching dialogue. Constructive dialogue education depends on productive friction, moments of discomfort, and the deliberate withholding of answers. A chatbot tuned to please the user is a chatbot tuned to undermine the conditions under which dialogue skills develop.

The brief identifies three risks that can emerge when generic AI is used for dialogue education.

The first is over-scaffolding. When AI is too quick to do the thinking for students, it becomes a crutch that substitutes for the skill it aims to build. A student who never struggles to generate a charitable reading of an opposing view never learns to do it on their own.

The second is the illusion of understanding. Working with AI can leave students feeling more confident without actually internalizing skills. A coaching session that ends with positive feedback can produce a student who is more sure of their abilities but no better at exercising them. The risk shows up in real conversations, where overconfident students may enter less humble and less prepared than before they practiced.

The third is persuasion in place of understanding. AI now outperforms human persuaders in controlled studies, and conversational AI advocating for political candidates has shifted voter preferences on the basis of inaccurate claims. Constructive dialogue isn't about persuading others. It's about mutual understanding. A tool that persuades works against the goals it's meant to serve.

The exact qualities that make AI such an agreeable assistant make it a bad dialogue teacher. Generic AI may be useful for exploration and reflection, but dialogue education requires more intentional design.

What designed for dialogue actually means

Tools designed for dialogue share a set of features that distinguish them from their off the shelf counterparts.

They preserve friction. A well-designed coach withholds answers, prompts reflection, and asks students to articulate their thinking before offering its own. That friction is the mechanism through which skills get built.

They pair practice with feedback. Practice alone doesn't build skill. Research on AI negotiation training found that participants improved only when they received targeted feedback between rounds, mirroring decades of research showing that experience without instruction produces limited skill gains. Without an accompanying coach, practice against a realistic AI counterpart may not build transferable skills.

They're built with pedagogical guardrails. In dialogue education, guardrails take concrete form. Feedback rubrics specify what skilled listening and questioning look like, so the AI evaluates against learning criteria rather than its own conversational instincts. System prompts shape what the AI will and won't do, what it will and won't praise, and where it directs a student's attention. Practice scenarios are designed by educators rather than left to the model to invent. Each of these choices requires people with pedagogical expertise in the design loop, not just engineers. The underlying model matters less than what gets layered on top of it.

They account for cultural differences. AI coaches trained too narrowly can enforce stylistic norms rather than develop genuine skill, rewarding one cultural style of expression over another. Even models that appear unbiased on standard tests can carry stereotypical associations that shape their outputs in subtle but consequential ways. Accounting for cultural differences in design requires diverse review of feedback rubrics and ongoing audits of model behavior.

They constrain the AI's role. One of the white paper’s key findings is that the more constrained and pedagogically structured the AI's role, the lower the risk and the stronger the current evidence for its use. AI coaching, the most constrained role, has the strongest support. AI conversation partners, the least constrained role, have not yet shown clear skill-building benefits, while carrying risks that outweigh those benefits.

Why this matters now

Campuses are adopting AI tools at speed, often without the evaluation frameworks needed to distinguish a designed tool from a generic one. The pace isn't slowing, and the stakes are rising with it.

The persuasion risk makes the moment especially urgent.

The risk of a dialogue tool that persuades rather than teaches doesn't just fail to build skill. It also works directly against the goals of constructive dialogue itself.

A growing share of the language shaping public discourse is produced not by citizens but by machines, and the decisions universities make about which AI tools their students engage with will shape how the next generation of citizens enters that discourse.

The choice ahead

AI is already playing a role in dialogue education. The open question is whether that role gets shaped by deliberate design or left to default behaviors the evidence tells us are counterproductive. Off-the-shelf AI may be a starting point, but the evidence suggests that dialogue education requires additional pedagogical design to achieve its goals.

The full white paper examines the three roles AI can play in constructive dialogue, the evidence behind each, and what campus leaders should look for when evaluating tools for adoption.