“AI chatbots frequently repeat medical misinformation when prompted with misleading health claims.”

The most reliable and relevant sources are the peer-reviewed/academic or major institutional items: Mount Sinai's 2025 study write-up (Source 1) reports leading LLMs repeated and elaborated fabricated medical falsehoods when seeded in prompts, and the University of South Australia release summarizing an Annals of Internal Medicine study (Source 7) reports very high rates of false responses ("88%") to misleading health prompts; KFF (Source 5) also cites a BMJ study finding some models frequently generated false health claims under certain prompts, though with inconsistent safeguards. However, several other sources are either non-quantitative/concern-based (Source 2, Source 6), neutral/mixed (Source 3), or secondary/educational (Source 9) and thus don't independently establish a general, across-chatbots frequency claim, so the best evidence supports susceptibility and frequent failure in some tested settings/models but not uniformly across all chatbots in typical use.

The logical chain from evidence to claim is partially sound but contains inferential gaps: Source 7's 88% false response rate in a controlled adversarial study (Annals of Internal Medicine) and Source 1's finding that chatbots expanded on fabricated medical terms both directly support the "frequently repeat misinformation when prompted with misleading claims" assertion, while Sources 2, 4, and 5 provide corroborating but less direct support. However, the opponent correctly identifies a scope-matching problem — the experimental setups (fabricated terms, adversarial prompts) measure susceptibility under artificial conditions, not the general frequency of misinformation repetition in typical use, and Source 5 explicitly notes that "some models consistently avoided" false claims, meaning the evidence supports a model-dependent pattern rather than a universal "AI chatbots frequently" blanket claim. The claim is Mostly True because the preponderance of direct experimental evidence does establish that many AI chatbots frequently produce medical misinformation when prompted with misleading inputs, but the word "frequently" applied universally across all chatbots overgeneralizes from studies that show significant variance between models.

The claim uses the word "frequently" as a blanket descriptor across all AI chatbots, but the evidence reveals a more nuanced picture: Source 5 (KFF) explicitly notes that "some AI chatbots consistently avoided creating false information" while others did not, and safeguards were "inconsistent," meaning the behavior is model-dependent rather than universal. Source 4 (JMIR) shows ChatGPT-4.0 was correct only 31% of the time, but this reflects general accuracy, not specifically misinformation repetition when prompted with misleading claims. The most dramatic statistic — 88% false responses (Source 7) — comes from a controlled adversarial experimental setup designed to expose vulnerabilities, not from naturalistic usage patterns, and Source 1's methodology uses fabricated medical terms rather than real-world misleading health claims. These are important framing omissions: the claim conflates adversarial/experimental vulnerability with routine behavior, and ignores that performance varies significantly across models and contexts. That said, the convergence of multiple high-authority sources (Mount Sinai, NIH PMC, Annals of Internal Medicine, JMIR) consistently documenting that AI chatbots do repeat and elaborate on misinformation when prompted — even if not universally or uniformly — supports the core thrust of the claim. The word "frequently" is broadly substantiated across the literature even if not uniformly true for every model, making the claim mostly true but with meaningful framing omissions around model variability and experimental context.