Microsoft’s AI Diagnostic System Beat Generalist Doctors 83% to 36% on Medicine’s Hardest Cases — Now It Faces Real Patients

Microsoft’s AI diagnostic system, MAI-DxO, correctly diagnosed 83.2% of complex medical cases in a peer-reviewed study published Thursday in Science — more than double the 36.1% scored by generalist physicians working alone with reference materials. Why it matters: it is one of the strongest peer-reviewed results yet for AI diagnosis, and Microsoft is now moving the system toward real hospital evaluation.

October 22, 2026

What happened?

The paper, “Sequential Diagnosis with Language Models,” takes Microsoft’s study through peer review after an earlier non-peer-reviewed preprint posted last year. MAI-DxO — short for Microsoft AI Diagnostic Orchestrator — is not handed a full chart. Each case starts with a short chief complaint, and the system must decide which questions to ask and which tests to order, step by step, before committing to a diagnosis — much closer to a real clinical encounter than a multiple-choice exam.

How does MAI-DxO work?

DxO runs several AI agents as a virtual panel of physicians. Agents propose diagnoses, select tests, challenge the panel’s emerging view, and weigh what each test might reveal against what it costs. Microsoft’s benchmark, SDBench, converts 304 complex case reports from The New England Journal of Medicine into stepwise encounters where every test carries a virtual cost. Microsoft tested the orchestrator with models from OpenAI, Google, Anthropic, Meta, and DeepSeek; the strongest results came with OpenAI’s GPT-5 and GPT-5.5, where DxO lifted GPT-5.5 from 72% to nearly 87% on complex cases.

What were the results — and the caveats?

On the NEJM cases, DxO reached 83.2%, against 36.1% for primary care doctors and general internists using references and about 22% without them. On emergency department cases from Beth Israel Deaconess Medical Center in Boston, it scored 80.3% to the physicians’ 46.8%. The caveats matter: the doctors were generalists working alone, while cases this hard normally go to specialists or teams, who were not tested — the researchers expect the gap to narrow against experts. Every case was retrospective, not a live patient. And the study’s own data shows most of the lift comes from the underlying frontier models, with Microsoft’s orchestration adding gains on top. Cost claims are also simulated: with GPT-5, DxO cut an estimated $8,400 per-case cost by roughly $3,000 while adding about four accuracy points.

What happens next?

Microsoft says DxO will be evaluated alongside standard care at Beth Israel Deaconess, with testing expected to begin within the next couple of months: patients get normal treatment, and consenting patients also engage with DxO in parallel for comparison. Microsoft is also working with the Mayo Clinic on a healthcare foundation model planned for next year, and intends to release SDBench publicly. Microsoft AI CEO Mustafa Suleyman frames the work as a step toward “medical superintelligence.”

Why it matters

The honest takeaway is narrower than the headline and still significant: on medicine’s hardest published cases, a panel of AI agents working sequentially now clearly outperforms a lone generalist — in a peer-reviewed journal, not a press release. What it has not yet done is diagnose a live patient in this study. The Beth Israel evaluation will show whether benchmark diagnosis survives contact with a real emergency department — exactly the kind of claim worth tracking, because everything AI is only worth reporting once it is tested.

Frequently asked questions

Did Microsoft’s AI really beat doctors 83% to 36%?

On this benchmark, yes — 83.2% versus 36.1% for generalists with references, on 304 complex NEJM cases. The physicians were generalists working alone, not the specialists who normally handle such cases.

Is MAI-DxO being used on real patients?

Not in this study. A parallel evaluation alongside standard care at Beth Israel Deaconess is expected to begin within the next couple of months.

Which models powered the best results?

OpenAI’s GPT-5 and GPT-5.5. Microsoft also ran models from Google, Anthropic, Meta, and DeepSeek through the same orchestrator.

Sources: TechCrunch, Gizmodo, Science