Microsoft AI beats doctors on hard cases

For patients with complex medical histories, the prospect of a diagnostic second opinion is often limited to waiting for a specialist. A new study published in Science this week indicates that artificial intelligence is beginning to close that gap. Microsoft's latest diagnostic system, known as MAI-DxO, correctly identified conditions in difficult cases more than twice as often as generalist physicians working individually. While this technology is still in the simulation phase, the results suggest that ordinary patients may soon benefit from more accurate and rapid preliminary diagnoses, potentially reducing the time spent in uncertain diagnostic loops.

The core claim behind this development is that coordinated AI agents can mimic the deliberative process of a medical panel. The system functions by having multiple agents question the patient, order tests, and debate potential diagnoses before settling on a conclusion. In tests using complex cases from the New England Journal of Medicine, the AI achieved an 83% accuracy rate. By comparison, primary care doctors and general internists, who had access to reference materials, were correct only 36% of the time. This disparity highlights a significant capability jump, although it is important to note that these difficult cases are typically handled by specialist teams in real-world settings, who were not included in the direct comparison.

However, the study reveals that the underlying power of the AI model matters more than the orchestration system itself. Microsoft tested the system with models from OpenAI, Google, Anthropic, Meta, and DeepSeek. The highest results came from OpenAI's GPT-5 and GPT-5.5. When using GPT-5 alone with a basic prompt, the accuracy was 78.9%. The DxO system raised this to approximately 83%, but at a slightly lower cost. The improvement was more pronounced with older or less capable models, suggesting that the coordination layer adds value primarily when the base model is not yet at its peak performance.

Cost and efficiency were also key metrics in the evaluation. Running the system with GPT-5 cost about $8,400 per complex case. The DxO system reduced this by roughly $3,000 while improving accuracy by only four points. In emergency department scenarios, the system showed little to no improvement in either accuracy or cost over the standalone model. This suggests that for routine or time-sensitive emergencies, the added complexity of the multi-agent system may not provide sufficient return on investment compared to a single, powerful model.

Mustafa Suleyman, CEO of Microsoft AI, described the findings as a step toward medical superintelligence. He is working with the Mayo Clinic to develop a foundation model for healthcare, with a release planned for next year. Suleyman believes that by 2027, the technology will enable a breakthrough in medical record processing. He envisions a system that can stream highly accurate predictions about how a medical record might unfold, providing clinicians with a level of foresight that is currently rare in practice.

The study also positions Microsoft in direct competition with Google and OpenAI, which have both recently claimed similar capabilities. To validate these results outside of simulations, Microsoft is partnering with Beth Israel Deaconess Medical Center in Boston. The goal is to evaluate the clinical reasoning of the system across different hospital settings, including urgent care environments. Patients will receive standard treatment, while those who consent will also interact with the AI. The recommendations will then be compared to those of human doctors to assess real-world utility.

There are caveats to consider. The study notes that accuracy gains shrink as models approach the theoretical maximum of performance. Furthermore, the current system is designed to support clinicians rather than replace them. Microsoft is moving beyond simple benchmarking tasks, such as answering exam questions, toward integrating with actual healthcare systems. This broader ambition reflects a strategic shift from building isolated tools to creating foundational infrastructure for clinical decision support.

Suleyman, who previously co-founded DeepMind, has long focused on applying AI to medical challenges. He expects healthcare models to improve significantly in capability within the next two to three years. He compares this trajectory to the rapid progress seen in coding and chat applications, where real-world usage has driven iterative improvements. For Microsoft, which already sells dictation and radiology tools, this new diagnostic capability represents an expansion of its portfolio. The company aims to lead the market in clinical decision support, believing that the integration of advanced AI into daily medical practice is the next major frontier.

The release of this study coincides with a manifesto from Microsoft outlining the requirements for medical superintelligence. The company defines this as the capacity to surpass contemporary human-led health systems in medical reasoning and coordination. The focus is on integrating AI across the full arc of care, rather than just addressing isolated tasks. As the technology moves toward real-world deployment in Boston hospitals, the coming months will provide critical evidence on whether these simulation successes translate into tangible benefits for patients and providers alike.