Microsoft’s AI Surpasses Doctors in Diagnosing Complex Medical Cases, Study Reveals
Microsoft has recently made a significant contribution to the medical field with its MAI-DxO (Microsoft AI Diagnostic Orchestrator). According to a peer-reviewed study published in the esteemed journal Science, MAI-DxO accurately diagnosed 83.2% of complex medical cases. These cases were sourced from the New England Journal of Medicine, and the AI’s performance was compared to generalist physicians who had access to reference materials and the internet, yet only achieved a success rate of 36.1%.
The system, which was elaborated on in a paper titled “Sequential Diagnosis with Language Models”, functions like a panel of virtual doctors. Multiple AI agents interact with a simulated patient, order diagnostic tests, debate various hypotheses, and eventually agree on a diagnosis. All the while, they manage the estimated costs of the tests.
In a separate evaluation involving 325 emergency department cases, MAI-DxO demonstrated a success rate of 80.3%, significantly outperforming the physicians’ score of 46.8%. The team behind this groundbreaking technology includes all 15 authors who are Microsoft employees, one of whom is Mustafa Suleyman, the CEO of Microsoft AI. Suleyman has described these results as a stride towards what he terms “medical superintelligence.”
“Our MAI Diagnostic system is now 1.7x more accurate than physicians in the ED.” – Mustafa Suleyman, CEO of Microsoft AI
However, experts have pointed out important caveats to consider. The study utilized historical, text-based case records rather than real patients and compared MAI-DxO’s performance against solo generalist physicians instead of specialist care teams. Despite these considerations, Microsoft has announced plans for a real-world clinical trial at the Beth Israel Deaconess Medical Center.
The publication of this study in Science signifies a major milestone for the application of AI in medicine. Source: Technology.org — Microsoft MAI-DxO Diagnoses 83% of Hard Cases