Highlights
- AI got the right diagnosis 67 per cent of the time.
- Researchers say AI won't replace doctors, but will work alongside them.
- Experts warn AI still isn't ready to be used on its own in hospitals.
Study found that AI systems outperformed human doctors in high-pressure emergency triage, making more accurate diagnoses when patients were first rushed to hospital.
Independent experts called the results a “genuine step forward” in AI clinical reasoning.
The findings came from trials comparing AI performance against responses from hundreds of doctors.
The study looked at 76 patients at a Boston hospital. Doctors and an AI were both given the same patient information, including vital signs, basic details, and a nurse's note.
The AI got the right answer 67 per cent of the time. Doctors got it right 50–55 per cent of the time.
The AI used was OpenAI's o1 model. When given more detailed patient information, it was correct 82 per cent of the time, compared to 70–79 per cent for doctors.
The AI also did much better when making treatment plans, scoring 89 per cent compared to just 34 per cent for doctors.
One example stood out: A patient came in with a blood clot in the lungs. Doctors thought the medication wasn't working.
But the AI spotted something they missed. The patient had lupus, which was causing the problem. The AI turned out to be right.
Doctors are still needed
The researchers were clear: AI is not replacing doctors anytime soon.
"AI doesn't replace doctors," told Harvard's Arjun Manra to The Guardian. "But it is a big change that will reshape medicine."
His colleague Dr. Adam Rodman noted AI will become a kind of third partner in care, working with both doctors and patients.
There are also big limits to what the AI can do. It only read text and could not see how a patient looked or acted, which matters a lot in real care.
Experts also raised concerns about mistakes and who is responsible when AI gets it wrong. There is currently no clear rulebook for that.
One expert said AI is starting to look like a useful second opinion tool. But another warned that doctors might start relying on AI too much instead of thinking for themselves, and that the study doesn't prove AI is safe enough to use routinely in hospitals.
