Google’s health AI successfully passes difficult US medical exam

• Google’s AI-powered medical chatbot has passed a challenging US medical licensing exam.
• The passing score for the USMLE is typically around 60% for medical students and physicians in training.
• ChatGPT has achieved passing or near-passing scores in its performance, as reported in a study.

Google‘s AI-powered medical chatbot has passed a challenging US medical licensing exam, as stated in a recently published peer-reviewed study. However, the study also highlights that the chatbot’s responses still lag behind those of human physicians.

The release of ChatGPT, created by OpenAI and supported by Microsoft, sparked a wave of competition among tech giants in the rapidly advancing field of AI.

The healthcare sector has witnessed significant advancements in AI, despite ongoing discussions about its potential benefits and risks. For example, specific AI algorithms have shown the ability to interpret specific medical scans as accurately as human professionals.

In a preprint study published in December, Google introduced its AI solution called Med-PaLM, designed to address medical queries. Unlike ChatGPT, Med-PaLM is not currently accessible to the public.

I am very excited that the MedPaLM paper is now published in @Nature https://t.co/BDqshkSIDf

Advertisement

It is a great way to invite the broader scientific community to LLMs. I feel like LLMs are more adopted in the general public than in the scientific community. There are just so many…

— Hyung Won Chung (@hwchung27) July 12, 2023

According to the American tech conglomerate, Med-PaLM is the first large language model to pass the US Medical Licensing Examination (USMLE). Med-PaLM is an AI method that has been trained on vast amounts of human-generated text.

In the United States, the passing score for the USMLE, which is taken by medical students and physicians in training, is typically around 60%.

According to a study conducted in February, ChatGPT has achieved passing or near-passing scores in its performance.

In a peer-reviewed study published in the Nature Journal on Wednesday, Google researchers revealed that Med-PaLM achieved a score of 67.6% on multiple-choice questions designed to mimic the format of the USMLE.

Google has introduced a new evaluation benchmark to address and minimize “hallucinations,” which occur when AI models generate inaccurate data. This initiative aims to pinpoint and reduce such instances within technology.

Also Read

Can Google’s Bard surpass OpenAI’s ChatGPT in the EU, Brazil?

Google releases AI chatbot Bard in the EU and Brazil, entering an...

Karan Singhal, a Google researcher and the main author of the study, shared that the team utilized the new evaluation benchmark to assess a more recent version of their AI model. The results were described as “super exciting” by Singhal.

In a preprint study released in May that has yet to undergo peer review, Med-PaLM 2 achieved a score of 86.5% on the USMLE exam. This represents a significant improvement of nearly 20% compared to its predecessor.