ChatGPT's limits: AI fails to pass medical exam in Poland
ChatGPT never ceases to amaze us. Regularly improved, it offers increasingly better capabilities and is used in various AI-based applications. But is it as perfect as widely believed? It’s worth noting that despite its power, ChatGPT did not pass the medical exam.
22 November 2024 08:56
Artificial intelligence is undoubtedly a key element of our future. Already, many tasks are taken over by robots, which, on the one hand, is beneficial as it reduces the burden on workers, but on the other hand, it raises concerns about future jobs. Nevertheless, ChatGPT is still not capable of making medical diagnoses as it has not passed most medical exams.
ChatGPT couldn't handle the medical exam
People are increasingly wondering what the limits of artificial intelligence are. We know it can mimic voice, create images, and write scientific papers. Hence the question arises: could it also diagnose human ailments? Theoretically, yes, as it would suffice to input symptoms, and the technology could match them to diseases in a database. In practice, ChatGPT still cannot pass the medical exam to make this possible.
Researchers from the Collegium Medicum at Nicolaus Copernicus University conducted an experiment, using artificial intelligence to solve medical exams. ChatGPT performed the worst in internal medicine, scoring between 48% and 53%, which is well below the passing requirements. For comparison, the average student scores on the same exam ranged from 65% to 72%.
Researchers observed that ChatGPT handles simple questions better. Complex issues make the AI's responses less precise. Besides internal diseases, ChatGPT also took other exams, performing best in allergology (71%) and worst in cardiology (44%).
Can ChatGPT replace a doctor?
At the moment, the capabilities of artificial intelligence are too limited to compete with qualified doctors, but it can be expected to change over time. ChatGPT is constantly being developed, making it one of the fastest-growing tools.
Interestingly, ChatGPT is considered a more empathetic source of knowledge on medical forums. Patients claim its responses are more accurate than those received from human specialists on these platforms.
In conclusion, it is worth mentioning that ChatGPT had some limitations that may have affected exam results. Initially programmed in English, ChatGPT took the tests in Polish, which could have been the reason for the lower scores. American medical schools have often used ChatGPT and confirmed its effectiveness on English-language exams.
