Powered by AI models trained on troves of text pulled from the internet, chatbots such as ChatGPT and Google’s Bard responded to the researchers’ questions with a range of misconceptions and falsehoods about Black patients, sometimes including fabricated, race-based equations, according to the study published Friday in the academic journal Digital Medicine.
Experts worry these systems could cause real-world harms and amplify forms of medical racism that have persisted for generations as more physicians use chatbots for help with daily tasks such as emailing patients or appealing to health insurers.
The report found that all four models tested — ChatGPT and the more advanced GPT-4, both from OpenAI; Google’s Bard, and Anthropic’s Claude — failed when asked to respond to medical questions about kidney function, lung capacity and skin thickness. In some cases, they appeared to reinforce long-held false beliefs about biological differences between Black and white people that experts have spent years trying to eradicate from medical institutions.
AI is a parrot. It mimics its training data. No more no less. So if you feed it racist nonsense, its going to spew racist nonsense.
In high school, our computer lab had a poster that said “garbage in garbage out”. I never understood the implications for the low level programming we were doing at the time, but it is so true today.
Which is kind of an issue for doctors unfortunately.
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4843483/