why doctors should always question a.i.
When doctors start relying on AI tools to help diagnose cancer, their manual detection skills atrophy. The consequences can be dire.
In the classic 1958 short story The Feeling of Power by sci-fi legend Isaac Asimov, a man tasked with supervising interstellar artillery during a war in the far future figures out how to do basic math. To us, that sounds absurd. We all know how to add, divide, multiply, and subtract. But for humans in the far future, this is an astounding feat. For as long as they could remember, all mathematics were handled by computers.
No one had the wherewithal and determination to reverse engineer all that logic, and relied solely on their machinery to survive the ongoing war, which made the generals feel uneasy as any error was impossible to prevent, and only discovered after the fact. Humanity was out of the loop in a life or death situation, and this rediscovery allowed for a feeling of control and confidence in their future.
I thought of this story when reading about a new study which found that doctors who rely on AI to help them diagnose cancer, got worse at making accurate detections in a span of just a few months. They eventually regained those skills, but the fact that they would even lapse in the first place is disconcerting and signals that there needs to be a lot more education about the limitations of AI in a medical environment.
Now, it’s not that AI itself makes you dumber or worse at things. Despite what you’ve likely heard in countless outlets, there is no evidence of that. However, studies show that when given the ability to offload a task to a machine, our skills in performing that task will atrophy because we’re no longer practicing them.
This is not as big of a deal when it comes to navigation, or handling your contacts, but it’s a huge red flag when you’re handling potentially life changing diagnoses. AI isn’t a foolproof magical box. It’s a probabilistic system relying on statistical formulas, so it is going to make mistakes. Using it in medicine is supposed to help doctors make sure they don’t miss a certain piece of data or dismiss a smudge on a scan as an artifact if it might be a sign of cancer.
Doctors are not supposed to trust the AI to make the diagnosis. They’re supposed to use its output in addition to what they’re already doing, and take every alarm it raises with enough salt to make a nephrologist spit out their coffee with concern before they make the call about what’s next for their patients.
When I’m working with code generated by an AI, it’s an almost adversarial process in which I scrutinize every line and operator for any irregularity and write unit tests for all the generated methods. More often than not, I find that the code mishandles boolean functions and invokes database columns or API calls that don’t exist.
If that’s the approach we have to take to using AI in apps, just imagine how important it is to question the output of a model designed to find cancer. While medical AI isn’t a large language model and is far more precise, it will still inevitably give false positives and negatives because that’s just how math works.
Simply handing it to doctors without this disclaimer and hammering away at it until it’s a reflex not to trust any AI out of the box, use it only as a helper, and rely on their skills accumulated over more than a decade of education and practice first and foremost, is a recipe for unnecessary untimely deaths, and billions in lawsuits.