Experts warn against using smartphone apps to identify skin lesions that might be cancer.
When it comes to smartphone technologies, today there’s an app for just about everything: You can play chess, learn to speak Mandarin, or map the best driving route to the airport. Health is a huge category for apps, but many of these programs don’t have the science to back their claims.
The apps that scan moles and other lesions — and claim to identify which are cancerous — fall into an area that needs much more research and development, say experts. In a paper published in the April 2022 issue of the British Journal of Dermatology (BJD), a team overseen by Memorial Sloan Kettering Cancer Center (MSK) dermatologists Veronica Rotemberg and Allan Halpern reported that the overall accuracy of these commercial apps is low. Of the apps that were evaluated, they were on average only 59% accurate.
“We downloaded every app we could find and tested them all to see how well they could identify melanoma,” Dr. Rotemberg says. “We found that, on average, they don’t do very well. Skin cancer is one area where we really need more data before encouraging patients to rely on an app.”
Obstacles to Using AI for Cancer Screening
Drs. Rotemberg and Halpern have been focused for years on developing artificial intelligence (AI) and other computer-based tools for detecting and diagnosing melanoma, the deadliest form of skin cancer. Metastatic melanoma is expected to kill more than 7,500 people in the United States this year. When it’s caught early, however, melanoma is almost always curable, which is why researchers are focused on improving early detection.
But there are many hurdles. Even the digital tools that have been created by top dermatologists and computer scientists have many flaws. Another recent paper, published in the May 2022 issue of The Lancet Digital Health, outlined why these diagnostic algorithms are so difficult to develop.
Physician-scientist Veronica Rotemberg is developing AI technologies for identifying skin cancer, including melanoma.
The researchers noted that although previous studies of AI in dermatology have shown that computers do better than dermatologists when diagnosing skin lesions, those studies did not adequately assess real-world clinical scenarios, using a broad range of images. When the computers were presented with types of images and skin conditions they had never “seen” before, they were less accurate than with reading more familiar images. Algorithms were on average only 6% accurate for a diagnosis they had not seen in training.
“One of the most important findings was that we really need to improve our algorithmic ability to identify a type of image that the system has not been trained on,” says Dr. Rotemberg, who was a primary author. “When it comes to unknown categories of images, these tools perform about equivalent to random chance.”
Dr. Rotemberg says AI can also get tripped up by flawed images. For example, stray strands of hair and pen marks on the photos reduced the AI tool’s accuracy.
Improving AI To Detect Skin Cancer
The Lancet Digital Health paper was the result of the 2019 Grand Challenge focused on developing algorithms to diagnose skin cancer. In the growing field of AI, Grand Challenge events gather teams of computer scientists from around the world to develop better machine-learning systems for medical imaging and other areas.
“This is an exciting time, and the research community is so interested in improving the detection of skin cancer,” Dr. Rotemberg says. “The building blocks are in place, but we still have many obstacles to overcome.” She adds that this research represents international collaborations with experts in dermatology and AI from around the world: “This kind of science happens in large teams.”
Another important problem to address is the fact these may not work nearly as well on people of color, the researchers acknowledge.
“Skin cancer is one area where we really need more data before encouraging patients to rely on an app.”
The BJD study illustrates the dangers of relying on AI-based tools that are not yet ready for prime time. “We were concerned that these apps would label everything as melanoma, leading to a lot of worry and unnecessary biopsies,” Dr. Rotemberg notes. “But actually, the opposite is true: They may be creating a false sense of security by failing to identify melanoma as the real thing.”
Experts agree: In terms of identifying skin cancer, it’s too soon to say, “There’s an app for that.”
