An international team of researchers has shown for the first time that artificial intelligence is better at diagnosing melanoma than human doctors. This particular form of machine learning, known as a deep learning convolutional neural network (CNN), was able to make more correct diagnoses and fewer misdiagnoses than some of the world’s most capable skin care oncologists.
Man vs machine
The CNN starts off as a blank slate. In order to teach the artificial neural network how to identify skin cancer, the researchers fed it a dataset of over 100,000 images of malignant melanomas and benign moles. With each iteration, it learned patterns of features characteristic of malignant and benign tumors, becoming increasingly better at differentiating between the two.
After this initial training round, the team of researchers led by Professor Holger Haenssle, senior managing physician at the University of Heidelberg, Germany, introduced the AI to two new sets of images sourced from the Heidelberg library. These dermoscopic images of various skin lesions were completely new to the CNN. One set of 300 images was meant to solely test the performance of the CNN. Another set of 100 images was comprised of some of the most difficult to diagnose lesions and was used to test both machine and real dermatologists.
Researchers were able to recruit 58 doctors from 17 countries. Among them, 17 (29%) indicated they had less than two years’ experience in dermoscopy, 11 (19%) said they had two to five years of experience, and 30 (52%) were experts with more than five years’ experience.
The volunteers were asked to make a decision about how to manage the condition — whether it was surgery, follow-up, or no action at all — based on two levels of information. At level I, the only information that the dermatologists had at their disposal was from dermoscopic images. Four weeks after making the level I assessment, each participant was asked to review their diagnosis at level II, where they were given far more information about the patient — including age, sex, and the location of the lesion, as well as magnified images of the same case.
At level I, humans could accurately detect melanomas 86.6% of the time and correctly identified benign lesions with an average score of 71.3%. The CNN, however, was able to detect benign moles 95% of the time. At level II, the dermatologists significantly improved their performance, as expected, having diagnosed 88.9% of malignant melanomas and 75.7% that were benign.
Even though the expert doctors were better at spotting melanoma than their less experienced counterparts, they were, on average, outperformed by the AI.
Around 232,000 new cases of melanoma are diagnosed worldwide every year, which result in 55,500 deaths annually. The cancer can be cured, but it typically requires an early diagnosis. This is why this CNN is so impressive — it would be able to identify more cancers early on, thereby saving lives.
“These findings show that deep learning convolutional neural networks are capable of out-performing dermatologists, including extensively trained experts, in the task of detecting melanomas,” Haenssle said.
Of course, all of this doesn’t mean that doctors will soon be scrapped. Far from it: the researchers say that the machine will augment the performance of doctors rather than replace them. Think of a second ‘expert’ opinion which doctors can instantly turn to.
“This CNN may serve physicians involved in skin cancer screening as an aid in their decision whether to biopsy a lesion or not. Most dermatologists already use digital dermoscopy systems to image and store lesions for documentation and follow-up. The CNN can then easily and rapidly evaluate the stored image for an ‘expert opinion’ on the probability of melanoma. We are currently planning prospective studies to assess the real-life impact of the CNN for physicians and patients,” according to Haenssle.
Concerning the study’s limitations, it’s important to note that the study’s participants made diagnoses in an artificial setting. Their decision-making process might look different in a ‘life or death’ situation, which might impact performance. The CNN also had some limitations of its own, such as poor performance with images of melanomas on certain sites such as the fingers, toes, and scalp. For this reason, there is still no substitute for a thorough clinical examination performed by a trained human physician.
That being said, these impressive results indicate that we’re about to experience a paradigm shift, not only in dermatology but in just about every medical field, thanks to developments in artificial intelligence.
The findings appeared in the journal Annals of Oncology.
