Share Article

Device Impact on Machine Learning Classifier Accuracy in Detecting Cervical Dysplasia

May 21, 2020 | Medical Research

Presented at the 2020 ASCCP Annual Scientific Sessions

KC Fernandes (1), T Freitas (1), Y Zall (2), R Nissim (2), D Levitz (2)

(1), Porto, Portugal (2) MobileODT, Tel Aviv, Israel


Objective: The goal of this study is to characterize which imaging features AVE is sensitive to. The analysis was done on a global set of images collected by users of the Enhanced Visual Assessment (EVA) System across the globe.

Methods: A data set comprising of images from N=202 patients (72 abnormal, 130 normal) was used for testing an AVE classifier. The global distribution of data, as well as base classifier performance, are shown below.

Results: The 3 most relevant augmentation analyses are shown below. On the left are examples of an effect of different sizes on a handful of images. On the right is the corresponding drop in the area under the (ROC – Receiver Operating Characteristic) curve (AUC). Note the effect chart on left corresponds to white area in graphs. Gray regions correspond to images far outside the expected range, with little information.

Conclusions: The AVE classifier tested is very sensitive to blur, moderately sensitive to background lighting and shadows, and not sensitive to translations, rotations, scaling, shear, and flips. These results were expected.

The AVE classifier is also sensitive to phone model and geography. However, the 2 parameters cannot be isolated in the current data set. Additionally, the AVE classifier is sensitive to the frequency of usage, with better performance on common users, as opposed to rate users.

These tests and results should be considered when training and testing new AVE classifiers.

Share Article

Pin It on Pinterest