I'm a software engineer working with medical device AI models that predict diseases and other conditions. For the most part, I don't design the models but I help with getting FDA clearance for them. These models are generally produced with supervised learning, and most of them are boosted decision trees, with a few CNNs or SVMs here and there.
Getting the initial training data for these models is often challenging, as the patient cases have to be reviewed by a physician and need to be labeled with a "gold standard" diagnosis (usually a binary label indicating whether the disease/condition is present or absent).
Once these models are deployed they are often directly integrated into Electronic Health Record (EHR) pipelines, and they can ingest lots of new patient information, but without the "gold standard" labeling (unless a patient happened to be evaluated for the specific disease or condition).
I have noticed that some models will generate a label for new patient data (according to the model's prediction) and then use the freshly labeled data to train the next iteration of the model. To me, this does not make sense. If the prediction is correct, the model made the right prediction already without the need for further training. If it is incorrect, false positives of the model are unduly reinforced.
Is there a legitimate reason for this approach or will it generally make models worse over time?
