@NicolasMartin Yeah sorry if not clear, im using their paper as a base then doing extra work on top (i.e. the clustering and retraining)

Yes class imbalance is one of they key aspects of this challenge, mostly I have just used external data to supplement the proportion of melanoma (like many in the challenge) up to 9% representation. My general thoughts are that the more class labels that are used as targets in training the stronger connections can be made for that specific class thereby lessening the error in other classes hence the head-scratching why I would be recommended this approach by a professor (who is yet to clarify why).

So I have used the results from the clustering (eg 3 clusters (targets 1,2,3)(targets 4,5,6,7)(melanoma)) and used these 'cluster labels' as the ground truth labels in the neural network training (with an imagenet pretrained model as base). One of the models is binary like you have suggested. Multiple people are performing this binary method but not clarifying why they are using that over the 9 target classes available.

Hi GooJ neither paper references clustering and unfortunately I dont have more time to perform additional training, thanks though.

updated my answer.

If you add a feature that has no impact on the target, linear regression training will give this feature a zero coefficient anyway so will ignore it in predictions. If a (false) linear relationship is detected between weather and height during training then it will add errors (although probably small errors) to your predictions.