I am trying to build a binary classification model with XGBoost. I made sure to split my data into the training, validation and test sets. I performed feature selection, early stoppage and hyperparameter tuning with Binary Search.
I tested the model on randomly generated random states and it looks like there is less overfitting compared to before (as shown by the learning curves) but the averaged metrics are still suspicious:
Average Accuracy: 0.9965 Average Precision: 0.98039 Average Recall: 1.0 Average F1 Score: 0.98989
