Questions tagged [hyperparameter]
A parameter that is not strictly for the statistical model (or data generating process), but a parameter for the statistical method. It could be a parameter for: a family of prior distributions, smoothing, a penalty in regularization methods, or an optimization algorithm.
651 questions
2 votes
1 answer
34 views
Tune a Model on Calibrated or Uncalibrated Probabilties?
I report model performance using log loss on calibrated probabilities, where calibration is temperature scaling fitted on train-only out-of-fold (OOF) predictions. For hyperparameter tuning, should ...
0 votes
0 answers
22 views
How can I improve mAP when optimizing YOLOv8 hyperparameters with metaheuristic algorithms (e.g., GWO)?
I am working on hyperparameter optimization for YOLOv8 using a metaheuristic algorithm. Currently, I am testing the Grey Wolf Optimizer (GWO), but I plan to also experiment with other algorithms. Here ...
- 101
2 votes
1 answer
151 views
How many folds should a unnnested CV have compared to a nested CV
I read in the mlr3 book about nested resampling that: Nested resampling is a method to compare models and to estimate the generalization performance of a tuned model, however, this is the performance ...
- 43
0 votes
0 answers
82 views
Hyperparameter tuning for CatBoost
I'm fine-tuning hyperparameter of my CatBoost model for a multi-class problems. May I ask is this all the hyperparameter that I should tune, and if the range is the optimal range to start-tuning? <...
- 33
2 votes
1 answer
144 views
LightGBM Hyperparameter Tuning Using Hyperopt Fmin
I'm encountering a multi-classification problem. I'm trying to use Hyperopt Fmin to perform hyperparameter tuning. However, I have no idea what's the appropriate value that I should use for the search ...
- 33
0 votes
0 answers
26 views
Does model retrain frequency in time series CV have to match production retrain frequency?
Lets assume that we retrain the model every year in production and we have accumulating 50 years of data. If using a time series CV (e.g TimeSeriesSplit in sklearn) for hyperparams recalibration at ...
- 121
0 votes
0 answers
87 views
How to tune hyperparameters for low calibration error under small dataset
I'm studying which variant of variational autoencoders (VAE) gives better expected calibration error (ECE) (see also this doc) under small dataset. According to google's tuning playbook, to compare ...
- 307
1 vote
0 answers
70 views
Is my XGBoost Model Still Overfitting (Binary Classifcation)?
I am trying to build a binary classification model with XGBoost. I made sure to split my data into the training, validation and test sets. I performed feature selection, early stoppage and ...
- 11