Table 2 Hyper-parameters for the developed models
Hyper-parameter name | Range | Selected value |
|---|---|---|
Scaling Method | min-max, standard, Yeo-Johnson, max-abs, normalize, robust, None | LR: max-abs; DT: robust; SVM: min-max; RF: Yeo-Johnson; XGB: min-max, MLP: max-abs, FIGS: None, HST: None |
Number of features | Uniform(5,17) | LR: 5; DT: 8; SVM: 15; RF: 16; XGB: 15, MLP: 16, FIGS: 10, HST: 9 |
Logistic Regression | ||
Penalty | l2, l1, elasticnet | elasticnet |
C | Uniform(0.5, 2) | 0.6662249852612048 |
Solver | SAGA | SAGA |
l1 Ratio | Uniform(0,1) | 0.31856895245132366 |
Decision Tree | ||
Criterion | gini, entropy | gini |
Splitter | best, random | best |
Max. Depth | Uniform(3,5) | 3 |
SVM | ||
Kernel | linear, rbf, sigmoid, poly | rbf |
C | Uniform(0,1) | 0.5096243767199001 |
Degree | Uniform(2,10) | NA |
Gamma | auto, scale | scale |
Random Forest | ||
Num. Estimators | Uniform(10,1000) | 787 |
Criterion | gini, entropy | gini |
Max. Depth | Uniform(1,100) | 4 |
Max. Features | sqrt, log2 | log2 |
XGBoost | ||
Eta | Uniform(0.01, 0.25) | 0.20057413447736855 |
Gamma | Uniform(0,100) | 1.3948395933415347 |
Subsample | Uniform(0.5, 1) | 0.75 |
Lambda | Uniform(0, 5) | 2.6673284087546447 |
Alpha | Uniform(0,5) | 1.6265515257949819 |
Num. Estimators | Uniform(10,1000) | 918 |
Max. Depth | Uniform(1,100) | 4 |
Scale Pos. Weight | Uniform(0,100) | 88.29 |
Multi-layer Perceptron | ||
Activation | relu, logistic, tanh | logistic |
Solver | adam, lbfgs, sgd | lbfgs |
Alpha | Uniform(0.0001, 0.1) | 0.05103420653681868 |
Learning rate | constant, adaptive, invscaling | NA |
Beta\(_1\) | Uniform(0,1) | NA |
Beta\(_2\) | Uniform(0,1) | NA |
Early stopping | True, False | True |
Hidden layer sizes | Uniform(18,100000) | 100 |