Model: CatBoost
Accuracy: 0.9930833333333333
ROC AUC: 0.8528828485636176
Precision: 0.8771929824561403
Recall: 0.7075471698113207
F1 Score: 0.783289817232376
Confusion Matrix:
 [[11767    21]
 [   62   150]]
Classification Report:
               precision    recall  f1-score   support

           0   0.994759  0.998219  0.996486     11788
           1   0.877193  0.707547  0.783290       212

    accuracy                       0.993083     12000
   macro avg   0.935976  0.852883  0.889888     12000
weighted avg   0.992682  0.993083  0.992719     12000


Model: LightGBM
Accuracy: 0.9929166666666667
ROC AUC: 0.8666944638295421
Precision: 0.8432432432432433
Recall: 0.7358490566037735
F1 Score: 0.7858942065491185
Confusion Matrix:
 [[11759    29]
 [   56   156]]
Classification Report:
               precision    recall  f1-score   support

           0   0.995260  0.997540  0.996399     11788
           1   0.843243  0.735849  0.785894       212

    accuracy                       0.992917     12000
   macro avg   0.919252  0.866694  0.891146     12000
weighted avg   0.992575  0.992917  0.992680     12000


Model: XGBoost
Accuracy: 0.9936666666666667
ROC AUC: 0.8601279843268819
Precision: 0.9
Recall: 0.7216981132075472
F1 Score: 0.8010471204188482
Confusion Matrix:
 [[11771    17]
 [   59   153]]
Classification Report:
               precision    recall  f1-score   support

           0   0.995013  0.998558  0.996782     11788
           1   0.900000  0.721698  0.801047       212

    accuracy                       0.993667     12000
   macro avg   0.947506  0.860128  0.898915     12000
weighted avg   0.993334  0.993667  0.993324     12000


Model: RandomForest
Accuracy: 0.9928333333333333
ROC AUC: 0.8365430786665045
Precision: 0.89375
Recall: 0.6745283018867925
F1 Score: 0.7688172043010753
Confusion Matrix:
 [[11771    17]
 [   69   143]]
Classification Report:
               precision    recall  f1-score   support

           0   0.994172  0.998558  0.996360     11788
           1   0.893750  0.674528  0.768817       212

    accuracy                       0.992833     12000
   macro avg   0.943961  0.836543  0.882589     12000
weighted avg   0.992398  0.992833  0.992340     12000


Model: AdaBoost
Accuracy: 0.9885833333333334
ROC AUC: 0.8019548181393296
Precision: 0.7049180327868853
Recall: 0.6084905660377359
F1 Score: 0.6531645569620254
Confusion Matrix:
 [[11734    54]
 [   83   129]]
Classification Report:
               precision    recall  f1-score   support

           0   0.992976  0.995419  0.994196     11788
           1   0.704918  0.608491  0.653165       212

    accuracy                       0.988583     12000
   macro avg   0.848947  0.801955  0.823680     12000
weighted avg   0.987887  0.988583  0.988171     12000


Ensemble Model Metrics:
Accuracy: 0.99375
ROC AUC: 0.8648025494426695
Precision: 0.8959537572254336
Recall: 0.7311320754716981
F1 Score: 0.8051948051948052
Confusion Matrix:
[[11770    18]
 [   57   155]]
Classification Report:
              precision    recall  f1-score   support

           0   0.995181  0.998473  0.996824     11788
           1   0.895954  0.731132  0.805195       212

    accuracy                       0.993750     12000
   macro avg   0.945567  0.864803  0.901009     12000
weighted avg   0.993428  0.993750  0.993439     12000


Top Model based on Accuracy:
Name: Ensemble
Accuracy: 0.99375
ROC AUC: 0.8648025494426695
Precision: 0.8959537572254336
Recall: 0.7311320754716981
F1 Score: 0.8051948051948052
Confusion Matrix:
 [[11770    18]
 [   57   155]]
Classification Report:
               precision    recall  f1-score   support

           0   0.995181  0.998473  0.996824     11788
           1   0.895954  0.731132  0.805195       212

    accuracy                       0.993750     12000
   macro avg   0.945567  0.864803  0.901009     12000
weighted avg   0.993428  0.993750  0.993439     12000

Top Model saved as 'aps_failure_training_imputed.pkl'