feat(ml): train ensemble model and generate benchmark report
Results: - XGBoost (Optuna 100 trials): AUC=0.7856, Precision@3=0.5783 - LightGBM (Optuna 100 trials): AUC=0.7833, Precision@3=0.5736 - MLP (3 layers 256-128-64): AUC=0.7743, Precision@3=0.5643 - Ensemble (weighted voting): AUC=0.7840, Precision@3=0.5814 Baseline XGBoost: Precision@3=0.5287 Delta: +0.0527 (+5.3%) — DEPLOY threshold met (+5%) Latency: 35ms/race, 69ms/full-day (well under 200ms limit) SHAP: 31/43 features selected, top features: rang_cote, implied_prob, cote_direct, ratio_cote_field All 12 regression/latency tests passing. Co-Authored-By: Paperclip <noreply@paperclip.ing>
This commit is contained in:
68
models/benchmark_report.md
Normal file
68
models/benchmark_report.md
Normal file
@@ -0,0 +1,68 @@
|
||||
# Benchmark ML Ensemble — Turf Prédictions
|
||||
|
||||
**Date:** 2026-04-25
|
||||
**Dataset:** 10,899 partants
|
||||
**Holdout:** 2,180 lignes (2026-04-19 → 2026-04-24)
|
||||
|
||||
## Résultats
|
||||
|
||||
| Modèle | Precision@3 | AUC | Latence/prédiction |
|
||||
|--------|-------------|-----|-------------------|
|
||||
| XGBoost (baseline) | 0.5287 | 0.7254 | — |
|
||||
| xgboost | 0.5783 | 0.7856 | 0.01 ms |
|
||||
| lightgbm | 0.5736 | 0.7833 | 0.00 ms |
|
||||
| mlp | 0.5643 | 0.7743 | 0.01 ms |
|
||||
| **Ensemble** | **0.5814** | **0.7840** | **0.02 ms** |
|
||||
|
||||
## Décision de déploiement
|
||||
|
||||
- Delta Precision@3 : **+0.0527** (+5.3%)
|
||||
- Seuil requis : **+5%**
|
||||
- Résultat : **✅ DEPLOIEMENT RECOMMANDE**
|
||||
|
||||
## Optimisation Optuna
|
||||
|
||||
- Trials XGBoost : 100
|
||||
- Trials LightGBM : 100
|
||||
- Pruning : MedianPruner
|
||||
|
||||
### Meilleurs hyperparamètres XGBoost
|
||||
```json
|
||||
{
|
||||
"n_estimators": 141,
|
||||
"max_depth": 5,
|
||||
"learning_rate": 0.016298172447266404,
|
||||
"subsample": 0.7660470794373848,
|
||||
"colsample_bytree": 0.471124415020467,
|
||||
"min_child_weight": 14,
|
||||
"reg_alpha": 1.9364166463791586,
|
||||
"reg_lambda": 6.018030083488602,
|
||||
"gamma": 4.614943551368141
|
||||
}
|
||||
```
|
||||
|
||||
### Meilleurs hyperparamètres LightGBM
|
||||
```json
|
||||
{
|
||||
"n_estimators": 186,
|
||||
"max_depth": 4,
|
||||
"learning_rate": 0.012915117465216954,
|
||||
"num_leaves": 141,
|
||||
"subsample": 0.6193119116922561,
|
||||
"colsample_bytree": 0.539310022549326,
|
||||
"min_child_samples": 9,
|
||||
"reg_alpha": 0.6864583098112754,
|
||||
"reg_lambda": 0.0549259590914184
|
||||
}
|
||||
```
|
||||
|
||||
## Features
|
||||
|
||||
- Total features : 43
|
||||
- Retenues par SHAP : 31
|
||||
|
||||
## Poids de l'ensemble
|
||||
|
||||
- xgboost : 0.2316
|
||||
- lightgbm : 0.2342
|
||||
- mlp : 0.2129
|
||||
Reference in New Issue
Block a user