← Back to work

Machine learning · 2026

Customer Churn Prediction

An end-to-end churn system for telecom: calibrated XGBoost, SHAP explanations, ROI-optimized thresholds, and FastAPI + Streamlit serving with drift monitoring.

ROC-AUC on holdout
0.847
simulated campaign ROI
86.5%
PR-AUC under class imbalance
0.657

Architecture

Select a stage to explore it.

01 · Data & validation

Raw telecom data is downloaded by script and passed through schema and quality validation, with a generated validation report — the pipeline starts by proving its inputs.

GitHub Actions CIDockerdrift reference profiling

Visuals

Customer Churn Prediction GitHub repository with system architecture
GitHub repository — end-to-end churn prediction system with XGBoost and SHAP

01 · Problem

Acquiring a telecom customer costs several times more than retaining one, but blanket retention discounts burn budget on customers who were never leaving. The useful question isn't who might churn — it's which interventions are worth their cost.

02 · Solution

A complete ML system, not a notebook: validated data in, compared and tuned models, calibrated probabilities, a threshold optimized for net savings, SHAP explanations per customer, and a served API + dashboard with drift monitoring for life after deployment.

03 · Engineering decisions

PR-AUC is treated as the primary comparison metric because churn is imbalanced and ROC-AUC flatters weak minority-class performance. Calibration comes before thresholding — an uncalibrated probability can't be converted to dollars. And the final threshold maximizes simulated business savings including incentive cannibalization, which is the number a retention team actually optimizes.

04 · Results

Calibrated XGBoost at 0.847 ROC-AUC and 0.657 PR-AUC on holdout, with a simulated net saving of about $20,000 and 86.5% campaign ROI against a do-nothing baseline. The standalone source repository is private; the full write-up and structure are documented in the portfolio monorepo.

05 · Lessons learned

A model's job ends at a probability; a system's job ends at a decision. Calibration, thresholds, explanations, and drift monitoring are what turn the first into the second.