Theory & Background

1. Introduction to Churn Prediction

Customer churn prediction is a critical business problem where the goal is to identify customers likely to discontinue their service or subscription. This enables proactive retention strategies that are significantly more cost-effective than acquiring new customers.

Key Business Impact

• Acquiring new customers costs 5-25x more than retaining existing ones
• A 5% increase in retention can increase profits by 25-95%
• Churned customers rarely return (only 10-15% win-back rate)

The prediction problem is formulated as binary classification: given customer features X, predict whether the customer will churn (Y=1) or stay (Y=0) within a defined time window.

2. XGBoost Algorithm

XGBoost (eXtreme Gradient Boosting) is an optimized gradient boosting framework that has become the dominant algorithm for structured/tabular data problems.

2.1 Gradient Boosting Fundamentals

Gradient boosting builds an ensemble of weak learners (decision trees) sequentially, where each new tree corrects errors made by the previous ensemble:

F_m(x) = F_{m-1}(x) + η · h_m(x)

Where:

• F_m(x) = prediction at iteration m

• η = learning rate

• h_m(x) = new tree fitted to pseudo-residuals

2.2 XGBoost Objective Function

XGBoost optimizes a regularized objective:

L(θ) = Σ l(y_i, ŷ_i) + Σ Ω(f_k)

Where Ω(f) = γT + ½λ||w||²

• T = number of leaves

• w = leaf weights

• γ, λ = regularization parameters

2.3 Key Hyperparameters

Parameter	Description	Typical Range
n_estimators	Number of trees	100-1000
max_depth	Maximum tree depth	3-10
learning_rate	Shrinkage factor	0.01-0.3
subsample	Row sampling ratio	0.5-1.0
scale_pos_weight	Class imbalance handling	sum(neg)/sum(pos)

3. SHAP Explainability

SHAP (SHapley Additive exPlanations) provides a unified framework for interpreting predictions based on game-theoretic Shapley values.

3.1 Shapley Values

The Shapley value for feature i is the average marginal contribution across all possible feature coalitions:

φ_i = Σ_{S⊆N\{i}} |S|!(|N|-|S|-1)!/|N|! [f(S∪{i}) - f(S)]

3.2 SHAP Properties

Local Accuracy: Sum of SHAP values equals model output
Missingness: Missing features have zero contribution
Consistency: If a feature's contribution increases, its SHAP value increases

3.3 TreeSHAP

For tree-based models, TreeSHAP computes exact Shapley values in O(TLD²) time, making it practical for large-scale applications.

4. Handling Class Imbalance

Churn datasets are typically imbalanced (10-30% churn rate). Several techniques address this:

Cost-Sensitive Learning

Assign higher misclassification cost to minority class using scale_pos_weight parameter.

SMOTE

Synthetic Minority Over-sampling Technique creates synthetic examples by interpolating between existing minority samples.

Undersampling

Remove majority class examples (may lose information).

Threshold Tuning

Adjust classification threshold based on precision-recall trade-off.

5. Evaluation Metrics

Metric	Formula	Best For
AUC-ROC	Area under ROC curve	Overall ranking quality
Precision	TP / (TP + FP)	When FP cost is high
Recall	TP / (TP + FN)	When catching all churners matters
F1 Score	2 × (P × R) / (P + R)	Balance precision/recall
PR-AUC	Area under PR curve	Imbalanced datasets

6. References

• Chen, T., & Guestrin, C. (2016). XGBoost: A Scalable Tree Boosting System. KDD.
• Lundberg, S., & Lee, S. (2017). A Unified Approach to Interpreting Model Predictions. NeurIPS.
• Reichheld, F. (1996). The Loyalty Effect: The Hidden Force Behind Growth, Profits, and Lasting Value.
• Chawla, N. V. (2002). SMOTE: Synthetic Minority Over-sampling Technique. JAIR.

Table of Contents