Theory & Background

Understanding the mathematics behind fraud detection

Table of Contents

1. Class Imbalance Problem 2. Evaluation Metrics 3. Isolation Forest 4. Gradient Boosting 5. Stream Processing 6. Graph-Based Detection

1. The Class Imbalance Problem

Fraud detection presents a fundamental class imbalance challenge. In typical transaction data, fraudulent transactions represent only 0.1-2% of all transactions, creating a highly skewed distribution that standard ML algorithms struggle with.

SMOTE: Synthetic Minority Over-sampling

SMOTE generates synthetic samples by interpolating between existing minority class instances:

xnew = xi + λ · (xnn - xi)

Where xi is a minority sample, xnn is one of its k-nearest neighbors, and λ ∈ [0,1] is random.

Cost-Sensitive Learning

Alternative approach using asymmetric costs in the loss function:

L = -Σ [w1·y·log(p) + w0·(1-y)·log(1-p)]

Where w1 >> w0 to penalize missing fraud cases more heavily.

2. Evaluation Metrics for Fraud Detection

Why Not Accuracy?

With 99% legitimate transactions, a model predicting "not fraud" for everything achieves 99% accuracy but catches zero fraud. Instead, we use:

Precision
TP / (TP + FP)

Of flagged transactions, how many are actually fraud?

Recall
TP / (TP + FN)

Of all fraud cases, how many did we catch?

AUC-ROC

Probability that model ranks random fraud higher than random legit.

PR-AUC

Area under Precision-Recall curve; better for imbalanced data.

3. Isolation Forest Algorithm

Isolation Forest exploits the principle that anomalies are "few and different" - they require fewer random splits to isolate.

Anomaly Score

s(x, n) = 2-E[h(x)] / c(n)

Where:

  • h(x) = path length for instance x
  • c(n) = average path length in a binary search tree
  • E[h(x)] = expected path length over all trees
Key Insight

Score close to 1 = anomaly; Score close to 0.5 = normal; Score close to 0 = very normal

4. Gradient Boosting for Fraud

XGBoost and LightGBM are ensemble methods that sequentially build decision trees, with each tree correcting errors of previous ones.

Objective Function

Obj = Σ L(yi, ŷi) + Σ Ω(fk)

L = Loss function (log loss for classification)
Ω = Regularization term to prevent overfitting

XGBoost Key Parameters for Fraud

scale_pos_weight Balance positive/negative weights (set to ratio of negatives/positives)
max_depth Limit tree depth (3-8 typical for fraud)
subsample Row sampling per tree (0.8-1.0)

5. Stream Processing Architecture

Apache Kafka + Spark Streaming

Real-time fraud detection requires processing transactions within milliseconds:

Transaction
Kafka Topic
Spark Consumer
ML Scoring
Decision

Latency Requirements

  • P99 latency target: < 100ms for card-present transactions
  • Feature computation: Real-time aggregations via Redis/Flink
  • Model inference: Optimized with ONNX or TensorRT

6. Graph-Based Fraud Detection

Fraud rings share resources (devices, IPs, addresses). Graph analysis reveals these hidden connections.

Entity Resolution Graph

Nodes represent entities (cards, devices, emails, addresses). Edges connect entities that appear in the same transaction or share attributes.

PageRank

Identifies important nodes in fraud networks

Community Detection

Louvain algorithm finds fraud ring clusters

Label Propagation

Spreads fraud labels through connected entities

Graph Neural Networks

Learn representations from graph structure

References

  • Liu, F.T., et al. (2008). "Isolation Forest." ICDM.
  • Chen, T. & Guestrin, C. (2016). "XGBoost: A Scalable Tree Boosting System." KDD.
  • Chawla, N.V., et al. (2002). "SMOTE: Synthetic Minority Over-sampling Technique."
  • Akoglu, L., et al. (2015). "Graph-Based Anomaly Detection and Description: A Survey."