ROC vs AUC: A Beginner’s Guide to Model Evaluation

ROC AUC Explained with Deep Mathematical Understanding

📚 Table of Contents

• Introduction
• ROC Curve Mathematics
• Deep Mathematical Explanation
• Probability Interpretation
• Numerical Example
• Threshold Analysis
• Conclusion

📘 Introduction

ROC AUC is not just a performance metric — it is a mathematical representation of ranking quality. To truly understand it, we must go beyond definitions and explore the math behind it.

📈 ROC Curve Mathematics

True Positive Rate (TPR)

$$ TPR = \frac{TP}{TP + FN} $$

Explanation:

• TP = correctly predicted positives
• FN = missed positives
• Denominator = total actual positives

👉 Meaning: Out of all real positives, how many did we correctly identify?

False Positive Rate (FPR)

$$ FPR = \frac{FP}{FP + TN} $$

Explanation:

• FP = incorrectly predicted positives
• TN = correctly predicted negatives
• Denominator = total actual negatives

👉 Meaning: Out of all real negatives, how many did we wrongly classify?

📐 Deep Mathematical Explanation

ROC Curve Construction

Each point on ROC curve corresponds to a threshold \(t\).

$$ TPR(t), \quad FPR(t) $$

As threshold decreases:

• More samples become positive
• TPR increases
• FPR also increases

Integral Definition of AUC

$$ AUC = \int_0^1 TPR(FPR^{-1}(x)) dx $$

Explanation:

• We sweep across all FPR values from 0 to 1
• For each FPR, we compute TPR
• The integral sums the entire curve area

👉 This is equivalent to calculating area under curve using calculus.

🎯 Probability Interpretation

One of the most powerful interpretations:

$$ AUC = P(score_{positive} > score_{negative}) $$

Explanation:

• Pick one positive sample
• Pick one negative sample
• Check if model ranks positive higher

👉 AUC = probability that model ranks correctly

Expanded Pairwise Form

$$ AUC = \frac{1}{|P||N|} \sum_{p \in P} \sum_{n \in N} I(s_p > s_n) $$

Where:

• P = set of positives
• N = set of negatives
• I = indicator function (1 if true, 0 otherwise)

👉 This shows AUC is based on pairwise comparisons.

🧮 Numerical Example

Suppose:

• Positive scores: 0.9, 0.8
• Negative scores: 0.6, 0.4

Compare all pairs:

0.9 > 0.6 ✔
0.9 > 0.4 ✔
0.8 > 0.6 ✔
0.8 > 0.4 ✔

All comparisons correct → AUC = 1

🎚 Threshold Analysis

Threshold controls classification:

• High threshold → strict → fewer positives
• Low threshold → lenient → more positives

Mathematically:

$$ \hat{y} = \begin{cases} 1 & \text{if } score \geq t \\ 0 & \text{otherwise} \end{cases} $$

🎯 Conclusion

✔ ROC AUC measures ranking quality ✔ Based on probability & integration ✔ Not dependent on a single threshold