Breaking Down RSS, TSS, and ESS for Better Regression Understanding

Understanding RSS, TSS, ESS & R² in Regression

In regression analysis, we measure how well a model explains variation in data using three core quantities: Total Sum of Squares (TSS), Residual Sum of Squares (RSS), and Explained Sum of Squares (ESS).

---

🎯 Learning Goal

Understand how total variation in data is decomposed into explained and unexplained parts.

💡 Core Identity: TSS = ESS + RSS

---

📘 Key Definitions

1️⃣ Total Sum of Squares (TSS)

Definition: Measures total variation in y around its mean.

TSS = Σ (y_i - y_mean)^2

• y_i → actual values
• y_mean → mean of y

💡 TSS represents total variability before modeling.

---

2️⃣ Residual Sum of Squares (RSS)

Definition: Measures unexplained variation (model error).

RSS = Σ (y_i - y_hat_i)^2

• y_hat_i → predicted values

💡 RSS measures how wrong the model is.

---

3️⃣ Explained Sum of Squares (ESS)

Definition: Measures variation explained by the model.

ESS = Σ (y_hat_i - y_mean)^2

💡 ESS measures how much the model explains.

---

🔗 The Fundamental Relationship

TSS = ESS + RSS

The total variability in y is split into:

• Explained part (ESS)
• Unexplained part (RSS)

💡 Regression decomposes total variation into explained and residual components.

---

📊 Visual Interpretation (Conceptual)

Think of It Geometrically

TSS → Distance from actual points to the mean ESS → Distance from predictions to the mean RSS → Distance from actual points to predictions

Graphically:

• Mean line → baseline model
• Regression line → improved model
• Vertical gaps → residuals

---

📈 Coefficient of Determination (R²)

R^2 = ESS / TSS
R^2 = 1 - (RSS / TSS)

Interpretation

• R² = 1 → Perfect fit
• R² = 0 → No improvement over mean

💡 R² measures the proportion of total variance explained by the model.

---

🧪 Step-by-Step Example Logic

How You Compute in Practice

1. Compute y_mean
2. Calculate TSS using actual values
3. Fit regression → obtain y_hat
4. Calculate RSS
5. Compute ESS = TSS − RSS
6. Compute R²

---

📌 Final Summary

• TSS → Total variability
• ESS → Explained variability
• RSS → Unexplained variability

💡 A good regression model minimizes RSS and maximizes ESS.

---

End of Interactive Learning Guide

Related Topics: Data Science, Machine Learning & AI

Dive deeper into data science concepts, machine learning techniques, and artificial intelligence foundations. These articles simplify complex topics like NumPy arrays, regression analysis, neural networks, and reinforcement learning for practical understanding.

• NumPy Array Dimensions Explained: Breaking Down the (2, 2, 4) Shape
• Breaking Down Neural Turing Machines and Attention in Computer Vision: Simplified for Everyone
• Residuals and RSS in Linear Regression: A Simple Explanation
• Breaking Down Decision-Making: The Hierarchy of Abstract Machines in Reinforcement Learning

These resources are ideal for learners and professionals looking to strengthen their understanding of core data science concepts, from numerical computing with NumPy to advanced AI models and decision-making systems.