What Is Multicollinearity in Regression? A Beginner’s Guide

Multicollinearity occurs when two or more predictor variables in a regression model are highly correlated, meaning they provide similar information about the response variable. This can make it difficult to determine the individual effect of each predictor on the outcome.

### Simple Example

Let's say you want to predict someone's **weight** based on their **height** and **shoe size**.

- **Height** and **shoe size** are often correlated because taller people generally have larger feet.

- In this case, both height and shoe size might explain some of the variation in weight, but because they are correlated, it’s hard to figure out how much of the variation in weight is due to height and how much is due to shoe size. 

### Why It’s a Problem

When multicollinearity is present:

1. **Coefficient Estimates Become Unreliable**: The estimates of the coefficients for height and shoe size could become very large or even have unexpected signs (like a positive relationship showing up as negative).

2. **Statistical Significance**: It might appear that neither height nor shoe size is statistically significant, even though together they should explain weight well.

### How to Spot It

One way to spot multicollinearity is by looking at the **correlation** between the predictors. If height and shoe size have a high correlation (close to 1 or -1), there might be multicollinearity.

### Simplified Visualization

- Imagine a Venn diagram with two overlapping circles, one for height and one for shoe size. The overlapping part represents the shared information. Multicollinearity means there's a large overlap, making it hard to separate the individual contributions of height and shoe size to the prediction of weight.

In summary, multicollinearity makes it difficult to distinguish the separate effects of highly correlated variables on the outcome, leading to unstable estimates and potential misinterpretation of the model.