What Is a Cost Function? Understanding Its Role in Model Training

Evaluating a Model with a Cost Function

Understanding Mean Squared Error using a house price prediction example

When building machine learning models, predictions are rarely perfect. To understand how well a model performs, we use a cost function. This page walks through a concrete example using house prices and Mean Squared Error (MSE).

Scenario Overview

Imagine you have a model that predicts house prices, and you want to evaluate how accurate those predictions are.

🏠 Actual vs Predicted Prices

House	Actual Price ($)	Predicted Price ($)
1	200,000	210,000
2	250,000	240,000
3	300,000	290,000

What Is a Cost Function?

A cost function measures how far the model’s predictions are from the actual values. In regression problems, a commonly used cost function is Mean Squared Error (MSE).

Step-by-Step Cost Calculation

1️⃣ Calculate Errors

The error is the difference between the actual price and the predicted price.

House 1: 200,000 - 210,000 = -10,000
House 2: 250,000 - 240,000 =  10,000
House 3: 300,000 - 290,000 =  10,000

2️⃣ Square the Errors

Squaring the errors ensures that negative and positive errors do not cancel each other out. It also penalizes larger mistakes more heavily.

(-10,000)² = 100,000,000
( 10,000)² = 100,000,000
( 10,000)² = 100,000,000

3️⃣ Average the Squared Errors (MSE)

To get the Mean Squared Error, we take the average of all squared errors.

MSE = (100,000,000 + 100,000,000 + 100,000,000) / 3
MSE = 100,000,000

Interpreting the Result

An MSE of 100,000,000 means that, on average, the model’s predictions deviate significantly from the actual prices.

The purpose of training a machine learning model is to adjust its parameters so this cost function is minimized. As the MSE decreases, predictions become closer to real-world values.

💡 Key Takeaways

• Cost functions quantify prediction error
• Mean Squared Error penalizes large mistakes
• Squaring prevents error cancellation
• Lower MSE indicates a better-performing model
• Training aims to minimize the cost function

Educational walkthrough of Mean Squared Error in regression models.