KNN Regression: A Complete Solved Numerical Example
Scenario: Used Laptop Price Estimation
The Objective: Predict the exact resale price of a used laptop by mathematically finding similar past sales and averaging their values.
Core Mechanics▼
- Same Start, Different End: Like Classification, it begins by finding the nearest neighbors. The only difference is that instead of a category, you are predicting a continuous number.
- Average, Don't Vote: Since you aren't choosing a class, you take the mean (average) of your neighbors' values. The result is a specific numerical prediction, not a count.
- Outlier Sensitivity: In Classification, one rogue neighbor rarely changes the outcome. In Regression, a single extreme neighbor can dramatically pull your average away from the truth.
- The Smoothness Tradeoff: A small captures local detail but is jittery and noisy. A large produces smooth predictions but can drift too far toward the global average, losing all local accuracy.
Step 1: The Historical Data & Target Point
To predict a continuous value, KNN Regression looks at the most similar historical data points. We are predicting the Resale_Price_GBP for a new target point with features: [20, 78] using K = 3.
| Data Point | Age_Months | Battery_Health_Percent | Resale_Price_GBP |
|---|---|---|---|
| P1 | 6 | 95 | 850 |
| P2 | 12 | 88 | 720 |
| P3 | 18 | 80 | 600 |
| P4 | 24 | 72 | 500 |
| P5 | 30 | 65 | 420 |
| P6 | 8 | 92 | 800 |
| P7 | 15 | 85 | 650 |
| P8 | 36 | 55 | 320 |
| Target | 20 | 78 | ? |
Step 2: Calculate Euclidean Distances
First, we measure exactly how "far" our target point is from every single historical row using the Euclidean distance formula.
d = √((20 - 6)² + (78 - 95)²)
d = √(196 + 289)
d = √485
d = 22.023
d = √((20 - 12)² + (78 - 88)²)
d = √(64 + 100)
d = √164
d = 12.806
d = √((20 - 18)² + (78 - 80)²)
d = √(4 + 4)
d = √8
d = 2.828
d = √((20 - 24)² + (78 - 72)²)
d = √(16 + 36)
d = √52
d = 7.211
d = √((20 - 30)² + (78 - 65)²)
d = √(100 + 169)
d = √269
d = 16.401
d = √((20 - 8)² + (78 - 92)²)
d = √(144 + 196)
d = √340
d = 18.439
d = √((20 - 15)² + (78 - 85)²)
d = √(25 + 49)
d = √74
d = 8.602
d = √((20 - 36)² + (78 - 55)²)
d = √(256 + 529)
d = √785
d = 28.018
Step 3: Select the Top K Neighbors
We rearrange the calculated distances in ascending order (smallest to largest) and select the top K = 3 closest neighbors.
| Rank | Point | Distance | Target Value |
|---|---|---|---|
| #1 | P3 | 2.828 | 600 |
| #2 | P4 | 7.211 | 500 |
| #3 | P7 | 8.602 | 650 |
| #4 | P2 | 12.806 | 720 |
| #5 | P5 | 16.401 | 420 |
| #6 | P6 | 18.439 | 800 |
| #7 | P1 | 22.023 | 850 |
| #8 | P8 | 28.018 | 320 |
Step 4: Final Regression Prediction
In KNN Regression, the final prediction is simply the average (mean) of the target values from our selected K neighbors.
Prediction = (600 + 500 + 650) / 3
Prediction = 1750 / 3
Prediction = 583.333
Final Takeaway
Notice the critical difference in Step 4 compared to standard KNN Classification! Instead of taking a rigid majority vote, KNN Regression calculates the mathematical average of the 3 closest laptops (600, 500, and 650) to predict a highly specific, continuous price.