Me
When Zero means Fifty percent
Today I learned something interesting while building my own ML models from scratch. A model outputting 0 doesn’t necessarily mean a 0% probability.
The linear regression model I’ve been building can output almost any number. That’s fine when predicting values, but what if I want to predict the probability of something happening?
My first instinct was to scale the outputs between 0 and 1. But what happens when the model produces a number outside the range I originally used? That’s where the sigmoid function comes in.
It transforms any real number into a value between 0 and 1.
-3→ ~4.7%0→ 50%3→ ~95.3%
Of course, turning a score into a number between 0 and 1 doesn’t automatically make it a well-calibrated probability. But what I find most interesting isn’t the maths itself. It’s understanding why we need these functions rather than blindly using them through a library.
Every little concept makes building my first neural network feel more achievable.