Sam Newby
← Learning ML

Me

When Zero means Fifty percent

Today I learned something interesting while building my own ML models from scratch. A model outputting 0 doesn’t necessarily mean a 0% probability.

The linear regression model I’ve been building can output almost any number. That’s fine when predicting values, but what if I want to predict the probability of something happening?

My first instinct was to scale the outputs between 0 and 1. But what happens when the model produces a number outside the range I originally used? That’s where the sigmoid function comes in.

It transforms any real number into a value between 0 and 1.

  • -3 → ~4.7%
  • 0 → 50%
  • 3 → ~95.3%

Of course, turning a score into a number between 0 and 1 doesn’t automatically make it a well-calibrated probability. But what I find most interesting isn’t the maths itself. It’s understanding why we need these functions rather than blindly using them through a library.

Every little concept makes building my first neural network feel more achievable.