Experiments / live

ReLU playground

Two things one bent line can do. On the left, ReLU next to its gentler cousins — drag the leaky slope and watch how each variant treats the negative side while agreeing far from zero. On the right, the universal-approximation trick made tangible: a few ReLU bumps, each with its own breakpoint and weight, summed into a piecewise-linear curve you tune by hand toward a target. Get the knots right and the error drops toward zero. The full story — why the bend beats the S-curve, and how a sum of hinges bends into anything — is in the essay.