Writing
Essays
Things I have learned. Things I built, broke, and eventually figured out.
- Logistic regression: bending a line until it becomes a probability Apr 25, 2017 · 8 min You can't answer a yes-or-no question with a straight line — a line runs off to infinity in both directions, and probabilities are stuck between 0 and 1. Logistic regression fixes this with one elegant move: keep the linear part, then squash it through a sigmoid. Here's why the log-odds turn out linear, how the Bernoulli likelihood collapses into cross-entropy, why there's no tidy closed form this time, and how a single clean gradient carries the whole thing downhill. machine learning · math · statistics
- Linear regression: the honest workhorse Feb 21, 2017 · 9 min Fit a line, and least squares hands you numbers. Turn it into a model — with a claim about the world, error bars on every coefficient, and a theorem that says you can't do better without giving something up — and you have linear regression. Here's the model it actually asserts, why its estimator is provably the best of its kind, what each coefficient really means, and the four quiet assumptions holding the whole thing up. machine learning · math · statistics
- Least squares: the line that argues with every point Jan 17, 2017 · 8 min Draw a line through a cloud of points and you've already made a choice about what 'best' means. Least squares makes that choice precise — and it turns out to be two stories at once: a calculus problem you can solve by setting a gradient to zero, and a geometry problem where the answer is a shadow. Here's both derivations, why we square the errors in the first place, and a live playground where you drag the points and watch the line fight back. machine learning · math · statistics
- The directions that matter Dec 20, 2016 · 10 min Given a cloud of data in high dimensions, which few directions carry almost all of the story? Principal component analysis answers it by finding the axes along which the data varies most — and those axes turn out to be exactly the top eigenvectors of the covariance matrix, or equivalently the top singular vectors of the data. It's the spectral theorem, positive-definiteness, and the SVD all cashing out at once. Here's the derivation, why the leftover directions are the ones safe to drop, and a playground where you reshape a point cloud and watch its principal axes swing to follow. linear algebra · machine learning · math · statistics
- Positive definite: the shape of a bowl Sep 13, 2016 · 9 min Feed a symmetric matrix a vector from both sides and you get a single number — and the sign of that number, for every vector at once, decides whether a surface curves up like a bowl, down like a dome, or twists into a saddle. Positive-definite matrices are the bowls, and they're everywhere machine learning lives: covariance matrices, Hessians, the loss surfaces you roll downhill. Here's the quadratic form, the tests for definiteness, and a playground where you bend a matrix from bowl to saddle and back. linear algebra · machine learning · math · statistics