kaiyin.zhong

Writing

Essays

Things I have learned. Things I built, broke, and eventually figured out.

  1. The directions that matter Dec 20, 2016 · 10 min Given a cloud of data in high dimensions, which few directions carry almost all of the story? Principal component analysis answers it by finding the axes along which the data varies most — and those axes turn out to be exactly the top eigenvectors of the covariance matrix, or equivalently the top singular vectors of the data. It's the spectral theorem, positive-definiteness, and the SVD all cashing out at once. Here's the derivation, why the leftover directions are the ones safe to drop, and a playground where you reshape a point cloud and watch its principal axes swing to follow. linear algebra · machine learning · math · statistics
  2. Every matrix in three acts Nov 8, 2016 · 10 min There's one factorization that works for every matrix — square or not, invertible or not — and it says something almost unreasonable: whatever a matrix does, it does it as a rotation, then a set of independent stretches, then another rotation. That's the singular value decomposition. Here's the geometry that turns a circle into an ellipse, why the singular values are the square roots of eigenvalues in disguise, the Eckart–Young theorem behind every low-rank approximation, and a playground where you watch the three acts play out. linear algebra · machine learning · math
  3. Positive definite: the shape of a bowl Sep 13, 2016 · 9 min Feed a symmetric matrix a vector from both sides and you get a single number — and the sign of that number, for every vector at once, decides whether a surface curves up like a bowl, down like a dome, or twists into a saddle. Positive-definite matrices are the bowls, and they're everywhere machine learning lives: covariance matrices, Hessians, the loss surfaces you roll downhill. Here's the quadratic form, the tests for definiteness, and a playground where you bend a matrix from bowl to saddle and back. linear algebra · machine learning · math · statistics
  4. Eigenvectors and the spectral theorem Jul 5, 2016 · 10 min Most vectors get knocked off course when a matrix hits them. A precious few don't — they only stretch or shrink, never turn. Those are the eigenvectors, and they're the closest thing a matrix has to a set of natural axes. Here's what Av = λv really means, how diagonalization makes powers of a matrix trivial, the spectral theorem that hands symmetric matrices a perfect orthonormal set of axes, and a playground where you watch power iteration home in on the dominant direction. linear algebra · machine learning · math
  5. The shadow of a vector May 3, 2016 · 9 min Ask where a vector's shadow falls on a wall and you've asked the deepest question in applied linear algebra. Projection is how you find the closest point in a subspace to a point outside it — and the leftover, the part you couldn't reach, always lands exactly perpendicular. Here's projection onto a line and onto a subspace, the projection matrix, Gram–Schmidt and QR, and a playground where you drag a vector and watch its shadow and its perpendicular residual move with it. linear algebra · machine learning · math
  6. The four fundamental subspaces Mar 8, 2016 · 9 min Every matrix quietly carves its two worlds — input space and output space — into four subspaces, and once you see them, the whole question of 'does this system have a solution?' becomes a matter of geometry. Here's the column space and null space, the rank–nullity theorem that ties their sizes together, why solvability means 'b lives in the column space', and a playground where you pick a target and watch it fall inside or outside the reachable set. linear algebra · machine learning · math
  7. A matrix is a verb: what a linear map does Jan 19, 2016 · 9 min A matrix isn't a grid of numbers you memorize rules for — it's a verb. It takes a vector and moves it. Its columns tell you exactly where the basis vectors land, and that alone pins down everything the matrix does to everything else. Here's the linear map hiding behind the notation, why matrix multiplication is really function composition, what the determinant measures, and a live playground where you drag the columns and watch the whole plane bend with them. linear algebra · machine learning · math