Experiments / live

Cross entropy playground

Two ways to feel the loss that trains almost every classifier. On the left, slide the predicted probability of the true class and watch the −log(p) curve fall gently when you're confident and correct, and spike toward infinity as you drift into a confident mistake. On the right, drag the bars of a predicted distribution q against a fixed one-hot truth and watch cross entropy H(p,q) and KL divergence D(p‖q) bottom out exactly when your bars match reality. The full story — entropy, KL, and why the gradient collapses to (prediction − target) — is in the essay.