Information Theory

Shannon entropy

Definition

A measure of the uncertainty in a random variable, foundational to information theory.