All Research

Information Theory

Shannon entropy, KL-divergence, coding theory, and the physical nature of information.

Shannon EntropyMutual InformationKL-DivergenceCross-EntropyChannel CapacityInformation EncodingCompressionInformation in Physics