Shannon Entropy in Physical Signal Encoding
Applying Shannon's information measures to characterize the entropy of signals captured from physical systems.
Information Content of Signals
When we capture a signal from a physical system, how much information does it contain? Shannon entropy provides a rigorous answer, measuring the average surprise per observation in a signal stream.
Entropy and Spectral Structure
A signal with a flat spectrum (white noise) has maximum entropy - it carries no structural information. Conversely, a signal with sharp spectral peaks has low entropy, indicating that the system’s energy is concentrated at specific frequencies.
The entropy of a spectral signature is inversely related to the amount of structural information it carries about the source system.
Spectral Entropy as a Diagnostic
We define spectral entropy as the Shannon entropy of the normalized power spectral density. This single scalar quantity serves as a first-pass diagnostic for the information richness of a captured signal.
Topics covered