IT I: Contents
Block Diagram of a Communications System, Entropy, Relative Entropy.
Concave Functions, Jensen's Inequality, Conditional Entropy, Mutual Information.
Fano's Inequality, Non-Singular Codes, Uniquely Decodable Codes, Prefix-Free Codes.
Kraft's Inequality, Shannon and Huffman Codes.
Shannon's Source Coding Theorem.
Data Processing Inequality, Log-Sum Inequality, Convexity of Relative Entropy, Typical Sequences, AEP.
Data Compression and Typicality, Channel Capacity.
Concavity/Convexity of Mutual Information, Kuhn-Tucker Conditions, (Weakly) Symmetric Channels.
Block Codes/Encoder/Decoder, Converse of the Channel Coding Theorem, Joint Typicality.
Channel Coding Theorem (direct part), Source-Channel Separation Theorem.
Channels with Feedback.
A Glimpse at Multi-Terminal Information Theory.
© Signal and Information Processing Laboratory (ISI), ETH Zurich