MODULE-I: Discrete Information Sources: Source alphabets and entropy. Joint and conditional entropy, Entropy, Entropy of symbol blocks and chain rule, Relative and Mutual information. Joint information. Conditional information, Jensen’s Inequality, Log-Sum Inequality, Fano’s Inequality. [04 Hours]

MODULE-II: Asymptotic Equipartition property, Markov Chain, Entropy Rates, Markov source, and source with memory. Markov chain, Entropy rates. [04 Hours]

MODULE-III: Source coding. Kraft Inequality, Source Coding Theorem, Prefix Codes, Optimal Codes, Huffman coding, Shanon Fano Coding, dictionary codes and Lempel-Ziv Coding. Arithmetic coding. [05 Hours]

MODULE-IV: Information Channels, Mutual information of channels, Channel Capacity, Examples of Channel Capacity, Reliable Messages and Unreliable Channels, Example of Coding to correct errors, Shannon’s Second theorem. [08 Hours]

MODULE-IV: Discrete Algebra, Vector Spaces, Convolution codes: Structural properties of convolution codes, Viterbi algorithm for hard decision and soft decision. Puncture Convolution codes. Trellis Coded Modulation: Multi-amplitude/Multi phase discrete memory less channels. Systematic recursive convolution encoder, signal mapping and set partitioning, Trellis codes for PSK and QAM. [15 Hours]

To build concepts on secure communication by knowing techniques of source coding, channel capacity and channel coding.

Build in-depth concept on information theoretic approach in coding for digital communication and data compression.

Masters students will be able to know the operational principle of various encoding and decoding methodologies and their present day advancements and industry requirements.

CO1: Develop a basic understanding of the statistical approaches and mathematical tools needed for understanding the information-theoretic perspective of communication.

CO2: Understand how to apply information theory to code the data to be communicated.

CO3: Apply the statistical approaches and tools and develop codes for transmitted data .

CO4: Develop an indepth understanding of how to transmit information securely through unreliable channels by applying the concept of information theory to channels.

CO5: Extend the knowledge gained in information theory to design various error control codes and their respective encoding and decoding schemes Apply the knowledge to design new error control coding schemes.

Thomas M. Cover, Joy A. Thomas, Elements of Information Theory,, Wiley , 2nd Edition 2006

Shu Lin and Daniel Jr. Costello,, Error Control Coding,, Prentice Hall , Second Edition

Norman Abramson,, Information Theory and Coding,, McGraw Hill , Electronic Sciences Series

R Bose, Information Theory, Coding and Cryptography, McGraw Hill Education India Private Limited , 2nd Edition 2017