Syllabus

Module 1:
Information theory: Concept of amount of information, information units Entropy: marginal, conditional, joint and relative entropies, relation among entropies Mutual information, information rate, channel capacity, redundancy and efficiency of channels Discrete channels – Symmetric channels, Binary Symmetric Channel, Binary Erasure Channel, Noise-Free Channel, Channel with independent I/O, Cascaded channels, repetition of symbols, Binary asymmetric channel, Shannon theorem. [ 8 hours].

Module 2:
Source coding – Encoding techniques, Purpose of encoding, Instantaneous codes, Construction of instantaneous codes, Kraft’s inequality, Coding efficiency and redundancy, Source coding theorem. Construction of basic source codes – Shannon Fano coding, Shannon Fano Elias coding, Huffman coding, Minimum variance Huffman coding, Adaptive Huffman coding, Arithmetic coding, Dictionary coding – LZ77, LZ78, LZW, ZIP coding Channel coding, Channel coding theorem for DMC. [8 hours]

Module 3:
Codes for error detection and correction – Parity check coding, Linear block codes, Error detecting and correcting capabilities, Generator and Parity check matrices, Standard array and Syndrome decoding, Hamming codes Cyclic codes – Generator polynomial, Generator and Parity check matrices, Encoding of cyclic codes, Syndrome computation and error detection, Decoding of cyclic codes, BCH codes, RS codes, Burst error correction. [10 hours]

Module 4:
Convolutional codes – Encoding and State, Tree and Trellis diagrams, Maximum likelihood decoding of convolutional codes -Viterbi algorithm, Sequential decoding -Stack algorithm. Interleaving techniques – Block and convolutional interleaving, Coding and interleaving applied to CD digital audio system - CIRC encoding and decoding, interpolation and muting. ARQ – Types of ARQ, Performance of ARQ, Probability of error and throughput. [10 hours]

To equip students with the basic understanding of the fundamental concept of entropy and information theory.

To learn the principles and applications of information theory in modern communication systems

To learn how to design data compression codes to improve the efficiency of information transmission.

To understand the theoretical framework upon which error-control codes are built.

CO1: Able to carry out research and development of the encoding strategy and system design.
CO2: Become well aware on variety of information theoretic statistical approaches and their diverge fields of applications in data communication, wired or wireless.
CO3: Able to do in-depth analysis on technology variations in combination of encoder and decoder depending on channel condition, suitable channel modeling, and exercise the information theory for encoder design.
CO4: Able to differentiate between lossy and lossless compression techniques and decide an efficient data compression scheme for a given information source to industry applications as commercially used in wireless communication systems, data communication in computer networks.
CO5: Able to do data encryption and decryption for crucial applications, i.e. in various confidential data communication, security, banking systems.

Thomas M. Cover and Joy A. Thomas, Elements of Information Theory, John Wiley , 2014 Edition

Jorge C. Moreira and Patric G Farrell, Essentials of Error Control and Coding, John Wiley , 2015 Edition

John C. Hancock, An Introduction to the Principles of Communication Theory, McGraw-Hill , 2015 Edition

R.Bose, Information Theory Coding and Cryptography, McGraw-Hill