Course Details
Subject {L-T-P / C} : EE6139 : Information Theory and Coding { 3-0-0 / 3}
Subject Nature : Theory
Coordinator : Prof. Dipti Patra
Syllabus
Module 1:
Information theory: Concept of amount of information, information units Entropy: marginal, conditional, joint and relative entropies, relation among entropies Mutual information, information rate, channel capacity, redundancy and efficiency of channels Discrete channels – Symmetric channels, Binary Symmetric Channel, Binary Erasure Channel, Noise-Free Channel, Channel with independent I/O, Cascaded channels, repetition of symbols, Binary asymmetric channel, Shannon theorem. [ 8 hours].
Module 2:
Source coding – Encoding techniques, Purpose of encoding, Instantaneous codes, Construction of instantaneous codes, Kraft’s inequality, Coding efficiency and redundancy, Source coding theorem. Construction of basic source codes – Shannon Fano coding, Shannon Fano Elias coding, Huffman coding, Minimum variance Huffman coding, Adaptive Huffman coding, Arithmetic coding, Dictionary coding – LZ77, LZ78, LZW, ZIP coding Channel coding, Channel coding theorem for DMC. [8 hours]
Module 3:
Codes for error detection and correction – Parity check coding, Linear block codes, Error detecting and correcting capabilities, Generator and Parity check matrices, Standard array and Syndrome decoding, Hamming codes Cyclic codes – Generator polynomial, Generator and Parity check matrices, Encoding of cyclic codes, Syndrome computation and error detection, Decoding of cyclic codes, BCH codes, RS codes, Burst error correction. [10 hours]
Module 4:
Convolutional codes – Encoding and State, Tree and Trellis diagrams, Maximum likelihood decoding of convolutional codes -Viterbi algorithm, Sequential decoding -Stack algorithm. Interleaving techniques – Block and convolutional interleaving, Coding and interleaving applied to CD digital audio system - CIRC encoding and decoding, interpolation and muting. ARQ – Types of ARQ, Performance of ARQ, Probability of error and throughput. [10 hours]
Course Objectives
- To equip students with the basic understanding of the fundamental concept of entropy and information theory.
- To learn the principles and applications of information theory in modern communication systems
- To learn how to design data compression codes to improve the efficiency of information transmission.
- To understand the theoretical framework upon which error-control codes are built.
Course Outcomes
CO1: Able to carry out research and development of the encoding strategy and system design. <br />CO2: Become well aware on variety of information theoretic statistical approaches and their diverge fields of applications in data communication, wired or wireless. <br />CO3: Able to do in-depth analysis on technology variations in combination of encoder and decoder depending on channel condition, suitable channel modeling, and exercise the information theory for encoder design. <br />CO4: Able to differentiate between lossy and lossless compression techniques and decide an efficient data compression scheme for a given information source to industry applications as commercially used in wireless communication systems, data communication in computer networks. <br />CO5: Able to do data encryption and decryption for crucial applications, i.e. in various confidential data communication, security, banking systems.
Essential Reading
- Thomas M. Cover and Joy A. Thomas, Elements of Information Theory, John Wiley , 2014 Edition
- Jorge C. Moreira and Patric G Farrell, Essentials of Error Control and Coding, John Wiley , 2015 Edition
Supplementary Reading
- John C. Hancock, An Introduction to the Principles of Communication Theory, McGraw-Hill , 2015 Edition
- R.Bose, Information Theory Coding and Cryptography, McGraw-Hill