Course Details
Subject {L-T-P / C} : EC4507 : Information Theory and Coding { 3-0-0 / 3}
Subject Nature : Theory
Coordinator : Prof. Siddharth Deshmukh
Syllabus
MODULE-I: Discrete Information Sources: Source alphabets and entropy. Joint and conditional entropy, Entropy of symbol blocks and chain rule. [06 Hours]
MODULE-II: Mutual information. Joint information. Conditional information. [05 Hours]
MODULE-III: Source coding. Huffman coding, dictionary codes and Lempel-Ziv Coding. Arithmetic coding. [05 Hours]
MODULE-IV: Discrete Channels and Capacity: Discrete memory less channel model, channel capacity and binary symmetric channel, Block coding and Shannon’s second theorem. [07 Hours]
MODULE-V: Markov source and source with memory. Markov chain and data processing. Constrained channels. Autocorrelation and power spectrum of sequences. [03 Hours]
MODULE-VI: Data translation codes. Run Length Codes: Prefix and Block codes, State dependent fixed length block codes. Variable length fixed rate codes. Look ahead codes. DC free codes. Linear Block and Error Correcting Codes: Binary fields and Binary vector spaces. Linear block codes. Decoding linear block codes. Hamming Codes. Error rate performance bounds for linear block codes. Cyclic Codes: Polynomial representation of cyclic codes. Generation and decoding of cyclic codes. Cyclic Hamming codes, BCH codes, Burst error correcting codes, cyclic redundant check codes. Convolution codes: Structural properties of convolution codes, Viterbi algorithm for hard decision and soft decision. Puncture Convolution codes. Trellis Coded Modulation: Multi-amplitude/Multi phase discrete memory less channels. Systematic recursive convolution encoder, signal mapping and set partitioning, Trellis codes for PSK and QAM. [10 Hours]
MODULE-VII: Introduction to cryptography: Basic concept of crypt system, Attacks on crypto system, Prefect secrecy, language entropy and cipher text attack, Computational security, Product cipher system, Public-key Crypto systems. [02 Hours]
Course Objectives
- To introduce basic concepts of information theory, such as entropy, mutual information, etc.
- To introduce concepts on source coding, specifically, lossless and lossy encoding.
- To introduce concepts on channel coding, specifically, block codes, FEC codes etc.
- To introduce fundamental concepts on capacity of communication systems.
Course Outcomes
CO1: Able to carry out research and development of the encoding strategy and system design. <br />CO2:Become well aware on variety of information theoretic statistical approaches and their diverge fields of applications in data communication, wired or wireless. <br />CO3:Able to do in-depth analysis on technology variations in combination of encoder and decoder depending on channel condition, suitable channel modeling, and exercise the information theory for encoder design. <br />CO4:Acquire improvise the knowledge on statistical approaches of signal processing through rigorous analysis are attractive to industry applications as commercially used wireless communication systems, RADAR systems in various military application and also, data communication in computer networks.
Essential Reading
- Thomas M. Cover, Joy A. Thomas, Elements of Information Theory, Wiley , 2nd Edition 2006
- Richard B. Well, Applied Coding and Information Theory for Engineers, Pearson , Low price edition 1999
Supplementary Reading
- Stefan M. Moser, Po-ling Chen, A student’s Guide to Coding and Information Theory, Cambridge , 1st Edition 2012
- R Bose, Information Theory, Coding and Cryptography, McGraw Hill Education India Private Limited , 3rd Edition 2017