Syllabus

Introduction: Basics of statistical decision theory, learning theory to signal processing, information theory, and adaptive control. [5]

Stochastic Processes, Empirical Processes, Concentration and Deviation Inequalities, overview of supervised learning, problems in learning, estimation and optimization. [4]

Unsupervised Learning: Association Rules, Cluster Analysis, Self-Organizing Maps, Principal Components, Curves; [4]

Ensemble Learning: Boosting and Regularization Paths, Learning Ensembles; Undirected Graphical Models: Markov Graphs and Their Properties; [6]

Undirected Graphical Models for Continuous and discrete Variables, Estimation of the Parameters, Graph Structure, hidden nodes. [5]

Statistical Learning: Empirical Risk Minimization, Chaining for Sub-Gaussian Processes, Summarization, Combinatorial Dimensions, bounds; Sequential Prediction and Decision Making: [4]

Prediction with Expert Advice, Exponential Weights Algorithm, Sequential Minimax Theorem; Sequential Covering Numbers, Chaining, Dudley-type Bound; Combinatorial Dimensions, Learnability in Supervised Setting; [6]

From Sequential to Statistical Learning; Optimality of Mirror Descent; Stochastic and Deterministic Settings, Redundancy-Capacity Theorem; Prequential Statistics, Calibration of Forecasters, Testing, Algorithmic Stability, Aggregation of Estimators. [5]

Rigorous study on the statistical approaches in signal processing, estimation and detection techniques.

Information theoretic approaches in signal processing for system learning for machine intelligence.

Adaptive control, decision making and system stability analysis.

Students will able to know stochastic processes and their use in machine leanrning.

Students will able to formulate statistical learning methods for various practical applications.

Students will able to develop unsupervised learning tools applicable to detection, recognition, classification and data mining in signal, image and video.

Students will able to implement specialized techniques for prediction of events in nowcasting and forecasting.

Students will test industrial data and propose intelligent learning techniques for system control and engineering.

Kevin P. Murphy, Probabilistic Machine Learning: An Introduction, The MIT Press Cambridge, Massachusetts London, England, Edition 2022

Gareth James, Daniel Witten, Traevor Hastie and Robert Tibshirani, An introduction to Statistical Learning with Application in R, Springer, Second Edition, 2021.

John W. Pratt, Howard Raiffa and Robert Schlaifer, Introduction to Statistical Decision Theory, The MIT Press Cambridge, Massachusetts London, England, Edition 1995

Traevor Hastie, Robert Tibshirani and Jerome Friedman, The Elements of Statistical Learning: Data Mining, Inference, and Prediction, Springer, Second Edition, 2009.

1. Aad W. van der Vaart and Jon A. Wellner, Weak Convergence and Empirical Processes: With Applications to Statistics, Springer; 1st ed. 1996. Corr. 2nd printing 2000 edition (1 December 2000).

2. Sara A. van de Geer, Empirical Processes in M-Estimation, Cambridge University Press, 1st ed. 2000, Reprinted 2006.