Course Details
Subject {L-T-P / C} : EE6334 : Stochastic Control Theory { 3-0-0 / 3}
Subject Nature : Theory
Coordinator : Prof. Asim Kumar Naskar
Syllabus
Basic issues in optimization stochastic vs. deterministic methods Random search, Recursive methods: LMS, RLS, Kalman filter, Stochastic approximation (SA), Stochastic gradient methods, Finite-difference SA (FDSA), Simultaneous perturbation SA (SPSA), Markov chains, Markov Decision Process (MDP), Dynamic Programming method, Design of approximate controllers for MDPs, Stochastic linear quadratic control, Finite horizon control.
Course Objectives
- Students will be competent in critical questioning and analysis of the mathematical tools used in stochastic process and control.
Course Outcomes
At the end of the course, students will be able <br /> (I) To understand basic principles of stochastic optimization theory, probability theory and stochastic dynamical systems including Markov chains. <br />(II) To formulate stochastic control problems as Markov Decision Process (MDP) problems. <br />(III) To solve linear stochastic control problems.
Essential Reading
- K. J. Astrom, Introduction to Stochastic Control Theory, Dover , 2004
- D. Bertsekas, Dynamic programming and Optimal Control, Athena Scientific , 2007
Supplementary Reading
- P. R. Kumar and P. Varaiya, Stochastic Systems: Estimation, Identification and Adaptive Control, SIAM , 2016
- H. Kwakernaak and R. Sivan, Linear Optimal Control, John Wiley , 1972