Course Details

Subject {L-T-P / C} : EC6514 : Optimization Methods in Signal Processing for Communication Systems { 3-0-0 / 3}

Subject Nature : Theory

Coordinator : Dr. Pankaj Kumar Sharma

Syllabus

Module 1: Introduction to Linear Algebra Concepts (8 hours):

Field and subfield, System of linear equations, Linear combination, Matrices, Elementary row operations, Row-reduced matrix, Echelon matrix, Vector spaces, Linear combination of vectors, Vector subspaces, Span, Linear independence, basis and dimension, Coordinates, Matrix multiplication, Inverse of a matrix, Four fundamental subspaces of a matrix: Column space, Null space, Row space, Left null space, Solution to system of linear equations, Augmented matrix, Rank of a matrix, Length of a vector (Euclidean norm), Inner product, Cosines and projections onto lines, Cauchy-Schwarz inequality, Orthogonal vectors and subspaces,
Linear transformations, Fundamental matrix transformations: Stretching, Rotation, Reflection, Projection, Determinants, Eigen values and eigen vectors, Diagonalization of a matrix, Hermitian matrix and its properties, Quadratic form, Positive definite matrices, Test for positive definiteness, Singular value decomposition (SVD).

Module 2: Fundamentals of Geometry and Calculus (4 hours):

Geometrical Concepts: Lines and line segments, Affine sets and affine hull, Convex sets and convex hull, Cone and conic hull, Hyperplanes and halfspaces, Neighbourhood (Euclidean ball) and ellipsoid, Interior point, Boundary point, Open and closed sets, Boundary set, Compact set, Polyhedra, Separating and supporting hyperplanes, Basics of Calculus: Sequences and limits, Affine functions, Differentiability, Derivative matrix, Hessian, Level sets and gradients, Graph, Taylor's series, Mean value theorem.

Module 3: Introduction to Optimization (5 hours):

Basics of an optimization problem, Conditions for local minimizers: First order necessary condition (FONC), Second order necessary condition (SONC), Second order sufficient condition (SOSC).

Unconstrained Optimization Algorithms: One dimensional search algorithms: Exhaustive search, Golden section method, Fibonacci method, Bisection method, Newton's method, Secant method, Bracketing, Gradient methods: Gradient descent algorithm, Steepest descent method, Newton's method (revisited): Levenberg-Marquardt modification, Conjugate direction and gradient algorithms.

Module 4: Convex Optimization-I (9 hours):

Convex and quasi-convex functions, Convexity preserving operations, Basic structure of convex optimization problems, Equivalent representations and transforms, Convex problems with inequality constraints.

Linear Programming: Standard form of linear program (LP), Transformation to standard form using surplus and slack variables, Geometry of LP, Basic solutions, Fundamental theorem of LP, Graphical solution, Simplex method: Canonical augmented matrix, Updating procedure for augmented matrix, Simplex algorithm, Matrix form of Simplex method, Two-phase Simplex method, Dual LP, Non-simplex methods, Integer linear programming.

Introduction to Geometric Programming and Quadratic Programming.

Module 5: Convex Optimization-II (9 hours):

Second-order cone programming (SOCP), Semidefinite Programming (SDP): QCQP and SOCP as SDP via Schur complement, S-procedure.

Duality: Lagrange dual function and conjugate function, Lagrange dual problems, Strong duality: Slater's condition, S-Lemma, Karush-Kuhn-Tucker (KKT) optimality conditions, Lagrange dual optimization, Alternating direction method of multipliers (ADMM), Duality of problems with generalized inequalities, Theorems of alternatives.

Interior-point Methods: Inequality and equality constrained convex problems, Newton's method and barrier function, Central path, Barrier method, Primal-dual interior point method.

Course Objectives

Course Outcomes

After the completion of this course, students will be able to: <br /> <br />CO1: apply the concepts of linear algebra for modeling research problems in the field of communications and signal processing. <br /> <br />CO2: identify the appropriate convex optimization problem for modeling typical research problems in the field of communications and signal processing. <br /> <br />CO3: develop algorithms for modern wireless communications and networking, e.g., optimal resource allocation, energy-efficiency maximization, sum-rate maximization, optimal beamforming, etc. <br /> <br />CO4: model and analyze the research problems for 5G and beyond wireless networks, e.g., massive MIMO networks, mmWave networks, energy harvesting networks, UAV networks, etc. <br /> <br />CO5: apply the optimization theory for typical signal processing applications, e.g., blind source separation for biomedical and hyperspectral image analysis, filter design, etc.

Essential Reading

Supplementary Reading