Syllabus

History of Deep Learning, Deep Learning Success Stories, McCulloch Pitts Neuron, Thresholding Logic, Perceptrons, Perceptron Learning Algorithm and Convergence, Multilayer Perceptrons (MLPs), Representation Power of MLPs, Sigmoid Neurons, Gradient Descent, Feedforward Neural Networks, Representation Power of Feedforward Neural Networks, Feedforward Neural Networks, Backpropagation.

Optimizers: Gradient Descent (GD), Momentum-Based GD, Nesterov Accelerated GD, Stochastic GD, AdaGrad, RMSProp, Adam, Learning Rate Scheduling.
Bias Variance Tradeoff, L2 regularization, Early stopping, Dataset augmentation, Parameter sharing and tying, Injecting noise at input, Ensemble methods, Dropout, Greedy Layerwise Pre-training, Activation Functions, Weight Initialization methods, Batch Normalization.

Convolutional Neural Networks (CNNs): Convolution for Images, Padding and Strides, Pooling, LeNet, Modern CNNs: AlexNet, VGGNet, GoogLeNet, ResNet, DenseNet, Visualizing Convolutional Neural Networks, Guided Backpropagation, Deep Dream, Deep Art, Fooling Convolutional Neural Networks, Transfer Learning, Early fusion, Late fusion, Intermediate fusion, Learning Vectorial Representations of Words.
Hyperparameter optimization: Grid Search, Random Search, Bayesian Optimization.

Recurrent Neural Networks(RNNs): Working with Sequences, Converting Raw Text to Sequence Data, Language Models, Backpropagation Through Time (BPTT), Vanishing and Exploding Gradients, Truncated BPTT, Long Short Term Memory (LSTM) Network, Gated Recurrent Units (GRUs), Deep RNNs, Solving the vanishing gradient problem with LSTMs, Bidirectional RNNs, Bidirectional Models (BiLSTM, BiGRU).
Encoder Decoder Models, Attention Mechanism, Attention over images, Hierarchical Attention, Attention Pooling by Similarity, Attention Scoring Functions, The Bahdanau Attention Mechanism, Multi-Head Attention, Self-Attention and Positional Encoding, The Transformer Architecture, Transformers for Vision, Large-Scale Pretraining with Transformers.

Understand the fundamentals of feedforward, recurrent and convolutional neural networks and apply them to solve various real-world problems.

Develop a deeper understanding in different optimization algorithms gradient descent, stochastic gradient descent and recent developments like ADAM and RMSprop.

Explore state-of-the-art architectures (CNN, LSTM, GRU, Bidirectional Models, and Transformer Models) and learn transfer learning with feature and decision level fusion.

Learn to improve the computational efficiency and performance of DL models for a specific problem by optimizing hyperparameters and employing GPUs

Develop in depth understanding of the key deep learning models and concepts.

Gain practical skills and theoretical knowledge necessary to apply deep learning techniques to a wide range of problems.

Improve the performance (tuning hyperparameters) and computational efficiency (using GPUs) of deep learning models for real-world problems.

Determining an appropriate deep learning model for an application.

A. Zhang, Z.C. Lipton, M. Li, A.J. Smola, Dive into Deep Learning, Cambridge University Press , Website: https://d2l.ai

Ian Goodfellow, Yoshua Bengio, Aaron Courville, Deep Learning, MIT Press , Website: https://www.deeplearningbook.org/

Charu C. Aggarwal , Neural Networks and Deep Learning, Springer

Francois Chollet, Deep Learning with Python, Manning Publishers

E. Stevens, L. Antiga, T. Viehmann, Deep Learning with PyTorch, Manning Publishers

IEEE Transactions on Neural Networks and Learning Systems

IEEE Transactions on Pattern Analysis and Machine Intelligence

AAAI Conference on Artificial Intelligence