Course Outline
The course is divided into three distinct days, with the third day being optional.
Day 1 - Machine Learning & Deep Learning: Theoretical Concepts
1. Introduction to AI, Machine Learning & Deep Learning
- History, fundamental concepts, and typical applications of artificial intelligence, away from the fantasies associated with this field.
- Collective intelligence: aggregating knowledge shared by numerous virtual agents.
- Genetic algorithms: evolving a population of virtual agents through selection.
- Typical Machine Learning: definition.
- Types of tasks: supervised learning, unsupervised learning, reinforcement learning.
- Types of actions: classification, regression, clustering, density estimation, dimensionality reduction.
- Examples of Machine Learning algorithms: Linear Regression, Naive Bayes, Random Tree.
- Machine Learning VS Deep Learning: problems where Machine Learning remains the state of the art today (Random Forests & XGBoosts).
2. Fundamental Concepts of a Neural Network (Application: multi-layer perceptron)
- Review of mathematical basics.
- Definition of a neural network: classic architecture, activation functions, weighting of previous activations, network depth.
- Definition of neural network learning: cost functions, back-propagation, stochastic gradient descent, maximum likelihood.
- Modeling a neural network: modeling input and output data based on the problem type (regression, classification...). Curse of dimensionality. Distinction between multi-feature data and signal. Choosing a cost function based on data.
- Approximating a function with a neural network: presentation and examples.
- Approximating a distribution with a neural network: presentation and examples.
- Data Augmentation: how to balance a dataset.
- Generalization of neural network results.
- Neural network initializations and regularizations: L1/L2 regularization, Batch Normalization...
- Optimizations and convergence algorithms.
3. Typical ML/DL Tools
A simple presentation outlining advantages, disadvantages, position in the ecosystem, and usage is planned.
- Data management tools: Apache Spark, Apache Hadoop
- Typical Machine Learning tools: Numpy, Scipy, Sci-kit
- High-level DL frameworks: PyTorch, Keras, Lasagne
- Low-level DL frameworks: Theano, Torch, Caffe, Tensorflow
Day 2 - Convolutional and Recurrent Networks
4. Convolutional Neural Networks (CNN).
- Presentation of CNNs: fundamental principles and applications.
- Fundamental operation of a CNN: convolutional layer, kernel usage, padding & stride, feature map generation, pooling layers. 1D, 2D, and 3D extensions.
- Presentation of different CNN architectures that have pushed the state of the art in image classification: LeNet, VGG Networks, Network in Network, Inception, Resnet. Presentation of innovations introduced by each architecture and their broader applications (1x1 Convolution or residual connections).
- Use of an attention model.
- Application to a typical classification scenario (text or image).
- CNNs for generation: super-resolution, pixel-wise segmentation. Presentation of main feature map augmentation strategies for image generation.
5. Recurrent Neural Networks (RNN).
- Presentation of RNNs: fundamental principles and applications.
- Fundamental operation of the RNN: hidden activation, back propagation through time, unfolded version.
- Evolution towards GRUs (Gated Recurrent Units) and LSTMs (Long Short-Term Memory). Presentation of different states and advancements introduced by these architectures.
- Convergence problems and vanishing gradients.
- Types of classic architectures: Time series prediction, classification...
- RNN Encoder-Decoder architecture. Use of an attention model.
- NLP applications: word/character encoding, translation.
- Video applications: predicting the next image generated by a video sequence.
Day 3 - Generative Models and Reinforcement Learning
6. Generative Models: Variational AutoEncoder (VAE) and Generative Adversarial Networks (GAN).
- Presentation of generative models, link with CNNs covered on Day 2.
- Auto-encoder: dimensionality reduction and limited generation.
- Variational Auto-encoder: generative model and approximation of data distribution. Definition and use of the latent space. Reparameterization trick. Applications and observed limitations.
- Generative Adversarial Networks: fundamental principles. Two-network architecture (generator and discriminator) with alternating learning, available cost functions.
- GAN convergence and difficulties encountered.
- Improved convergence: Wasserstein GAN, BeGAN. Earth Mover's Distance.
- Applications in image or photo generation, text generation, super-
resolution.
7. Deep Reinforcement Learning.
- Presentation of reinforcement learning: controlling an agent in an environment defined by a state and possible actions.
- Use of a neural network to approximate the state function.
- Deep Q Learning: experience replay, and application to video game control.
- Optimization of the learning policy. On-policy && off-policy. Actor-critic architecture. A3C.
- Applications: control of a simple video game or a digital system.
Requirements
Engineer level
Testimonials (1)
The training was organized and well-planned out, and I come out of it with systematized knowledge and a good look at topics we looked at
