Artificial Neural Networks, Machine Learning, Deep Thinking Training Course

DAY 1 - ARTIFICIAL NEURAL NETWORKS

Overview and Structural Frameworks of ANNs.

• The relationship between biological and artificial neurons.
• The structural model of an Artificial Neural Network.
• Common activation functions employed in ANNs.
• Standard categories of network architectures.

Mathematical Underpinnings and Learning Processes.

• A review of vector and matrix algebra.
• Concepts related to state spaces.
• Principles of optimization.
• Error-correction based learning.
• Memory-based learning approaches.
• Hebbian learning principles.
• Mechanisms of competitive learning.

Single-Layer Perceptrons.

• The architecture and training of perceptrons.
• Pattern classification, including an introduction to Bayes' classifiers.
• Utilizing the perceptron as a pattern classifier.
• The concept of perceptron convergence.
• Inherent limitations of perceptrons.

Feedforward Artificial Neural Networks.

• The structure of multi-layer feedforward networks.
• The backpropagation algorithm.
• Training dynamics and convergence in backpropagation.
• Functional approximation techniques using backpropagation.
• Practical considerations and design challenges in backpropagation learning.

Radial Basis Function (RBF) Networks.

• Issues regarding pattern separability and interpolation.
• Theoretical aspects of regularization.
• The application of regularization in RBF networks.
• Design and training methodologies for RBF networks.
• Approximation capabilities of RBFs.

Competitive Learning and Self-Organizing Artificial Neural Networks.

• Standard clustering procedures.
• Learning Vector Quantization (LVQ).
• Algorithms and architectures for competitive learning.
• The structure of self-organizing feature maps.
• Key properties of feature maps.

Fuzzy Neural Networks.

• The concept of neuro-fuzzy systems.
• Foundations of fuzzy sets and fuzzy logic.
• Design principles for fuzzy systems.
• Architectural design of fuzzy ANNs.

Practical Applications

• A discussion on selected examples of Neural Network applications, highlighting their benefits and associated challenges.

DAY 2 - MACHINE LEARNING

• The PAC Learning Framework
  • Guarantees for finite hypothesis sets in the consistent case
  • Guarantees for finite hypothesis sets in the inconsistent case
  • General considerations
    • Deterministic versus Stochastic scenarios
    • Bayes error noise
    • Estimation and approximation errors
    • Strategies for model selection
• Rademacher Complexity and VC Dimension
• The Bias-Variance tradeoff
• Regularization techniques
• Mitigating overfitting
• Model validation methods
• Support Vector Machines
• Kriging (Gaussian Process regression)
• Principal Component Analysis (PCA) and Kernel PCA
• Self-Organizing Maps (SOM)
• Kernel-induced vector spaces
  • Mercer Kernels and kernel-induced similarity metrics
• Reinforcement Learning

DAY 3 - DEEP LEARNING

This session builds upon the topics covered in Day 1 and Day 2.

• Logistic and Softmax Regression
• Sparse Autoencoders
• Vectorization, PCA, and Whitening
• Self-Taught Learning
• Deep Network Architectures
• Linear Decoders
• Convolution and Pooling operations
• Sparse Coding
• Independent Component Analysis
• Canonical Correlation Analysis
• Demonstrations and Real-World Applications