A collection of some useful mathematics and computer science courses
Prof. Justin Johnson, University of Michigan, 2019
- Linear classifiers
- Stochastic gradient descent
- Fully-connected networks
- Convolutional networks
- Recurrent networks
- Attention and transformers
- Object detection
- Image segmentation
- Video classification
- Generative models (GANs, VAEs, autoregressive models)
- Reinforcement Learning
π₯ Lectures: https://youtube.com/playlist?list=PL5-TkQAfAZFbzxjBHtzdVCWE0Zbhomg7r
Roman Vershynin
π₯ Lectures: https://youtube.com/playlist?list=PLPjEEUWIWhQV7X6dXfrVP3w0KBBLBVJ0j
π Textbook : https://www.math.uci.edu/~rvershyn/papers/HDP-book/HDP-book.html
Prof. Pieter Abbeel, UC Berkeley, 2024
- Autoregressive Models
- Flow Models
- Latent Variable Models & Variational AutoEncoders (VAEs)
- Generative Adversarial Networks (GANs)
- Diffusion Models
- Self-Supervised Learning
- Large Language Models (LLMs)
- Generative Video
- Semisupervised Learning & Unsupervised Distribution Alignment
- Generative Modeling for Science
- Neural Radiance Fields
- Multimodal Models
- Parallelization
π₯ Lectures: https://youtube.com/playlist?list=PLwRJQ4m4UJjPIvv4kgBkvu_uygrV3ut_U
Prof. Stefano Ermon, Stanford University, 2023
- Autoregressive Models
- Maximum Likelihood Learning
- Variational AutoEncoders (VAEs)
- Normalizing Flows
- Generative Adversarial Networks (GANs)
- Energy Based Models (EBMs)
- Score Based Models
- Evaluation of Generative Models
π₯ Lectures: https://youtube.com/playlist?list=PLoROMvodv4rPOWA-omMM6STXaWW4FvJT8
π Course page containing lecture notes: https://deepgenerativemodels.github.io
African Masterβs in Machine Intelligence, 2022
- High-Dimensional Learning
- Geometric Priors
- Graphs & Sets
- Grids
- Groups
- Geodesics & Manifolds
- Gauges
π₯ Lectures: https://youtube.com/playlist?list=PLn2-dEmQeTfSLXW8yXP4q_Ii58wFdxb3C
π webpage : https://geometricdeeplearning.com
Prof. Jure Leskovec, Stanford University, 2021
π₯ Lectures: https://youtube.com/playlist?list=PLoROMvodv4rPLKxIpqhjhPgdQy7imNkDn
Prof. Csaba SzepesvΓ‘ri, University of Alberta, 2022
- MDP, Fundamental Theorem
- Value and Policy Iteration
- Local Planning
- Function Approximation
- Approximate Policy Iteration
- Planning Complexity, TensorPlan
- Lower Bound for API and POLITEX
- Policy Search
- Batch RL
- Online RL
π₯ Lectures: https://www.youtube.com/playlist?list=PLQCZ7_TRKVIzODPXorEyvhCk25TlcTANC
π webpage : https://rltheory.github.io/
π₯ Lectures: https://www.youtube.com/playlist?list=PLkoCa1tf0XjCU6GkAfRCkChOOSH6-JC_2
π Textbook : http://www.marl-book.com/
University of TΓΌbingen, 2023
- Numerical Linear Algebra
- Scaling Gaussian Processes
- Computation-aware Gaussian Processes
- State Space Models
- Solving Ordinary Differential Equations
- Probabilistic Numerical ODE Solvers
- Partial Differential Equations
- Monte Carlo
- Bayesian Quadrature
- Optimization for Deep Learning
- Second-order Optimization for Deep Learning
- Uncertainty in Deep Learning
π₯ Lectures : https://www.youtube.com/playlist?list=PL05umP7R6ij2lwDdj7IkuHoP9vHlEcH0s
Stanford, Prof. Emma Brunskill, 2024
- Introduction to Reinforcement Learning
- Tabular MDP planning
- Policy Evaluation
- Q learning and Function Approximation
- Policy Search
- Offline RL
- Exploration
- Multi-Agent Games
- Value Alignment
π₯ Lectures: https://www.youtube.com/playlist?list=PLoROMvodv4rN4wG6Nk6sNpTEbuOSosZdX
Purdue University, Prof. Charles A. Bouman, 2020
- Probability
- Causal Gaussian Models
- Non-Causal Gaussian Models
- MAP with Gaussian Priors
- Non-Gaussian Markov Random Fields
- Non-Gaussian MAP
- Majorization
- Constrained Optimization
- Plug and Play
- EM Algorithm
- Markov Chains and HMMs
- General MRFs
- Stochastic Simulation
- MAP Segmentation
π₯ Lectures : https://www.youtube.com/playlist?list=PL3ZrjaBngMS0mTSoMsy7P6rTFSgsmsMw3
π Textbook: https://engineering.purdue.edu/~bouman/publications/FCI-book/
Stanford, Prof. Stephen Boyd, 2023
- Convex sets
- Convex functions
- Convex optimization problems
- Duality
- Approximation and fitting
- Statistical estimation
- Geometric problems
- Numerical linear algebra background
- Unconstrained minimization
- Equality constrained minimization
- Interior-point methods
π₯ Lectures: https://www.youtube.com/playlist?list=PLoROMvodv4rMJqxxviPa4AmDClvcbHi6h
π Textbook : https://stanford.edu/~boyd/cvxbook/
University of Pennsylvania, Prof. Aaron Roth, 2023
A mathematical course focusing on the interplay between game theory and machine learning:
- Introduction to sequential learning
- Halving algorithm
- Follow the perturbed leader
- Follow the regularized leader
- Online convex optimization
- Zero-sum games, Minimax theorem
- Deriving a no regret learning algorithm
- Correlated equilibrium, Swap regret
- The adversary moves first framework
- Multi-objective sequential learning
π₯ YouTube (24 lectures) : https://www.youtube.com/playlist?list=PLlIlhe_rS4U2p_fxYwB15nhzHEKF53xUl
π Lecture notes: https://mlgametheory.com/
Princeton University, Prof. Chi Jin, 2024
A graduate level course on theoretical foundations of reinforcement learning:
- MDP basics and planning
- Concentration inequalities, Martingale concentrations
- Generative models, value iteration
- Online RL, exploration, optimism
- Minimax lower bound
- Offline RL, pessimism
- Policy optimization
- Large state space, linear function approximation
- General function approximation
- Game theory and multiagent RL
- Learning Markov games
- Partial observable MDP
π₯ YouTube (22 lectures) : https://www.youtube.com/playlist?list=PLYXvCE1En13epbogBmgafC_Yyyk9oQogl
π Course page containing lecture notes: https://sites.google.com/view/cjin/teaching/ece524
Harvard Center of Mathematical Sciences and Applications, Dr. Eli Grigsby, 2024
A course on geometric aspects of deep learning theory:
- The geometry and combinatorics of feedforward ReLU neural networks as piecewise linear function classes
- Neural networks as universal approximators: discrete and non-discrete versions
- The role of the superposition hypothesis in mechanistic interpretability of neural networks
- Neural network architectures for sequence-to-sequence processing
- Representing finite state automata using sequence-to-sequence architectures
- Geometric distortion in deep networks and the importance of residual connections
- Symmetries of overparameterized ReLU neural networks, optimization, and generalization
- Algorithmic computation of topological invariants of decision boundaries/regions
π₯ YouTube: https://youtube.com/playlist?list=PL0NRmB0fnLJSEXFQHGF0q5JcedxTqK4AJ&si=G0rk4GBgywt6kypK
π Course page : https://sites.google.com/bc.edu/eli-grigsby/mt875-mechanistic-interpretability
University of British Columbia, Dr. Frank Wood, 2021
- Introduction to Model-Based Reasoning
- Graphical Models
- Inference, Learning, Monte Carlo, Sampling
- Markov Chain Monte Carlo
- First Order Probabilistic Programming Languages
- Graphical Model Compilation
- Graph-Based Inference
- Hamiltonian Monte Carlo
- Evaluation-based Inference
- Variational Inference
- Higher Order Probabilistic Programming Languages
- Amortized Inference / Guide Programs / Inference Compilation
- Reinforcement Learning as Inference
- Alternative Variational Bounds
- Reparametrization and Normalizing Flows
π₯ 25 lectures on YouTube: https://youtube.com/playlist?list=PLRBUAK6di_6XlF7KAZBPRgcP0zD5sVXcN&si=9hjsRE1bav7vTqbG
π An Introduction to Probabilistic Programming: https://arxiv.org/abs/1809.10756
UCLA, Prof. Adnan Darwiche
- Propositional Logic
- Probability Calculus: Beliefs and Hard Evidence, Soft Evidence
- Bayesian Networks: Syntax and Semantics
- Bayesian Networks: Independence and d-Separation
- Probabilistic Queries and their Complexity
- Building Bayesian Networks
- Inference by Variable Elimination
- The Jointree Algorithm
- Inference by Conditioning
- Arithmetic Circuits
- Loopy Belief Propagation
- Learning Parameters
- Learning Network Structure
- Bayesian Learning
- Causality
- Sensitivity Analysis
- Reasoning about Classifiers
- Explaining Classifiers
π₯ YouTube Playlist(32 lectures + 4 additional lectures on causality): https://youtube.com/playlist?list=PLlDG_zCuBub6ywAIrM1DfJp8xaeVjyvwx
π Textbook: Modeling and Reasoning with Bayesian Networks, Adnan Darwiche
ENS Paris-Saclay, Dr. Julien Mairal, Dr. Jean-Philippe Vert
- Positive definite kernels
- Reproducing Kernel Hilbert Space
- Smoothness functional, Kernel trick, Representer theorem
- Kernel ridge and logistic regression
- Large-margin classifiers, SVMs
- Unsupervised kernel methods
- Green, Mercer, Herglotz, Bochner and friends
- Kernels for graphs
- Multiple kernels learning
- Large-scale learning
- Deep kernel machines
- Kernels for probabilistic models
- Kernel mean embedding
- Characteristic kernels
π₯ YouTube Playlist (25 lectures): https://www.youtube.com/playlist?list=PLD93kGj6_EdrkNj27AZMecbRlQ1SMkp_o
UC Berkeley, Prof. Pieter Abbeel, 2019
- Markov Decision Processes: Exact Methods
- Discretization of Continuous State Space MDPs
- Function Approximation
- LQR, iterative LQR, Differential Dynamic Programming
- Unconstrained Optimization
- Constrained Optimization
- Optimization-based Control
- Motion Planning
- Kalman Filtering, EKF, UKF
- Smoother, MAP, Maximum Likelihood, EM, KF parameter estimation
- Particle Filters
- Partially Observable MDPs
- Imitation Learning
- RL : Policy Gradients, Off-policy RL, Model-based RL
- Physics Simulation
π₯ YouTube Playlist (24 lectures) : https://youtube.com/playlist?list=PLwRJQ4m4UJjNBPJdt8WamRAt4XKc639wF&si=LrZXaiXafs6Qj07x
Carnegie Mellon University, Prof. Larry Wasserman, 2016
- Function Spaces
- Concentration of Measure
- Linear Regression
- Non-Parametric Regression
- Trend Filtering
- Linear Classification
- Non-Parametric Classification
- Minimax Theory
- Non-Parametric Bayes
- Boosting
- Clustering
- Graphical Models
- Dimension Reduction
- Random Matrix Theory
- Differential Privacy
π₯ YouTube Playlist (24 lectures) : https://youtube.com/playlist?list=PLTB9VQq8WiaCBK2XrtYn5t9uuPdsNm7YE&si=T5N31V-7ZPA_onXN
KIT(2020), Dr. Julius Pfrommer
- Introduction, Convexity and Gradient Descent
- Newtonβs Method
- Inequality Constrained Optimization
- Equality Constrained Optimization
- Applications: Mechanical Design, Model-Predictive Control, Optimization in Finance
- Automatic Differentiation and Neural Networks
- Vector Spaces, Norms and the Projection Theorem
- Fast First-Order Optimization
- Duality and Primal-Dual Algorithms
- SVM and the Reproducing Kernel Hilbert Space
- Conic Programming
- Alternating Methods and the EM Algorithm
- Applications: Graph Problems, Computer Vision and Generalized Low-Rank Models
- Gradient-Free and Non-Convex Optimization
π₯ Lectures on YouTube : https://youtube.com/playlist?list=PLdkTDauaUnQpzuOCZyUUZc0lxf4-PXNR5&si=x3fYVDBXH7Y4TAmY
TU Dortmund, Prof. Stefan Harmeling, 2022
π₯ Video lectures (28 sessions): https://youtube.com/playlist?list=PLzrCXlf6ypbzDYKDchKfM-I9s20mFCL0q&si=IuKihyN1QdWIuY8d
MIT, Dr. Chris Rackauckas, 2021
- Getting Started with Julia
- Optimizing Serial Code
- Physics-Informed Neural Networks
- Introduction to Discrete Dynamical Systems
- The Basics of Single Node Parallel Computing
- Styles of Parallelism
- Ordinary Differential Equations
- Forward-Mode Automatic Differentiation
- Solving Stiff Ordinary Differential Equations
- Basic Parameter Estimation, Reverse-Mode AD, and Inverse Problems
- Differentiable Programming and Neural Differential Equations
- MPI for Distributed Computing
- Mathematics of ML and HPC
- GPU Computing
- Partial Differential Equations and Convolutional Neural Networks
- Probabilistic Programming
- Global Sensitivity Analysis
- Code Profiling and Optimization
- Uncertainty Programming and Generalized Uncertainty Quantification
π₯ Video Lectures: https://youtube.com/playlist?list=PLCAl7tjCwWyGjdzOOnlbGnVNZk0kB8VSa&si=-5MJhyhshyQ1SpcQ
π Lecture notes as an online book: https://book.sciml.ai/
University of Cambridge, Dr. Sean Holden
YouTube Playlist(15 lectures): https://youtube.com/playlist?list=PLdLk2RYEiAhp9Slj6F_LCMXUv7_Fi3V_Y&si=E-A3Igj-C3xrQJU2
Iowa State University (2017), Prof. Steve Butler
π₯ Lectures (32 Sessions): https://www.youtube.com/playlist?list=PLi4h0n4UP8d9VGPqO8vLQga9ZApO65TLW
π Textbook: An Introduction to the Theory of Graph Spectra
Stanford, Prof. Tim Roughgarden
- Mechanism Design Basics
- Myerson's Lemma
- Algorithmic Mechanism Design
- Revenue-Maximizing Auctions
- Simple Near-Optimal Auctions
- VCG Mechanism
- Spectrum Auctions
- Beyond Quasi-Linearity
- Kidney Exchange, Stable Matching
- Selfish Routing and the POA
- Network Over-Provisioning
- Hierarchy of Equilibrium Concepts
- Smooth Games
- Best-Case and Strong Nash Equilibria
- Best-Response Dynamics
- No-Regret Dynamics
- Swap Regret; Minimax
- Pure NE and PLS-Completeness
- Mixed NE and PPAD-Completeness
π₯ Lectures: https://youtube.com/playlist?list=PLEGCF-WLh2RJBqmxvZ0_ie-mleCFhi2N4&si=7r52R_RF8miNr_N2
Stanford, Prof. Tim Roughgarden
- Ascending and Ex Post Incentive Compatible Mechanisms
- Unit-Demand Bidders and Walrasian Equilibria
- The Crawford-Knoer Auction
- The Clinching Auction
- The Gross Substitutes Condition
- Gross Substitutes-Welfare Maximization in Polynomial Time
- Submodular Valuations
- MIR and MIDR Mechanisms
- MIDR Mechanisms via Scaling Algorithms
- Coverage Valuations and Convex Rounding
- Undominated Implementations and the Shrinking Auction
- Bayesian Incentive-Compatibility
- Black Box Reductions
- The Price of Anarchy in Simple Auctions
- The Price of Anarchy of Bayes-Nash Equilibria
- The Price of Anarchy in First-Price Auctions
- Demand Reduction in Multi-Unit Auctions Revisited
- Beyond Smoothness and XOS Valuations
- Multi-Parameter Revenue-Maximization
- Interim Rules and Borderβs Theorem
- Characterization of Revenue-Maximizing Auctions
π₯ Lectures: https://youtube.com/playlist?list=PLEGCF-WLh2RI77PL4gwLld_OU9Zh3TCX9
Prof. Thomas Kesselheim
- Online Algorithms
- Online Learning Algorithms and Online Convex Optimization
- Markov Decision Processes
- Stochastic and Robust Optimization
π₯ Lectures: https://www.youtube.com/playlist?list=PLyzcvvgje7aDZRFMJZgaVgOW5t5KLvD1-
University of California, Prof. Melvin Leok, San Diego, 2022
π₯ Lectures: https://www.youtube.com/playlist?list=PLHZhjPByiV3L94AeJ9FcK1yrnRDOt3Vit
The University of Iceland, Prof. Ing Morris Riedel
High Performance Computing
- Parallel Programming with MPI
- Parallelization Fundamentals
- Advanced MPI Techniques
- Parallel Algorithms & Data Structures
- Parallel Programming with OpenMP
- Hybrid Programming & Patterns
- Debugging & Profiling & Performance Analysis
- Accelerators & Graphical Processing Units
- Parallel & Scalable Machine & Deep Learning
- HPC in Health & Neurosciences
- Computational Fluid Dynamics & Finite Elements
- Systems Biology & Bioinformatics
- Molecular Systems & Material Sciences
- Terrestrial Systems & Climate
π₯ 2024 Lectures (ongoing): https://www.youtube.com/playlist?list=PLmJwSK7qduwVAnNfpueCgQqfchcSIEMg9
π₯ 2023 Lectures: https://www.youtube.com/playlist?list=PLmJwSK7qduwUBwrFn3SY8vi4AYa2rVTWH
π₯ 2022 Lectures: https://www.youtube.com/playlist?list=PLmJwSK7qduwWyqcSEB45HOyxq--z8njix
ETH ZΓΌrich, Prof. Siddhartha Mishra, Dr. Benjamin Moseley, 2023
- Introduction to Deep Learning
- Physics-Informed Neural Networks
- Operator Learning
- Neural Operators
- Fourier Neural Operators and Convolutional Neural Operators
- Differentiable Physics
π₯ Course lectures: https://www.youtube.com/playlist?list=PLJkYEExhe7rYY5HjpIJbgo-tDZ3bIAqAm
Prof. Tadashi Tokieda
π₯ Lectures: https://www.youtube.com/playlist?list=PLTBqohhFNBE_09L0i-lf3fYXF5woAbrzJ
UC Berkeley, Prof. Sergey Levine
In addition to the standard RL topics, the course also includes:
- RL and language models
- Offline RL
- Inverse RL
- RL as probabilistic inference
- Uncertainty and RL
- Transfer learning and meta learning
π₯ Lectures(2021-2023): https://www.youtube.com/playlist?list=PL_iWQOsE6TfVYGEGiAOMaOzzv41Jfm_Ps
Harvard, Prof. Gregory Falkovich, 2022
π₯ Lectures: https://www.youtube.com/playlist?list=PLDEN2FPNHwVZKAFqfFl1b_NNAESTJwV9o
π Textbook (Physical Nature of Information): https://www.weizmann.ac.il/complex/falkovich/sites/complex.falkovich/files/uploads/PNI22.pdf
Virginia Tech, Prof. Scotland Leman, 2023
- Philosophy: What is probability?
- Fisher vs Neyman vs Jeffreys.
- The Likelihood Principle
- Basic Bayesian constructions: Likelihoods, priors and posteriors
- Exponential families and conjugate priors
- Asymptotics, Bayesian t-tests, mixture models, hierarchical modeling, etc..
- Bayesian sequential updating
- More on priors: Jeffreys, Reference, Objective, Subjective, etc...
- Simulation procedures: Gibbs, Metropolis, etc...
- Model Selection: Theory and Computational Approaches
π₯ Video lectures for the 2023 course and also lectures for the past semesters: https://www.youtube.com/@lemanlectures8611/videos
π₯ First lecture: https://youtu.be/vHAoj0Q5Auw?si=68ymPihUCaAmvvgK
Saarland University, Prof. Roland Speicher, 2023
π₯ Recorded videos (29 lectures): https://youtube.com/playlist?list=PLY11JnnnTUCabY4nc0hKptrd5qEWtLoo2&si=9HLbybgfW6pBss88
University of Utah, Prof. Bei Wang, 2021
- Basic concepts (graphs, connected components, topological space, manifold, point cloud samples)
- Combinatorial structures on point cloud data (simplicial complexes)
- New techniques in dimension reduction (circular coordinates, etc.)
- Clustering (topology-based data partition, classification)
- Homology and persistent homology
- Topological signatures for classification
- Structural inference and reconstruction from data
- Topological algorithms for massive data
- Deep learning with TDA
- Multivariate and high-dimensional data analysis
- Topological data analysis for visualization (vector fields, topological structures)
- Practical applications of TDA
π₯ Playlist on YouTube (28 Lectures) : https://youtube.com/playlist?list=PLDZ6LA16SDbIvbgmCjcCuTA7mttfXjiec&si=FiadJKIdmKlJUIY7
Prof. Brittany Hamfeldt
π₯ Video Lectures: https://youtube.com/playlist?list=PLJ6garKOlK2qKVhRm6UwvcQ46wK-ciHbl&si=zeG5RCK_E04SRNww
Prof. Richard Borcherds
π₯ Lectures: https://www.youtube.com/@richarde.borcherds7998/playlists
Politecnico di Milano 2022
π₯ Lectures: https://youtube.com/playlist?list=PLvVaDdaHGtpesn2DHUo6ete-1pPhT1xzj&si=24WgTbFLChWMaJRx
King's College London, Dr Pierpaolo Vivo
π₯ Lectures : https://www.youtube.com/playlist?list=PLyHAvCibkccQEFYXdM6r8WG4GQULRKmRA
Colorado State University, Henry Adams, 2021
π₯ Videos (27 short lectures) : https://www.math.colostate.edu/~adams/teaching/dsci475spr2021/
MIT, Prof. Alan Edelman, Prof. Steven G. Johnson, 2023
π₯ YouTube (8 lectures): https://youtube.com/playlist?list=PLUl4u3cNGP62EaLLH92E_VCN4izBKK6OE&si=rNoLocGXOEXBQjMH
University of TΓΌbingen, Dr. Philipp Hennig, 2023
- Reasoning Under Uncertainty
- Continuous Random Variables
- Exponential Families
- Gaussian Probability Distributions
- Parametric Regression
- Gaussian Processes
- Understanding Gaussian Processes
- GP Regression
- Understanding Kernels and Gaussian Processes
- The role of Linear Algebra in Gaussian Processes
- Computation and Inference
- Logistic Regression
- GP Classification
- Deep Learning
- Probabilistic Deep Learning
- Uncertainty in Deep Learning
- Uses of Uncertainty for Deep Learning
- Gauss-Markov Models
- Parameter Inference
- Variational Inference
π₯ Lectures (25 lectures): https://youtube.com/playlist?list=PL05umP7R6ij2YE8rRJSb-olDNbntAQ_Bx&si=qivnfDBYjFOu1TOk
π Slides: https://github.com/philipphennig/Probabilistic_ML
Carnegie Mellon Universit
π₯ Lectures : https://www.youtube.com/playlist?list=PL9_jI1bdZmz0hIrNCMQW1YmZysAiIYSSS
MIT, Prof. Justin Solomon, 2023
π₯ Lectures : https://www.youtube.com/watch?v=Xt4p5gk24ss
MIT, Prof. Justin Solomon
π₯ Lectures : https://www.youtube.com/watch?v=VjyBp6PrvB4