Resources on statistics and computer science.
Machine Learning // Kernel Methods and Support Vector Machines // Graphs // Neural Networks // Optimization // Natural Language Processing // Information Theory and Related // Signal Processing // Bayesian Statistics (inc. Stochastic Processes) // Causal Inference // Statistics and Probability // Data Mining and Information Retrieval // Cryptography // Sapienza // Other Reading Lists // Other Lists of Resources
Machine Learning
- The Elements of Statistical Learning by Trevor Hastie, Robert Tibshirani and Jerome Friedman
- CS229: Machine Learning by Tengyu Ma, Andrew Ng and Chris Ré
- Theoretical Machine Learning by Rob Schapire
- A Course in Machine Learning by Hal Daumé III
- Unsupervised Feature Learning and Deep Learning Tutorial
- Oxford Machine Learning by Nando de Freitas
- An Introduction to Statistical Learning by Gareth James, Daniela Witten, Trevor Hastie and Robert Tibshirani
- Statistical Learning with Sparsity by Trevor Hastie, Robert Tibshirani and Martin Wainwright
- Pattern Recognition and Machine Learning by Christopher Bishop
- Foundations of Machine Learning by Mehryar Mohri, Afshin Rostamizadeh and Ameet Talwalkar
- Understanding Machine Learning: From Theory to Algorithms by Shai Shalev-Shwartz and Shai Ben-David
- Statistical Modeling and Analysis of Neural Data (Spring 2018) by Jonathan Pillow
Kernel Methods and Support Vector Machines
- Reproducing kernel Hilbert spaces in Machine Learning from Arthur Gretton
- Kernel Methods from A Course in Machine Learning by Hal Daumé III
- Notes on Kernel Methods and Support Vector Machines from CS229 (Fall 2020) by Andrew Ng
- Implicit Lifting and the Kernel Trick blog post by Gregory Gundersen
- Notes on Support Vector Machines from CS 511 Theoretical Machine Learning by Rob Schapire
- A Tutorial on Support Vector Machines for Pattern Recognition by Chris J.C. Burges [PDF]
Graphs
- CS224W: Machine Learning with Graphs taught by Jurij Leskovec
- Network Analysis and Modeling CSCI 5352 (Fall 2017) by Aaron Clauset
- Network Science by Albert-László Barabási
- Graph Representation Learning by William L. Hamilton
Neural Networks
- Deep Learning by Ian Goodfellow and Yoshua Bengio and Aaron Courville
- CS231n: Convolutional Neural Networks for Visual Recognition
- Yann LeCun’s Deep Learning Course at CDS [Home]
- Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gauges by Michael M. Bronstein, Joan Bruna, Taco Cohen, Petar Veličković
- Dive into Deep Learning
- DeepMind x UCL | Deep Learning Lecture Series 2020
- Neural Networks and Deep Learning by Michael Nielsen
- Christopher Olah’s Posts on Neural Networks
- Reinforcement Learning: An Introduction by Richard S. Sutton and Andrew G. Barto
- Algorithms of Reinforcement Learning by Csaba Szepesvári
- Machine Learning with PyTorch and Scikit-Learn by Sebastian Raschka, Yuxi Hayden Liu and Vahid Mirjalili (includes sections on Transformers, GANs, GCNs and RL)
- labml.ai Annotated PyTorch Paper Implementations - Multi-Headed Attention, Transformer Encoder and Decoder Models, Denoising Diffusion Probabilistic Models, Wasserstein GAN
Optimization
- Convex Optimization by Ryan Tibshirani
- Convex Optimization by Stephen Boyd and Lieven Vandenberghe
- Foundations for Optimization and Optimization by Mark Walker
- Short Lectures on Optimization by Michel Bierlaire
Natural Language Processing
- CS224n: Natural Language Processing with Deep Learning
- A Primer on Neural Network Models for Natural Language Processing by Yoav Goldberg
- Neural Network Methods for Natural Language Processing by Yoav Goldberg
- Natural Language Processing by Jacob Eisenstein [PDF]
- The Annotated Transformer by Alexander Rush
- Speech and Language Processing by Dan Jurafsky and James H. Martin
Information Theory and Related
- Information Theory and Statistics by John Duchi (Notes)
- Statistical Mechanics by Matthew D. Schwartz
- Information Theory andNetwork Coding by Raymond W. Yeung
- A First Course in Information Theory by Raymond W. Yeung
- Information, Physics, and Computation by Marc Mézard and Andrea Montanari
- Small Summaries for Big Data by Graham Cormode and Ke Yi
Signal Processing
- The Fourier Transform and its Applications (Lecture Notes EE 261) by Brad Osgood
- Signal Processing for Communications by Paolo Prandoni and Martin Vetterli
Bayesian Statistics (inc. Stochastic Processes)
- Scribe Notes from Bayesian Modeling and Inference (Stat260) by Michael I. Jordan
- A First Course in Bayesian Statistical Methods by Peter D. Hoff
- Applied Stochastic Analysis by Miranda Holmes-Cerfon
- Monte Carlo Methods (Arizona Math 577-002 2016) by Tom Kennedy
- Computational Cognition Cheat Sheets from various authors at Robert Jacobs’ Computational Cognition and Perception Lab
- Bayesian Modeling and Computation in Python by Osvaldo A. Martin, Ravin Kumar, Junpeng Lao
Causal Inference
- Introduction to Causal Inference by Brady Neal
- Causal Inference: What If by Miguel A. Hernán, James M. Robins
Statistics and Probability
- Generalized Linear Models by Germán Rodríguez
- StatLect by Marco Taboga
- STAT 414 Introduction to Probability Theory from the Eberly College of Science
- STAT 415 Introduction to Mathematical Statistics from the Eberly College of Science
- Random: Probability, Mathematical Statistics, Stochastic Processes Kyle Siegrist (Uni. of Alabama)
- Introduction to Probability, Statistics and Random Processes by Hossein Pishro-Nik (2014; Kappa Research LLC)
- Probability: Theory and Examples by Rick Durrett (2019)
- Statistics 200: Introduction to Statistical Inference by Zhou Fan (Stanford University, Autumn 2016)
- Inference! An interactive introduction (author unknown)
- Notes on Probability* by Greg Lawler [PDF]
- Probability: Theory and Examples by Rick Durrett (2019) (also via Drive)
- STAT 400 by John Millson University of Maryland. See Handouts for Stat 400 and Stat 401
Data Mining and Information Retrieval
- Mining of Massive Datasets by Jure Leskovec, Anand Rajaraman and Jeff Ullman [Book]
- Introduction to Information Retrieval by Christopher D. Manning, Prabhakar Raghavan and Hinrich Schütze
Cryptography
- Handbook of Applied Cryptography by Alfred J. Menezes, Paul C. van Oorschot and Scott A. Vanstone
Other Reading Lists
- Annotated Bibliography of Recommended Materials from the Center for Human-Compatible AI
- AI Safety Syllabus Reading List from 80,000 Hours