- Generalized Linear Models by Germán Rodríguez
- StatLect by Marco Taboga
- STAT 414 Introduction to Probability Theory from the Eberly College of Science
- STAT 415 Introduction to Mathematical Statistics from the Eberly College of Science
- Random: Probability, Mathematical Statistics, Stochastic Processes Kyle Siegrist (Uni. of Alabama)
- Introduction to Probability, Statistics and Random Processes by Hossein Pishro-Nik (2014; Kappa Research LLC)
- Probability: Theory and Examples by Rick Durrett (2019)
- Statistics 200: Introduction to Statistical Inference by Zhou Fan (Stanford University, Autumn 2016)
- Stats 202 · Stanford University
- Inference! An interactive introduction
- Notes on Probability* by Greg Lawler [PDF]
- Probability: Theory and Examples by Rick Durrett (2019) (also via Drive)
- STAT 400 by John Millson University of Maryland. See Handouts for Stat 400 and Stat 401
- Regression Modeling Strategies by Frank Harrell
- Probabilstic Graphical Models Notes (CS228)
- Advanced Statistical Computing
- Improving Your Statistical Inferences
Fundamentals
- Variance of the sum of two random variables - proof - better viewed on the website
- Additivity of the variance for independent random variables - proof
- Law of large numbers - Wikipedia - given a sample of independent and identically distributed values, the sample mean converges to the true mean
- Convergence of random variables - Wikipedia
- Almost surely - Wikipedia - the set of outcomes on which the event does not occur has probability 0, even though the set might not be empty…this distinction becomes important when the sample space is an infinite set, because an infinite set can have non-empty subsets of probability 0
- see also: Modes of convergence - Wikipedia
- Two Proofs of the Central Limit Theorem by Yuval Filmus — Toronto CS (January/February 2010)
- Berry–Esseen theorem - Wikipedia
- Intuitive explanation of Lyapunov condition for CLT -todo do not understand this at present
- Lebesgue measure - Wikipediatodo compartmentalise into a dedicated section on measure theory (and link to Statistical Learning Theory and Math)
Probability Theory
- ✨ Probability Theory by Michel Goemans — MIT Math 18.310 (February 10, 2015)
- inc. subsections on Expectation, Variance, Chebyshev’s Inequality, and Weak law of large numbers (proof via Chebyshev)
Probability Distributions
Resources:
- Probabilistic Building Blocks by Michael Betancourt (June 2019)
- ✨ Univariate distribution relationship chart - interactive and with link-throughs to short summary notes on each of the distributions
- Probability Distribution Explorer - from Justin Bois prepared at Caltech with financial support from the Donna and Benjamin M. Rosen Bioengineering Center
- Relationships among probability distributions - Wikipedia
- Distributome - interactive web-based resource for probability distributions
- The distribution zoo by Ben Lambert and Fergus Cooper
- Univariate Distribution Relationships (with J. McQueston), The American Statistician, Volume 62, Number 1, February 2008, 45-53
- 2021-03-17-the-probability-distributions
Distributions:
- Laplace distribution - Wikipedia - contains nice proof of the equivalence to the difference of two Exponential distributions via the multiplication of the characteristic functions (recall: Fourier transform of the PDF) of for
- Gaussian vs Laplace vs power-p Gaussian plot (Desmos toy; rough)
Concepts:
- Compound probability distribution - Wikipedia
- Characteristic function (probability theory) - Wikipedia
Gaussian Distribution
- How was the normal distribution derived?
- Derivation of Gaussian Distribution from Binomial
- Normal distribution - Wikipedia
Moment Generating Function
Bounds (Probabilistic Bounds and Inequalities)
- Inequalities of Markov and Chebyshev — Dartmouth Math 20
- Chernoff bounds, and some applications by Michel Goemans — MIT Math 18.310 (February 21, 2015)
- Probability - The Chernoff Bound - Ben Lynn - Stanford Crypto
- Chernoff Bounds - probabilitycourse.com - very minimal / simple
- Hoeffding’s inequality - Wikipedia
Bayesian Statistics & Stochastic Processes
- Scribe Notes from Bayesian Modeling and Inference (Stat260) by Michael I. Jordan
- A First Course in Bayesian Statistical Methods by Peter D. Hoff
- Monte Carlo Methods (Arizona Math 577-002 2016) by Tom Kennedy
- Computational Cognition Cheat Sheets from various authors at Robert Jacobs’ Computational Cognition and Perception Lab
- Bayesian Modeling and Computation in Python by Osvaldo A. Martin, Ravin Kumar, Junpeng Lao
- Applied Stochastic Analysis by Miranda Holmes-Cerfon
- De Finetti’s theorem - Wikipedia
Concepts (Bayesian Statistics & Stochastic Processes)
Mixed Effects (or Hierarchical or Multilevel) Models
- An Introduction to Hierarchical Modeling by Michael Freeman - This visual explanation introduces the statistical concept of Hierarchical Modeling, also known as Mixed Effects Modeling or by these other terms. This is an approach for modeling nested data. Keep reading to learn how to translate an understanding of your data into a hierarchical model specification.
- Thinking About Mixed Models (2016-05-05) by Michael Clark
- Mixed Model Estimation, a Connection to Additive Models, and Beyond… (2016-05-07) by Michael Clark
- Repeated Measures and Mixed Models by Michael Clark
- Learning Environment for Multilevel Methods and Applications
- Multilevel models: Random coefficient and intercept models from the National Centre for Research Methods (NCRM) (video playlist)
Statistics x Econometrics
Statistics x Econometrics » Structural Equation Modelling (SEM)
- Structural Equation Modelling (SEM) - video playlist from the National Centre for Research Methods (NCRM)