Data Analytics, Statistical Learning, and Engineering Statistics

Statistical Learning Slides

Here is a set of slides made for a PhD-level introduction to Modern Multivariate Statistical Learning.

  1. Introduction–StatLearningModule1
  2. Supervised Learning Generalities: Decision Theory, Variance-Bias Trade-off, Model Flexibility and Fitting, and Cross-Validation–StatLearningModule2
  3. The Gram-Schmidt Process and the Singular Value Decomposition–StatLearningModule3
  4. Basic and Kernel Principal Components–StatLearningModule4
  5. Non-OLS Linear Predictors Part 1: Ridge Regression, the Lasso, etc.–StatLearningModule5
  6. Non-OLS Linear Predictors Part 2: Least Angle Regression–StatLearningModule6
  7. Non-OLS Linear Predictors Part 3: PC and PLS Regression–StatLearningModule7
  8. Linear Prediction Using Basis Functions Part 1: Wavelet Bases–StatLearningModule8
  9. Linear Prediction Using Basis Functions Part 2: Piecewise Polynomials and Regression Splines–StatLearningModule9
  10. Linear Prediction Using Basis Functions Part 3: Tensor Product Bases and MARS–StatLearningModule10
  11. Smoothing Splines in 1-D–StatLearningModule11
  12. Penalized Fitting to N Responses: Smoothing Splines and Generalizations–StatLearningModule12
  13. Multi-Dimensional Smoothing Splines–StatLearningModule13
  14. Kernel Smoothing Methods–StatLearningModule14
  15. High-Dimensional Use of Low-Dimensional  Smoothers–StatLearningModule15
  16. Highly Flexible Non-Linear Parametric Regression Methods: Neural and Radial Basis Function Networks–StatLearningModule16
  17. Trees and Related Methods of Prediction Part 1: CART (Classification and Regression Trees)–StatLearningModule17
  18. Trees and Related Methods of Prediction Part 2: Random Forests–StatLearningModule18
  19. Trees and Related Methods of Prediction Part 3: PRIM–StatLearningModule19
  20. Predictors Built on Bootstrap Samples: Bagging and Bumping–StatLearningModule20
  21. Combining Predictors: Model Averaging, Stacking, and SEL Boosting–StatLearningModule21
  22. RKHSs and Penalized/Regularized Fitting Part 1: RKHSs and Cubic Smoothing Splines–StatLearningModule22
  23. RKHSs and Penalized/Regularized Fitting Part 2: p=1 Development of Reproducing Kernels from Functionals and Differential Operators–StatLearningModule23
  24. RKHSs and Penalized/Regularized Fitting Part 3: Beginning From a Kernel–StatLearningModule24
  25. RKHSs and Penalized/Regularized Fitting Part 4: Gaussian Spatial Processes and Prediction–StatLearningModule25
  26. Understanding and Predicting Predictor Performance Part 1: Cp, AIC, and BIC–StatLearningModule26
  27. Understanding and Predicting Predictor Performance Part 2: Cross-Validation and Bootstrap Estimation of Err–StatLearningModule27
  28. Generalities About Classification–StatLearningModule28
  29. Linear Methods of Classification Part 1: Linear and Quadratic Discriminant Analysis–StatLearningModule29
  30. Linear Methods of Classification Part 2: Logistic Regression and Separating Hyperplanes–StatLearningModule30
  31. Support Vector Machines Part 1: Linearly Separable Cases–StatLearningModule31
  32. Support Vector Machines Part 2: Linearly Non-Separable Cases–StatLearningModule32
  33. Support Vector Machines Part 3: Kernels and SVMs–StatLearningModule33
  34. Support Vector Machines Part 4: Other SVM-Related Issues–StatLearningModule34
  35. Boosting Part 1: The AdaBoost.M1 Algorithm–StatLearningModule35
  36. Boosting Part 2: Other Forms of Boosting and Some Issues–StatLearningModule36
  37. Prototype and Nearest Neighbor Methods of Classification–StatLearningModule37
  38. Association Rules/Market Basket Analysis–StatLearningModule38
  39. Clustering Part 1: Generalities, Partitioning Methods, and Hierarchical Methods–StatLearningModule39
  40. Clustering Part 2: Model-Based Clustering and Self-Organizing Maps–StatLearningModule40
  41. Multi-Dimensional Scaling–StatLearningModule41
  42. Clustering (Graphical) “Spectral” Features–StatLearningModule42
  43. Variations on Principal Components: Sparse PC’s, Non-Negative Matrix Factorization, and Archetypal Analysis–StatLearningModule43
  44. Independent Component Analysis–StatLearningModule44
  45. Density Estimation–StatLearningModule45
  46. Google™ PageRanks–StatLearningModule46
  47. Document Features and “String Kernels” for Text Processing–StatLearningModule47
  48. Kernel Mechanics–StatLearningModule48
  49. Undirected Graphical Models and Machine Learning–StatLearningModule49
  50. “Relevance Vector” Methods/Bayesian Parameter “Sparsity”–StatLearningModule50

The materials on this site may be used without charge for non-commercial personal educational purposes.  Any copies made of the materials must bear printed acknowledgment of their Analytics Iowa LLC source.