DBM4510: Machine Learning

Introduction to Machine Learning

• Welcome
• Scope, Terminology, Prediction, and Data
• Putting the Machine in Machine Learning
• Examples of Learning Systems
• Evaluating Learning Systems
• A Process for Building Learning Systems
• Assumptions and Reality of Learning
• End-of-Introduction
• About Our Setup
• The Need for Mathematical Language
• Our Software for Tackling Machine Learning
• Probability
• Linear Combinations, Weighted Sums, and Dot Products
• A Geometric View: Points in Space
• Notation and the Plus-One Trick
• Getting Groovy, Breaking the Straight-Jacket, and Nonlinearity
• NumPy versus “All the Maths”
• Floating-Point Issues
• End-of-Technical Background

Getting Started with Classifications and Regression

• Classification Tasks
• A Simple Classification Dataset
• Training and Testing: Don’t Teach to the Test
• Evaluation: Grading the Exam
• Simple Classifier #1: Nearest Neighbors, Long Distance Relationships, and Assumptions
• Simple Classifier #2: Naive Bayes, Probability, and Broken Promises
• Simplistic Evaluation of Classifiers
• End-of-Classifications
• A Simple Regression Dataset
• Nearest-Neighbors Regression and Summary Statistics
• Linear Regression and Errors
• Optimization: Picking the Best Answer
• Simple Evaluation and Comparison of Regressors
• End-of-Regression

Evaluating and Comparing Learners and Classifiers

• Evaluation and Why Less Is More
• Terminology for Learning Phases
• Major Tom, There’s Something Wrong: Overfitting and Underfitting
• From Errors to Costs
• (Re)Sampling: Making More from Less
• Break-It-Down: Deconstructing Error into Bias and Variance
• Graphical Evaluation and Comparison
• Comparing Learners with Cross-Validation
• End-of-Evaluating and Comparing Learners
• Baseline Classifiers
• Beyond Accuracy: Metrics for Classification
• ROC Curves
• Another Take on Multiclass: One-versus-One
• Precision-Recall Curves
• Cumulative Response and Lift Curves
• More Sophisticated Evaluation of Classifiers: Take Two
• End-of-Evaluating Classifiers

Evaluating Regressors, More Classification and Regression Methods

• Baseline Regressors
• Additional Measures for Regression
• Residual Plots
• A First Look at Standardization
• Evaluating Regressors in a More Sophisticated Way: Take Two
• End-of-Evaluating Regressors
• Revisiting Classification
• Decision Trees
• Support Vector Classifiers
• Logistic Regression
• Discriminant Analysis
• Assumptions, Biases, and Classifiers
• Comparison of Classifiers: Take Three
• End-of-Classification Methods
• Linear Regression in the Penalty Box: Regularization
• Support Vector Regression
• Piecewise Constant Regression
• Regression Trees
• Comparison of Regressors: Take Three
• End-of-Regression Methods

Manual Feature Engineering, Tuning Hyperparamete...g Learnings, and Feature Engineering for Domains

• Feature Engineering Terminology and Motivation
• Feature Selection and Data Reduction: Taking out the Trash
• Feature Scaling
• Discretization
• Categorical Coding
• Relationships and Interactions
• Target Manipulations
• End-of-Manual Feature Engineering
• Models, Parameters, Hyperparameters
• Tuning Hyperparameters
• Down the Recursive Rabbit Hole: Nested Cross-Validation
• Pipelines
• Pipelines and Tuning Together
• End-of-Tuning Hyperparameters and Pipelines
• Ensembles
• Voting Ensembles
• Bagging and Random Forests
• Boosting
• Comparing the Tree-Ensemble Methods
• End-of-Combining Learnings
• Working with Text
• Clustering
• Working with Images
• End-of-Domain Specific Learning

Appendix A: mlwpy.py Listing