The bootstrap is a widely used resampling technique first introduced by Bradley Efron in 1979 commonly used to quantify the uncertainty associated with a given estimator or statistical learning method. The bootstrap can be applied to many problems and methods, and is commonly used to estimate the standard errors of... [Read More]

Nested cross-validation can be viewed as an extension of simpler cross-validation techniques. When performing model selection or model evaluation, $k$-fold cross-validation is a crucial method for estimating a particular model’s test error on unseen observations. However, as Cawley and Talbot discussed in a 2010 paper, when performing model selection and... [Read More]

In a previous post I introduced the concept of cross-validation as a resampling technique. In particular, cross-validation is useful for estimating the test error of a particular model fit in order to evaluate its performance, or to decide on an optimal level of flexibility. In addition, cross-validation can also be... [Read More]

Resampling methods are a crucial tool used commonly in modern statistics and data science. These methods involve taking repeated samples from a training dataset and refitting a model of interest on each individual sample to obtain additional information about the fitted model. These methods allow us to learn new information... [Read More]

In this post, I’ll be exploring quadratic discriminant analysis. I’ll compare and contrast this method with linear discriminant analysis, and work through an example using scikit-learn and the slimmed down Titanic dataset from one of my prior posts on logistic regression. [Read More]

This post is the second in a series on linear discriminant analysis (LDA) for classification. In the first post, I introduced much of the theory behind linear discriminant analysis. In this post, I’ll explore the method using scikit-learn. I’ll also discuss classification metrics such as precision and recall, and compare... [Read More]

This post is the first in a series on the linear discriminant analysis method. In this series, I’ll discuss the underlying theory of linear discriminant analysis, as well as applications in Python. [Read More]

I recently read through this excellent Medium article about the ColumnTransformer estimator in scikit-learn and how it can be used in tandem with Pipelines and the OneHotEncoder estimator. To strengthen my own understanding of the concept, I decided to follow the post with my own working example, and summarize the... [Read More]

This post is the second in a series on the logistic regression model. In this post, I’ll work through an example using the well known Titanic dataset, scikit-learn and statsmodels. I’ll discuss one-hot encoding, create a 3D logistic regression plot using Plotly, and demonstrate multiclass logistic regression with scikit-learn. [Read More]

This is the first post in a series on the logistic regression model. The structure of this post was influenced by the fourth chapter of An Introduction to Statistical Learning: with Applications in R by Gareth James, Daniela Witten, Trevor Hastie, and Robert Tibshirani. [Read More]