Category Archives: Regression

Genetic algorithms and symbolic regression

Posted on June 13, 2015 by Jesse Johnson

A few months ago, I wrote a post about optimization using gradient descent, which involves searching for a model that best meets certain criteria by repeatedly making adjustments that improve things a little bit at a time. In many situations, this works … Continue reading →

Posted in Modeling, Regression | 1 Comment

Optimization

Posted on September 30, 2014 by Jesse Johnson

Optimization is a topic that has come up in a number of posts on this blog, but that I’ve never really addressed directly. So, I though it was about time that I gave it its own post. The term “optimization” … Continue reading →

Posted in Classification, Regression | 3 Comments

Logistic regression

Posted on May 17, 2013 by Jesse Johnson

In the last post, I introduced the Support Vector Machine (SVM) algorithm, which attempts to find a line/plane/hyperplane that separates the two classes of points in a given data set. This algorithm adapts elements of linear regression, a statistical tool (namely, … Continue reading →

Posted in Classification, Regression | 18 Comments

General regression and over fitting

Posted on March 26, 2013 by Jesse Johnson

In the last post, I discussed the statistical tool called linear regression for different dimensions/numbers of variables and described how it boils down to looking for a distribution concentrated near a hyperplane of dimension one less than the total number … Continue reading →

Posted in Modeling, Regression | 14 Comments

The geometry of linear regression

Posted on March 18, 2013 by Jesse Johnson

In this post, we’ll warm up our geometry muscles by looking at one of the most basic data analysis techniques: linear regression. You’ve probably encountered it elsewhere, but I want to think about it from the point of view of … Continue reading →

Posted in Modeling, Regression | 28 Comments