Boosting – On Adaptive Situated Agents

In this recent JMLR paper, David Mease and Abraham Wyner experimentally show that the statistical viewpoint to boosting, which is defined as a stagewise optimization problem, does not give a sufficient interpretation for understanding the behavior of boosting algorithms (I guess they just focus on AdaBoost).

Fortunately, the paper is followed by a series of discussions by people like Andreas Buja, Yoav Freund, Robert Schapire, Jerome Friedman, Trevor Hastie, Robert Tibshirani, Peter Bickel, and other great researchers.

I have to find time and read all these papers closely. Boosting is an interesting subject for me (and probably many others). Its “automatic” and performance-dependent feature selection property deserves better understanding.

Michael Steele has this course on “Probability Inequalities and Machine Learning”. This is the kind of course I’ve really liked to take. Anyway, for most of us who cannot attend the course, following his suggested reading may help somewhat.

There are some known papers/lecture notes in the reading list; such as Gabor Lugosi’s “Concentration of measure inequality” which is very readable.

There are also some other books that I hadn’t heard of it (what a shame), but seems to be nice: Pascal Massart‘s Concentration Inequalities and Model Selection.

Also in the page I found this paper by Mease and Wyner entitled “Evidence Contrary to the Statistical View of Boosting”. I definitely should read it.

well … that’s it for now!