Cost Sensitive Boosting Algorithms: Do We Really Need Them?

Gavin Brown

University of Manchester, UK

22 May 2017, 2:30 pm

Abstract

We provide a unifying perspective for two decades of work on cost-sensitive Boosting algorithms. When analyzing the literature 1997–2016, we find 15 distinct cost-sensitive variants of the original algorithm. of these has its own motivation and claims to superiority - so who should we believe? In this work we critique the Boosting literature using four theoretical frameworks: Bayesian decision theory, the functional gradient descent view, margin theory, and probabilistic modelling. We find, surprisingly, that almost all of the proposed variants are inconsistent with at least one of the theories. After an extensive empirical study, our final recommendation - based on simplicity, flexibility and performance - is to use the original Adaboost algorithm, albeit with a (very) slight tweak - details in the talk.

Bio sketch

GavinBrown is Reader in Machine Learning at the University of Manchester, and Director of Research for the School of Computer Science. His work, and that of his team, has been recognized twice (2004, 2013) with awards from the British Computer Society for outstanding UK PhD of the year. The team is currently conducting research on Machine Learning methods for clinical drug trials, methods for predicting domestic violence, and efficient modular deep neural networks, sponsored by the UK and European Union. Gavin is a keen public communicator, engaging in several public events per year on issues around artificial intelligence and machine learning - including several appearances on the BBC children's channel, explaining robots and AI.