Boosting理解

俗话说的好，三个臭皮匠顶个诸葛亮。Boosting大概就是作这种事情，也能够理解是一个很牛的企业管理者，充分的利用底下员工的各类优势，合理发挥每一个人的能量。

简单的理解boosting就是把不少个不一样的classifiers使用不一样的权重组合起来成一个新的strong classifier进行分类工做。

把个人学习笔记记录一下吧，以供之后使用这个算法的时候参考。

How to obtain the rough rule of thumb?

How to combine this rule of thumbs to get a good thumb?

What is the procedure to choose examples in each of rounds?

a)What is the procedure to choose examples in each of rounds?

We'll focus on the hardest examples so that we will choose the examples that the previous thumb misclassified.

b)How to combine this rule of thumbs to get a good thumb?

To take a majority vote or weighted majority vote.

 

Change a subset of examples to construct distribution D_t

That's the same thing.

The weight of D_t stands for how we concentrate on a particular round of boosting.

Combine all of h_t into a H_final classifier.

Q:

1.how do you construct the distribution D_t

2.how do we combine all of h_t into a H_final classifier?

AdaBoosting can solve this two questions.

A:

1.At the very first round, we don't have any information. We use uniform distribution weight.

At the following rounds, we try to focus on incorrectly classified examples. We cut the weight of the examples that are correctly classified.

2.

This formular is the weight vote of weak thumbs.

There's ten samples and three samples are misclassified.

So epsilon_1 = 3 / 10 = 0.30.

The alpha_t may bigger if one classifier has less error rate.

The classifier is a black box, so that you can choose whatever method that sitting around.

 

最后附上我本身使用Python实现的adaboosting的一个package或者说class供参考。若有错误请多不吝指正。

https://gitcafe.com/NeighborhoodGuo/Ada_boosting.git

 

Reference:

1.Adaboost Matlab Code

2.Boosting resources collected by Lyon

3.Boosting.org

4.MIT: Simple object detector with boosting

5.Boosting_VideoLectures.NET

6.Boosting算法简介-百度技术博客