Ensemble Learning in Machine Learning

Ensemble learning in machine learning combines the predictions from multiple models to improve the overall performance.

The following content will cover step by step explanation on Bagging and Boosting, which are the most common ensemble technique.

Let’s first look at what is traditional single model ML. In traditional single model ML, we train a single model by providing it with example data in certain domain. Most of the case we will find that the model is either high variance or/and high bias.

For bagging, we usually create several subsets of data from training sample. Each collection of subset data is used to train their own model in parallel way. Average of all prediction from different models are used which is more robust than a single model.

Step 1: Bootstrapping

Bootstrap refers to random sampling with replacement. In bootstrapping, training instances can be sampled several times for the same model.

Step 2: Parallel Training

Model can all be trained in parallel on each sample from bootstrapping.

Step 3: Aggregation

Aggregation function is typically the statical model for classification and average for regression. It helps reduce both bias and variance of a single model.

Related posts:

Customer support is key to keep your existing customers, it is key to maintain a good reputation, it is key to acquire new customers. A great way to take care of customer support is to work with a…

When I joined Coca-Cola in 1993, nobody knew what Yahoo! was. A new employee from IBM joined in the systems development department of Coca-Cola, a basement of Shibuya headquarters of Coca-Cola Japan…

In our present situation, the Corona Virus (COVID-19) pandemic makes a lot of people suffer. Therefore, many of us feel vulnerable. Since the start of this critical situation, we have worried about…