- Weak learner: a learner that performs ralatively poorly – its accuracy is above chance.
- Strong learner: a learner that achieves arbitarily good performance, much better than random guessing.
Ensemble learning combines weak learners into a strong learner, i.e., a learner with lower expected error. The expected error can are introduced in Bias-variance Trade-off.
There are several types of ensemble learning. - Bagging (Bootstrap aggregating)
- Boosting
- Stacking
- Mixture of Experts (MoE)