BCS AIF Exam - Topic 12 Question 32 Discussion

Weak Learner: Colloquially, a model that performs slightly better than a naive model.

More formally, the notion has been generalized to multi-class classification and has a different meaning beyond better than 50 percent accuracy.

For binary classification, it is well known that the exact requirement for weak learners is to be better than random guess. [...] Notice that requiring base learners to be better than random guess is too weak for multi-class problems, yet requiring better than 50% accuracy is too stringent.

--- Page 46,Ensemble Methods, 2012.

It is based on formal computational learning theory that proposes a class of learning methods that possess weakly learnability, meaning that they perform better than random guessing. Weak learnability is proposed as a simplification of the more desirable strong learnability, where a learnable achieved arbitrary good classification accuracy.

A weaker model of learnability, called weak learnability, drops the requirement that the learner be able to achieve arbitrarily high accuracy; a weak learning algorithm needs only output an hypothesis that performs slightly better (by an inverse polynomial) than random guessing.

---The Strength of Weak Learnability, 1990.

It is a useful concept as it is often used to describe the capabilities of contributing members of ensemble learning algorithms. For example, sometimes members of a bootstrap aggregation are referred to as weak learners as opposed to strong, at least in the colloquial meaning of the term.

More specifically, weak learners are the basis for the boosting class of ensemble learning algorithms.

The term boosting refers to a family of algorithms that are able to convert weak learners to strong learners.

https://machinelearningmastery.com/strong-learners-vs-weak-learners-for-ensemble-learning/

The best technique to adopt when a weak learner's hypothesis accuracy is only slightly better than 50% is boosting. Boosting is an ensemble learning technique that combines multiple weak learners (i.e., models with a low accuracy) to create a more powerful model. Boosting works by iteratively learning a series of weak learners, each of which is slightly better than random guessing. The output of each weak learner is then combined to form a more accurate model. Boosting is a powerful technique that has been proven to improve the accuracy of a wide range of machine learning tasks. For more information, please see the BCS Foundation Certificate In Artificial Intelligence Study Guide or the resources listed above.