4.5 Outro

4.5.1 Remarks

Other prominent classification algorithms:

  • Naive Bayes and other probabilistic approaches,
  • Support Vector Machines (SVMs) and other kernel methods,
  • (Artificial) (Deep) Neural Networks.

Interestingly, in the next chapter we will note that the logistic regression model is a special case of a feed-forward single layer neural network.

We will also generalise the binary logistic regression to the case of a multiclass classification.

The state-of-the art classifiers called Random Forests and XGBoost (see also: AdaBoost) are based on decision trees. They tend to be more accurate but – at the same time – they fail to exhibit the decision trees’ important feature: interpretability.

Trees can also be used for regression tasks, see R package rpart.

4.5.2 Further Reading

Recommended further reading:

  • (James et al. 2017: Chapters 4 and 8)


  • (Hastie et al. 2017: Chapters 4 and 7 as well as (*) Chapters 9, 10, 13, 15)