Combine trees with bagging and boosting, evaluate out-of-sample behavior, and diagnose feature importance safely.
For a small example, three independently trained trees predict 8, 10, and 15. Average their outputs and compare the variance with relying on the most extreme single tree.
Mixed tabular features contain nonlinear interactions. Compare random forest and gradient boosting with cross-validation, tune capacity, calibrate probabilities, and analyze SHAP cautiously.
Initializing Interactive Playground...
Predict how one control changes the Tree Ensembles result.
The starter deliberately follows a tempting but flawed assumption: Choosing by training score
Check: The browser checks pass and you can name the failure mechanism before opening the explanation.
Failing starter lab
Complete a transparent forest probability and convert it to a decision without hiding the policy threshold.
Run the incomplete starter before opening hints. Use the first wrong output or failed check to form a repair hypothesis.
Explanations after each answer · 80% to pass · Tree Ensembles
Chapter 1 of 5 · Learn
A single decision tree is easy to inspect but unstable: a small change in data can create a different first split and a different structure. Ensembles reduce that weakness by combining many trees.
Bagging trains trees independently on resampled data and averages their predictions. Random forests add random feature subsets, making trees less correlated so averaging helps more. Boosting trains trees sequentially; each new tree focuses on correcting the current model. Bagging mainly reduces variance. Boosting can reduce bias and variance, but is more sensitive to noise, leakage, and tuning.
Ideas to understand
Random Forests Part 1: Building, Using and Evaluating
StatQuest
See how bootstrap samples and randomized features turn many unstable trees into a stronger ensemble.
Primary sources and free technical reading for Tree Ensembles.
Leo Breiman’s foundational paper on random forests.
The foundational gradient-boosting paper.
Practical documentation for forests, gradient boosting, voting, and stacking.