Categories &

Functions List

Function Reference: fitrensemble

statistics: Mdl = fitrensemble (X, Y)

statistics: Mdl = fitrensemble (Tbl, ResponseVarName)

statistics: Mdl = fitrensemble (Tbl, formula)

statistics: Mdl = fitrensemble (Tbl, Y)

statistics: Mdl = fitrensemble (…, name, value)

Fit an ensemble of regression trees.

Mdl = fitrensemble (X, Y) grows 100 regression trees by LSBoost on the NxP predictor matrix X and the numeric response Y, and returns a RegressionEnsemble. With 'Method' set to 'Bag', or with LSBoost resampling, it returns a RegressionBaggedEnsemble. A row missing a predictor or the response is left out.

LSBoost starts from a prediction of zero. Each tree is fitted, with the observation weights, to the residual of the trees before it, and the prediction grows by the learning rate times that tree’s prediction, the learning rate being the tree’s weight. The first tree therefore fits the response itself. Bag grows each tree on a sample drawn in proportion to the weights and averages them.

LSBoost resamples when 'Resample' is 'on' or 'FResample' or 'Replace' is given. Each tree is then fitted to the residual of ceil (FResample * N) rows, drawn with replacement in proportion to the weights or without replacement uniformly, while the prediction and FitInfo run over every row, as in MATLAB R2024a. The ensemble is a RegressionBaggedEnsemble, which records the rows each tree drew and estimates the out-of-bag error.

Name-Value arguments:

NameValue
'Method''LSBoost' (default) or 'Bag'.
'NumLearningCycles'A positive integer, the number of trees to grow. The default is 100.
'Learners''tree' (default) or a template from templateTree, whose options override the defaults: for LSBoost MaxNumSplits 10, MinParentSize 10 and MinLeafSize 5; for Bag unlimited splits, MinParentSize 10, MinLeafSize 5 and NumVariablesToSample ceil (P / 3).
'LearnRate'A number greater than 0 and no greater than 1. The default is 1. LSBoost only.
'FResample'The share of the observations each tree draws, greater than 0 and no greater than 1. The default is 1. Given with LSBoost, the ensemble resamples.
'Replace''on' (default) or 'off', whether the trees draw with replacement. Given with LSBoost, the ensemble resamples.
'Resample''off' (default) or 'on', whether LSBoost resamples. Bag always does.
'NPrint''off' (default) or a positive integer n, to print a line after every n trees.
'Weights'A nonnegative single or double vector with one weight per observation. The default is uniform. The model’s W keeps the class of the weights, while every computation runs in double, so the predictions are double where MATLAB returns single.
'PredictorNames'A cell array of character vectors naming the columns of X.
'ResponseName'The name of the response variable.
'ResponseTransform''none' (default), 'exp', 'log' or a function handle, applied to the predictions. MATLAB R2024a accepts only a function handle here, failing on the named transforms.

'CrossVal' set to 'on', 'KFold', 'Holdout', 'Leaveout' or 'CVPartition', only one of them, fits the ensemble and cross-validates it as crossval does, returning a RegressionPartitionedEnsemble.

Binning and hyperparameter optimization are not implemented, and an option asking for one of them is refused. 'CategoricalPredictors', as indices, as a logical vector with one element per predictor, or as 'all', is passed on to every tree, which splits those predictors into sets of levels as fitrtree does. An ensemble is regularized and shrunk afterwards with the regularize, shrink and cvshrink methods.

See also: RegressionEnsemble, RegressionBaggedEnsemble, CompactRegressionEnsemble, templateTree, TreeBagger

Source Code: fitrensemble

Boost regression trees to predict sepal length from the other three measurements, and watch the training error fall as trees are added.

 load fisheriris
 X = meas(:,2:4);
 y = meas(:,1);
 Mdl = fitrensemble (X, y, 'NumLearningCycles', 50, 'LearnRate', 0.1);
 plot (loss (Mdl, X, y, 'Mode', 'cumulative'));
 xlabel ('Number of trees');
 ylabel ('Training mean squared error');
plotted figure

A bagged ensemble of regression trees predicts by averaging its trees.

 load fisheriris
 rng (42);
 Mdl = fitrensemble (meas(:,2:4), meas(:,1), 'Method', 'Bag', ...
                     'NumLearningCycles', 30);
 yfit = predict (Mdl, meas([1, 51, 101], 2:4))
yfit =

   5.0677
   6.5202
   6.8751

Fit from a table, and predict on one

 load fisheriris
 T = table (meas(:,2), meas(:,3), meas(:,4), meas(:,1), ...
            'VariableNames', {'SW', 'PL', 'PW', 'SL'});
 T.Wide = categorical (meas(:,2) > 3, [false true], {'narrow', 'wide'});

A model formula names the response and the predictors together, and holds main effects only

 Mdl = fitrensemble (T, 'SL ~ PL + Wide');
 Mdl.PredictorNames
ans =
  1x2 cell array

    {'PL'}    {'Wide'}
 Mdl.ResponseName
ans = SL

predict matches the table's variables by name, so a column the model was not fitted on is passed over

 yFit = predict (Mdl, T(1:5,:));
 yFit'
ans =

   5.0000   4.7246   5.0500   5.1461   5.0000