RegressionEnsemble
statistics: RegressionEnsemble
Boosted ensemble of regression trees
A RegressionEnsemble object holds the regression trees LSBoost
grew one after another, each fitted to the residual the trees before it
left, together with the data it was fitted on.
Create one with fitrensemble. A bagged ensemble is a
RegressionBaggedEnsemble, and compact returns a
CompactRegressionEnsemble without the data.
See also: fitrensemble, CompactRegressionEnsemble, RegressionBaggedEnsemble
Source Code: RegressionEnsemble
The RegressionEnsemble class contains the following properties:
Empty until regularize fills it with a structure of lasso
weights for the trees, and emptied again by resume. This
property is read-only.
The response the ensemble was fitted on, a row missing a value having been left out. This property is read-only.
The predictors the ensemble was fitted on, one row per observation. This property is read-only.
A logical column over the rows as supplied. This property is read-only.
The weights given, normalized to sum to one. This property is read-only.
A structure with the fields Type, Method,
LearnerTemplates, NLearn, the number of learning
cycles asked for in all, and for LSBoost LearnRate. This
property is read-only.
This property is read-only.
Always empty, binning not being implemented. This property is read-only.
Always empty, such optimization not being implemented. This property is read-only.
A cell array of character vectors. This property is read-only.
The predictors every tree treats as categorical, empty when none is. This property is read-only.
This property is read-only.
The same as PredictorNames. This property is read-only.
'LSBoost' or 'Bag'. This property is read-only.
Always {'Tree'}. This property is read-only.
This property is read-only.
For LSBoost, a column with the weighted mean squared error of each tree against the residual it was fitted to. Empty for Bag. This property is read-only.
This property is read-only.
Always empty, as MATLAB returns it for tree learners. This property is read-only.
This property is read-only.
A column cell array of CompactRegressionTree objects. This
property is read-only.
A column with one weight per tree. This property is read-only.
'WeightedSum' for LSBoost, 'WeightedAverage' for Bag.
This property is read-only.
See CompactRegressionEnsemble.ResponseTransform.
The RegressionEnsemble class offers the following public methods:
RegressionEnsemble: obj = RegressionEnsemble (X, Y)
RegressionEnsemble: obj = RegressionEnsemble (…, name, value)
fitrensemble is the documented way in, and its help lists the
options both take. A bagged ensemble is fitted by
RegressionBaggedEnsemble.
See also: fitrensemble, RegressionBaggedEnsemble
RegressionEnsemble: CVMdl = crossval (obj)
RegressionEnsemble: CVMdl = crossval (…, name, value)
CVMdl = crossval (obj) refits the ensemble on the
training part of each of ten folds, and returns a
RegressionPartitionedEnsemble. One of 'KFold', an
integer greater than 1, 'Holdout', a number between 0 and 1,
'Leaveout', 'on' for one fold per observation, or
'CVPartition', a cvpartition object, may choose the
partition instead.
See also: RegressionEnsemble, RegressionPartitionedEnsemble, cvpartition
RegressionEnsemble: CMdl = compact (obj)
CMdl = compact (obj) returns a
CompactRegressionEnsemble holding the trees and what prediction
needs. It predicts new data identically.
See also: RegressionEnsemble, CompactRegressionEnsemble
RegressionEnsemble: B = resume (obj, NumLearningCycles)
RegressionEnsemble: B = resume (…, ’NPrint’, n)
B is the ensemble with NumLearningCycles further trees
grown as though the fit had asked for them from the start.
'NPrint' is taken as by fitrensemble. A
Regularization is emptied, its weights no longer covering every
tree; MATLAB keeps it and applies its weights to the first trees.
See also: RegressionEnsemble, fitrensemble
RegressionEnsemble: yfit = predict (obj, X)
RegressionEnsemble: yfit = predict (…, name, value)
Behaves as CompactRegressionEnsemble.predict and takes the same
Name-Value arguments.
See also: RegressionEnsemble, CompactRegressionEnsemble.predict
RegressionEnsemble: L = loss (obj, X, Y)
RegressionEnsemble: L = loss (…, name, value)
Behaves as CompactRegressionEnsemble.loss and takes the same
Name-Value arguments.
See also: RegressionEnsemble, CompactRegressionEnsemble.loss
RegressionEnsemble: imp = predictorImportance (obj)
RegressionEnsemble: [imp, ma] = predictorImportance (obj)
Behaves as CompactRegressionEnsemble.predictorImportance.
See also: RegressionEnsemble, CompactRegressionEnsemble.predictorImportance
RegressionEnsemble: yfit = resubPredict (obj)
RegressionEnsemble: yfit = resubPredict (…, name, value)
predict on X, taking the same Name-Value arguments.
See also: RegressionEnsemble, RegressionEnsemble.predict
RegressionEnsemble: L = resubLoss (obj)
RegressionEnsemble: L = resubLoss (…, name, value)
loss on X and Y, weighted by W unless
'Weights' are given.
See also: RegressionEnsemble, RegressionEnsemble.loss
RegressionEnsemble: B = regularize (obj)
RegressionEnsemble: B = regularize (…, name, value)
B = regularize (obj) fits the training response by
the trees’ predictions with a lasso that has no intercept and whose
weights may not be negative, over a path of penalties, and returns the
ensemble with the result in Regularization. For a penalty
lambda the tree weights a minimize
sum (W .* (Y - P * a) .^ 2) / 2 + lambda * sum (a), where P
holds one column of training predictions per tree and W is the
observation weights, which sum to one. TrainedWeights is left as
it was.
| Name | Value | |
|---|---|---|
'Lambda' | A vector of non-negative penalties. The
default is 0 followed by nine values spaced evenly on a log scale from
lmax / 1000 to lmax, the smallest penalty that sets every
weight to zero, lmax = max (abs (P' * (W .* Y))). | |
'MaxIter' | The most passes of coordinate descent over the trees for each penalty. The default is 1e5. | |
'RelTol' | The descent stops after a pass in which no weight changed by more than this times the largest weight, or this when that weight is below one. The default is 1e-10. |
Regularization is a structure with the fields Method,
'Lasso'; TrainedWeights, one column per penalty;
Lambda; ResubstitutionMSE, the weighted mean squared
error of each column; and CombineWeights,
'WeightedSum'.
MATLAB’s solver can stop well short of the minimum at small penalties,
so its weights there differ from these, which are the minimum. Its
CombineWeights is a function handle, and it also takes
'Npass' and 'Verbose', which are not taken here.
See also: RegressionEnsemble, RegressionEnsemble.shrink, RegressionEnsemble.cvshrink, lasso
RegressionEnsemble: C = shrink (obj)
RegressionEnsemble: C = shrink (…, name, value)
C = shrink (obj) returns a
CompactRegressionEnsemble of the trees whose weight in a column
of Regularization.TrainedWeights is above a threshold, ordered
from the largest weight down, each carrying that weight and their
predictions summed. An ensemble that has not been regularized is
thresholded on its TrainedWeights and keeps its way of combining
its trees.
| Name | Value | |
|---|---|---|
'WeightColumn' | The column of weights to use, a positive integer. The default is 1. | |
'Threshold' | A non-negative number; a tree whose weight is not above it is dropped. The default is 0. | |
'Lambda' | Penalties to regularize with first, as
regularize does, which also takes 'MaxIter' and
'RelTol' here. |
MATLAB accepts a 'WeightColumn' that is not a whole number,
which is refused here.
See also: RegressionEnsemble, RegressionEnsemble.regularize, RegressionEnsemble.cvshrink
RegressionEnsemble: [vals, nlearn] = cvshrink (obj)
RegressionEnsemble: [vals, nlearn] = cvshrink (…, name, value)
[vals, nlearn] = cvshrink (obj) grows the
ensemble again on the training part of each fold, as crossval
does, regularizes it with each penalty, shrinks it at each threshold,
and predicts the fold’s held-out observations. vals holds one row
per penalty and one column per threshold: the weighted mean squared
error pooled over every held-out observation. nlearn holds the
matching mean number of trees kept per fold.
| Name | Value | |
|---|---|---|
'Lambda' | A vector of non-negative penalties. The
default is Regularization.Lambda; an ensemble that has not been
regularized must be given one. | |
'Threshold' | A vector of non-negative thresholds,
as shrink takes them. The default is 0. | |
'KFold', 'Holdout', 'Leaveout',
'CVPartition' The partition, as crossval takes
it, only one of them. The default is ten folds. | ||
'MaxIter', 'RelTol' | As regularize
takes them. |
MATLAB warns and returns empty outputs when an ensemble that has not been regularized is given no penalties; here that is an error.
See also: RegressionEnsemble, RegressionEnsemble.shrink, RegressionEnsemble.crossval