ClassificationTree
statistics: ClassificationTree
Binary decision tree for classification
The ClassificationTree class implements a CART binary decision
tree. Growth splits each node on the single predictor and cut point
that lower the impurity of the response the most, and stops when a node
is pure, too small to be a parent, or has no split leaving enough
observations on both sides. The grown tree is then optionally reduced,
first by merging the leaves that buy no accuracy and then by cost
complexity pruning, which orders the branch nodes by how little risk
their subtrees remove and records that order so a subtree of any size
can be recovered afterwards with prune.
Create a ClassificationTree object by using the fitctree
function or the class constructor.
The fit is carried out by the compiled engine treetrain and
predictions by treepredict, which the regression tree shares.
An observation missing the predictor a node cuts on descends to neither
child. It is counted in that node and in every node above it, and
predict stops it there and gives it that node’s answer, so a row
is never sent down a branch on evidence it does not carry.
A categorical predictor is split into two sets of levels, and an observation whose level a node did not see stops there, as one missing the predictor does.
What this class does not do yet. Surrogate splits and the
'twoing' split criterion are not implemented, and an option
asking for one of them is refused rather than quietly ignored. The six
Surrogate properties are therefore always empty, as they are in
MATLAB without surrogate splits.
See also: fitctree, treetrain, treepredict
Source Code: ClassificationTree
The ClassificationTree class contains the following properties:
A numeric matrix holding the predictor data the model was fitted on. Each column is one predictor and each row one observation. This property is read-only.
A logical or numeric column vector, a character array, or a cell array of character vectors with one row per row of X, holding the observed class label of each observation. This property is read-only.
A positive integer, the number of observations the model was fitted on. It counts the rows kept, so it is smaller than the number of rows given whenever a response was missing. This property is read-only.
A logical column vector with one element per row of the predictor data
as it was given, true for each row used for fitting. It is empty,
[], when every row was used, so a non-empty value means that
rows were dropped. Only a missing response drops a row; a row missing
some of its predictors is kept. This property is read-only.
A numeric column vector of the weights the fit used, one per retained observation. They are the weights given, scaled so that the observations of each class sum to that class’s prior, and so that all of them together sum to one. This property is read-only.
A cell array of character vectors with one name per column of X. This property is read-only.
A character vector naming the response. This property is read-only.
The distinct class labels, in the type the response was given in,
sorted or in the order given by the 'ClassNames' option. This
property is read-only.
A row vector of column indices into X, naming the predictors treated as categorical, empty when none is. This property is read-only.
A cell array of character vectors. It differs from
PredictorNames only when a categorical predictor has been
expanded into one column per level, which this class does not do, so
the two are always equal. This property is read-only.
A cell array with one column vector of bin edges per predictor, empty unless the predictors were binned before fitting. Binning is not implemented, so this is always empty. This property is read-only.
A structure recording the options the tree was grown under:
SplitCriterion, MinParent, MinLeaf,
MaxSplits, NVarToSample, MergeLeaves,
Prune, PruneCriterion, QEToler,
NSurrogate, MaxCat, AlgCat,
PredictorSelection, Method and Type.
MinParent is the value the fit used, which is
max (MinParentSize, 2 * MinLeafSize) and so may exceed the
'MinParentSize' asked for. This property is read-only.
Hyperparameter optimization is not implemented, so this is always empty. This property is read-only.
A positive integer, the number of nodes the tree holds, branch nodes and leaves together. Nodes are numbered as they are created, so a parent always carries a lower number than either of its children. This property is read-only.
A NumNodesx2 matrix naming the left and the right child of each node. A leaf carries a zero in both columns. This property is read-only.
A column vector naming the parent of each node. The root carries a zero. This property is read-only.
A logical column vector, true for each node that carries a split and false for each leaf. This property is read-only.
A cell array of character vectors with one entry per node, holding the name of the predictor the node splits on and an empty character vector at a leaf. This property is read-only.
A column vector holding, for each node, the column of X the node splits on, and zero at a leaf. This property is read-only.
A column vector holding, for each node, the value the split compares
the predictor against: an observation goes left when its value is less
than the cut point and right otherwise. A leaf and a categorical cut
carry NaN.
This property is read-only.
A cell array of character vectors holding 'continuous' at a
branch node that cuts a numeric predictor at a point,
'categorical' at one that splits a set of levels, and an empty
character vector at a leaf. This property is read-only.
A NumNodesx2 cell array holding, for a node that cuts a categorical predictor, the levels sent left and the levels sent right. Every other entry is empty. This property is read-only.
A Nx2 cell array with one row per categorical split, in node order, holding the levels sent left and the levels sent right. It is empty when no split is categorical. This property is read-only.
A column vector holding how many training observations reached each node. A row missing the predictor its node cuts on is counted at that node and at none below it, so a parent’s size is not in general the sum of its children’s. This property is read-only.
A cell array of character vectors naming, for each node, the class of least expected misclassification cost given the node’s class probabilities. Under the default cost that is simply the most probable class, with the first of the class names kept on a tie. This property is read-only.
A column vector holding, for each node, the expected misclassification cost of the class the node is assigned. Under the default cost that is the probability that the node’s class is wrong, one less the largest class probability. This property is read-only.
A column vector holding, for each node, the total weight of the
observations that reached it, the weights being those in W.
The root carries one. This property is read-only.
A column vector holding, for each node, the impurity of the node
weighted by the probability of reaching it, measured by whichever of
'gdi' and 'deviance' the tree was grown under.
A non-default Cost enters here rather than through the class
probabilities: the weights are scaled class by class by the total cost
of misclassifying that class, and the impurity is measured on the
scaled distribution. This property is read-only.
A NumNodesxK matrix holding how many training observations of each class reached each node. These are counts and take no notice of the observation weights. This property is read-only.
A NumNodesxK matrix holding, for each node, the weight of each class among the observations that reached it, as a proportion of the node’s total weight. The root row is the prior. This property is read-only.
A column vector holding, for each branch node, the level of the cost
complexity sequence at which it stops being a branch node, and zero at
a leaf. Pruning the tree to level L turns every node whose
level is between one and L into a leaf. It is empty when
neither 'Prune' nor 'MergeLeaves' was asked for, since
no sequence was then estimated. This property is read-only.
A column vector with one element per level of the pruning sequence,
the first of which is zero and stands for the unpruned tree. Level
L is the smallest subtree that is optimal for every complexity
parameter from PruneAlpha(L+1) up to the next one. This
property is read-only.
Surrogate splits are not implemented, so this is always empty. This property is read-only.
Surrogate splits are not implemented, so this is always empty. This property is read-only.
Surrogate splits are not implemented, so this is always empty. This property is read-only.
Surrogate splits are not implemented, so this is always empty. This property is read-only.
Surrogate splits are not implemented, so this is always empty. This property is read-only.
Surrogate splits are not implemented, so this is always empty. This property is read-only.
A numeric row vector with one element per class, summing to one. It defaults to the weight each class carries in the training data, and may be reassigned after fitting.
Reassigning it re-derives W and every node statistic that
depends on the class weights, so ClassProbability,
NodeProbability, NodeClass, NodeError and
NodeRisk all follow. The shape of the tree does not, having
been decided by the prior in force when it was grown.
A square numeric matrix with one row and column per class, where
Cost(i,j) is the cost of classifying an observation of class
i into class j. It defaults to 1 - eye (K) and may
be reassigned after fitting.
Reassigning it re-derives NodeClass, NodeError and
NodeRisk, and changes what predict answers. The shape
of the tree does not follow, having been decided by the cost in force
when it was grown.
A character vector naming the function predict applies to the
class probabilities before returning them, or a function handle. The
default is 'none'.
The ClassificationTree class offers the following public methods:
ClassificationTree: obj = ClassificationTree (X, Y)
ClassificationTree: obj = ClassificationTree (…, name, value)
obj = ClassificationTree (X, Y) grows a tree
on the NxP numeric matrix X of predictor data and the
Nx1 response Y, and returns it as a
ClassificationTree object. Y may be a numeric or logical
vector, a character array, or a cell array of character vectors, and
the class names come back in the type it was given in.
obj = ClassificationTree (…, name,
value) takes the options below.
| Name | Value |
|---|---|
'AlgorithmForCategorical' | How a node with three or
more classes splits a categorical predictor: 'exact',
'pullleft', 'pca' or 'ovabyclass'. By default
the exact search is taken when the node holds at most
'MaxNumCategories' levels, and otherwise the best split of
'ovabyclass', 'pca' and 'pullleft', leaving
'ovabyclass' out above four classes. |
'CategoricalPredictors' | The predictors whose values
are levels, as indices, as a logical vector with one element per
predictor, or as 'all'. |
'ClassNames' | The classes to fit, of the same type as Y. Observations of any other class are dropped. The model keeps the classes in this order; by default they are sorted. |
'Cost' | A square matrix with one row and column per
class, where element (i,j) is the cost of classifying an
observation of class i into class j, or a structure with
fields ClassNames and ClassificationCosts. The
default is 1 - eye (K). |
'MaxNumCategories' | A nonnegative integer, the most levels a node with three or more classes searches exactly by default. The default is 10. |
'MaxNumSplits' | A nonnegative integer, the largest number of branch nodes the tree may take. The default is one less than the number of observations, which is as many as a tree can have. |
'MergeLeaves' | 'on' (default) or
'off'. When on, a pair of leaves whose parent is no worse
than the two of them together is merged back into that parent. |
'MinLeafSize' | A positive integer, the fewest observations a leaf may hold. The default is 1. A split leaving fewer than this on either side is not taken. |
'MinParentSize' | A positive integer, the fewest
observations a node must hold to be split at all. The default is 10.
The value the fit uses is max (MinParentSize, 2 * MinLeafSize),
since a smaller node cannot give both children a legal leaf. |
'PredictorNames' | A cell array of character vectors naming the columns of X. |
'Prior' | 'empirical' (default),
'uniform', a numeric vector with one element per class, or a
structure with fields ClassNames and
ClassProbs. |
'Prune' | 'on' (default) or 'off'.
When on, the cost complexity pruning sequence is estimated and
reported in PruneList and PruneAlpha. The tree returned
is the unpruned one either way; prune takes a subtree out of
the sequence. |
'PruneCriterion' | 'error', the only
criterion implemented. |
'ResponseName' | A character vector naming the
response. The default is 'Y'. |
'ScoreTransform' | A character vector naming a
transform to apply to the scores, or a function handle. The default
is 'none'. |
'SplitCriterion' | 'gdi' (default), the Gini
diversity index, or 'deviance', the cross entropy. |
'Weights' | A nonnegative numeric vector with one element per observation. The default is uniform. |
On a node with three or more classes and more than
'MaxNumCategories' levels, the heuristic splits, the choice
between equally good partitions and which side each set of levels
takes may differ from MATLAB’s.
See also: fitctree, treetrain, treepredict
ClassificationTree: label = predict (obj, XC)
ClassificationTree: [label, score] = predict (…)
ClassificationTree: [label, score, node] = predict (…)
ClassificationTree: [label, score, node, cnum] = predict (…)
label = predict (obj, XC) sends each row of
XC down the tree and returns the class of the node it comes to
rest at. XC must have as many columns as the predictor data the
model was fitted on.
[label, score] = predict (…) also returns
score, an NxK matrix holding the class probabilities of
the node each row landed in, after ScoreTransform.
[label, score, node] = predict (…) also
returns the number of the node each row landed in, and
[label, score, node, cnum] = predict
(…) the index of the predicted class into ClassNames.
The label is the class of least expected misclassification cost, which
under the default Cost is the most probable class of the node.
A row missing the predictor a node cuts on is stopped at that node and takes its answer, rather than being sent down a branch on evidence the row does not carry.
See also: ClassificationTree, fitctree
ClassificationTree: obj2 = prune (obj)
ClassificationTree: obj2 = prune (obj, 'Level', L)
ClassificationTree: obj2 = prune (obj, 'Alpha', A)
ClassificationTree: obj2 = prune (obj, 'Nodes', N)
obj2 = prune (obj) returns the tree unchanged.
obj2 = prune (obj, turns
every branch node whose 'Level', L)PruneList level is between one and
L into a leaf and discards everything below it. Level zero is
the tree itself and the largest level is the root alone. A level
above the largest prunes to the root and warns.
obj2 = prune (obj, prunes
to the smallest subtree that is optimal for the cost complexity
parameter A, which is the largest level whose 'Alpha', A)PruneAlpha
does not exceed it.
obj2 = prune (obj, turns
the branch nodes named in N into leaves, along with everything
below them, and leaves the rest of the tree alone.
'Nodes', N)
Pruning renumbers the nodes, so the properties of the returned tree are those of a tree of that shape and not a subset of the original’s.
See also: ClassificationTree, fitctree, ClassificationTree.PruneList, ClassificationTree.PruneAlpha
ClassificationTree: E = cvloss (obj)
ClassificationTree: [E, SE, Nleaf, BestLevel] = cvloss (obj)
ClassificationTree: […] = cvloss (…, name, value)
E = cvloss (obj) partitions the training data into
ten stratified folds, grows a tree on the training part of each, and
returns the loss of the held-out part.
[E, SE, Nleaf, BestLevel] = cvloss
(…) also returns SE, the standard error of E over
the folds, Nleaf, the number of leaves each subtree holds, and
BestLevel, the pruning level chosen by 'TreeSize'. Each
has one element per subtree asked for.
[…] = cvloss (…, name, value) takes the
options below.
| Name | Value |
|---|---|
'SubTrees' | A vector of pruning levels in ascending
order, or 'all' for every level of the sequence. The default
is 0, the unpruned tree. |
'TreeSize' | 'se' (default), the smallest
subtree whose loss is within one standard error of the smallest loss,
or 'min', the smallest subtree of least loss. |
'KFold' | An integer greater than 1, the number of folds. The default is 10. A value above the number of observations is reduced to it. |
A fold’s tree is pruned to the level its own sequence gives for the geometric mean of the parent’s two neighbouring complexity parameters, which is the classical way a fold is matched to a subtree of the whole tree. The last level takes every fold’s tree back to its root.
Every fold is grown with the parent’s class names, prior, cost and observation weights, and the loss is weighed by the model’s own weights.
The standard error is not MATLAB’s. This is the standard error of the loss over the folds, which is what the name means and which is zero when every fold answers alike, as MATLAB’s is. Its value is not MATLAB’s: on the iris tree MATLAB reports 0.019956 where the folds give 0.024343, and no formula over the fold losses, the observation losses or the loss itself reproduces MATLAB’s number. E, Nleaf and BestLevel are measured and match.
See also: ClassificationTree, ClassificationTree.prune, ClassificationTree.crossval, ClassificationTree.loss
ClassificationTree: imp = predictorImportance (obj)
imp = predictorImportance (obj) returns a row
vector with one element per predictor, holding the total drop in risk
the splits on that predictor bring about, divided by the number of
branch nodes. A predictor the tree never splits on scores zero.
The drop at a branch node is its NodeRisk less the risk of its
two children and less what it holds back, so a predictor that is
chosen often, high up, and on nodes it separates well, scores
highest. The numbers are comparable
between predictors of one tree and not between trees.
See also: ClassificationTree, fitctree, ClassificationTree.NodeRisk
ClassificationTree: r = nodeVariableRange (obj, node)
r = nodeVariableRange (obj, node) returns a
structure with one field per predictor the path from the root to
node cuts on, holding the two-element range of values that reach
the node, or for a categorical predictor the levels that reach it. A
predictor the path never cuts on is unconstrained and is left out, so
the root gives a structure with no fields.
See also: ClassificationTree, fitctree
ClassificationTree: view (obj)
view (obj) prints one line per node: a branch node names
the predictor it cuts on, the cut point, and the node each side leads
to, and a leaf names the class it assigns. A branch node’s line ends
with the class it would assign itself, which is the answer an
observation missing that predictor gets.
See also: ClassificationTree, fitctree
ClassificationTree: CMdl = compact (obj)
CMdl = compact (obj) returns a
CompactClassificationTree object carrying the tree and
everything predict needs, but not the observations the model
was fitted on. It classifies new data identically and is far smaller
to keep or to ship.
See also: CompactClassificationTree, ClassificationTree
ClassificationTree: CVMdl = crossval (obj)
ClassificationTree: CVMdl = crossval (…, name, value)
CVMdl = crossval (obj) partitions the training data
into ten folds, or into as many folds as there are observations when
there are fewer than ten, grows a tree on the training part of each
and returns them as a ClassificationPartitionedModel.
CVMdl = crossval (…, name, value) takes
one of the following, and one only.
| Name | Value |
|---|---|
'KFold' | An integer greater than 1, the number of folds. |
'Holdout' | A value between 0 and 1, the fraction of the data held out for testing, which gives a single fold. |
'Leaveout' | 'on' or 'off', one fold
per observation. |
'CVPartition' | A cvpartition object. |
Every fold is grown with the parent’s class names, prior, cost and observation weights rather than being left to re-derive them from its own rows, so a fold reports the prior of the whole data and not its own frequencies.
See also: ClassificationPartitionedModel, ClassificationTree, cvpartition
ClassificationTree: m = margin (obj, X, Y)
m = margin (obj, X, Y) returns one
margin per observation: the score the model gives the observation’s
true class, less the largest score it gives any other class. A
positive margin means the observation is classified correctly, and a
larger one means it is classified more confidently.
See also: ClassificationTree, ClassificationTree.edge, ClassificationTree.loss, ClassificationTree.predict
ClassificationTree: e = edge (obj, X, Y)
ClassificationTree: e = edge (…, 'Weights', w)
e = edge (obj, X, Y) returns the
weighted mean of the margins, a single number summarising how
confidently the model classifies the data.
The weights are normalized within each class to that class’s prior before they are applied.
See also: ClassificationTree, ClassificationTree.margin, ClassificationTree.loss, ClassificationTree.predict
ClassificationTree: l = loss (obj, X, Y)
ClassificationTree: l = loss (…, name, value)
l = loss (obj, X, Y) returns the
minimum expected misclassification cost.
l = loss (…, name, value) takes the
following options.
| Name | Value |
|---|---|
'LossFun' | One of 'binodeviance',
'classifcost', 'classiferror', 'exponential',
'hinge', 'logit', 'mincost' (default) or
'quadratic'. |
'Weights' | A numeric vector of observation weights, one per row of X. |
See also: ClassificationTree, ClassificationTree.margin, ClassificationTree.edge, ClassificationTree.predict
ClassificationTree: label = resubPredict (obj)
ClassificationTree: [label, score, node, cnum] = resubPredict (obj)
label = resubPredict (obj) is
predict (obj, obj.X), and takes the same outputs.
See also: ClassificationTree, ClassificationTree.predict
ClassificationTree: m = resubMargin (obj)
m = resubMargin (obj) is
margin (obj, obj.X, obj.Y).
See also: ClassificationTree, ClassificationTree.margin
ClassificationTree: e = resubEdge (obj)
e = resubEdge (obj) is edge over the
training data, weighed as the fit weighed it.
See also: ClassificationTree, ClassificationTree.edge
ClassificationTree: l = resubLoss (obj)
ClassificationTree: l = resubLoss (…, name, value)
l = resubLoss (obj) is loss over the
training data, weighed as the fit weighed it, and takes the same
'LossFun' option. Giving 'Weights' weighs the
training data some other way instead, which MATLAB refuses rather
than honours.
See also: ClassificationTree, ClassificationTree.loss
ClassificationTree: savemodel (obj, filename)
savemodel (obj, filename) saves each property of a
ClassificationTree object into an Octave binary file, the name of
which is specified in filename, along with an extra variable,
which defines the type of classification object these variables
constitute. Use loadmodel in order to load a classification
object into Octave’s workspace.
See also: loadmodel, fitctree, ClassificationTree