RegressionSVM
statistics: RegressionSVM
Support vector machine regression model.
A RegressionSVM object holds a support vector machine fitted to a
continuous response by epsilon-insensitive regression, and predicts
the response for new data with the predict method. Errors smaller
than Epsilon cost nothing, so only the observations outside that
tube become support vectors, and a prediction is a weighted sum of kernel
evaluations against them. The fit is carried out by LIBSVM.
The object keeps its training data, which resubPredict,
resubLoss and crossval work on; compact drops it and
returns a CompactRegressionSVM, which still predicts.
Create a RegressionSVM object with fitrsvm or the class
constructor.
See also: fitrsvm, CompactRegressionSVM, ClassificationSVM
Source Code: RegressionSVM
The RegressionSVM class contains the following properties:
An NxP numeric matrix, as it was supplied to the constructor.
Where the model was fitted from a table, the predictors are the coded matrix and not the table: a variable holding levels is stored as its level codes, and the coding is kept with the model.
This property is read-only.
An Nx1 numeric vector, as it was supplied to the constructor. This property is read-only.
A positive integer scalar, counting only the rows that survived the removal of missing values. This property is read-only.
A logical column vector with the same length as the observations in the
original predictor data X, true for each row that was used for
fitting the RegressionSVM model. It is empty, [],
when every observation was used, so a non-empty value means that rows
holding missing values were dropped. This property is read-only.
A positive integer scalar. This property is read-only.
A cell array of character vectors. This property is read-only.
A character vector. This property is read-only.
A non-negative scalar. An error smaller than Epsilon costs
nothing, so only observations outside the tube become support vectors.
This property is read-only.
A row vector with one entry per predictor, used for standardization. Empty when the predictor data were not standardized. This property is read-only.
A row vector with one entry per predictor, used for standardization. Empty when the predictor data were not standardized. This property is read-only.
A structure holding the SVM formulation, the kernel and its parameters,
the box constraint, Epsilon and the solver settings. The engine
is LIBSVM and the record is LIBSVM’s, so SVMtype names its
formulation and Tolerance and Shrinking are its own
controls; the parameters MathWorks reports for its SMO and ISDA solvers
are absent, this class running neither.
KernelPolynomialOrder belongs to the polynomial kernel alone
and is empty under every other, as it is in MATLAB. Nu is
reported here where MATLAB leaves it empty on a regression model, this
class offering 'nu_svr' through SVMtype and the value
being a real one. This property is read-only.
A numeric column vector with one entry per support vector, holding the difference of the two multipliers each observation carries. Unlike a classifier’s, these are signed: there are no labels to take the sign into, so an observation above the tube and one below it are told apart by the sign of its coefficient. This property is read-only.
A numeric column vector, equal to
(obj.SupportVectors / s)' * obj.Alpha, where s is the
kernel scale, so that a prediction is (x / s) * Beta + Bias,
as in MATLAB. It exists only for a linear kernel; for any other kernel
there is no primal representation and this is empty. This property is
read-only.
A numeric scalar. With a linear kernel the prediction is
X * obj.Beta + obj.Bias. This property is read-only.
A logical column vector with one entry per training observation. This property is read-only.
A numeric matrix with one row per support vector, on the scale the
model was trained on, standardized where Mu is non-empty.
This property is read-only.
A structure with fields Function and Scale, and
Order for a polynomial kernel. Function names the
kernel as MATLAB names it, so a radial basis kernel reports
'gaussian' whichever spelling was given; the kernel the fit was
handed is unchanged in ModelParameters. This property is
read-only.
A numeric column vector with one entry per observation, holding the box
constraint the fit applied to it: n times BoxConstraint
times the observation’s weight in W, which is
BoxConstraint for every observation when no weights were given.
An observation missing a predictor is not fitted and holds NaN.
This property is read-only.
A numeric vector of column indices, and empty when none is. This property is read-only.
A cell array of character vectors. This property is read-only.
A numeric column vector with one entry per training observation,
normalized to sum to one, as MATLAB reports it. It has the class of the
'Weights' given, single or double. This property is read-only.
A cell array with one entry per predictor, holding that predictor’s bin edges where the learner discretized it before fitting. It is empty here and stays empty: this learner fits the predictors as they are, and MATLAB’s reports an empty cell for it as well.
This property is read-only.
Always empty. It is declared for MATLAB compatibility, where it holds what an automatic search over the hyperparameters found. This class fits the parameters it is given and runs no such search, so there is nothing to report. This property is read-only.
A function handle, applied by predict and resubPredict to
the model’s output. It defaults to the identity and may be set after
construction, either to a handle or to the name of a supported
transformation.
The RegressionSVM class offers the following public methods:
RegressionSVM: obj = RegressionSVM (X, Y)
RegressionSVM: obj = RegressionSVM (Tbl, ResponseVarName)
RegressionSVM: obj = RegressionSVM (Tbl, formula)
RegressionSVM: obj = RegressionSVM (Tbl, Y)
RegressionSVM: obj = RegressionSVM (…, name, value)
obj = RegressionSVM (X, Y) returns a support
vector regression model, obj, with X being the predictor data
and Y the continuous response of the observations in X.
The model is fitted by epsilon-insensitive regression: errors
smaller than Epsilon cost nothing, so only the observations
outside that tube become support vectors. Epsilon defaults to
iqr (Y) / 13.49, a robust estimate of a tenth of the
response’s standard deviation, which is what MATLAB uses.
obj = RegressionSVM (…, name, value)
returns a model with additional options specified by Name-Value
pair arguments listed below.
| Name | Value |
|---|---|
'Standardize' | A logical scalar specifying whether the
predictor data should be centred and scaled before training. The same
transformation is applied by predict. The default is
false. |
'CategoricalPredictors' | The predictors whose values
are levels, as indices, as a logical vector with one element per
predictor, or as 'all'. Each is dummy coded in its place, one
column of zeros and ones per level seen in training, named as in
'x1 == 2' in ExpandedPredictorNames, and the coded columns
are not standardized. An observation holding a level the training data
did not is predicted as a row missing a predictor, the weighted lower
median of the training response. A predictor may be named rather than
indexed, as a character matrix of one padded name per row, a string
array or a cellstr; a name must match an entry of
'PredictorNames' exactly, its case included. |
'PredictorNames' | A cell array of character vectors naming the predictors, in the order they appear in X. |
'ResponseName' | A character vector naming the response.
The default is 'Y'. |
'ResponseTransform' | A character vector naming one of
the supported transformations, or a function handle, applied to the
predicted response by predict and resubPredict. The
default is 'none'. |
'Epsilon' | A non-negative scalar, the half-width of the
insensitive tube. The default is iqr (Y) / 13.49, or
0.1 where that is zero. |
'BoxConstraint' | A positive scalar bounding the dual
coefficients, the cost of an error outside the tube. The default is
iqr (Y) / 1.349 for a Gaussian kernel, or 1 where that is
zero, and 1 for any other kernel. |
'Weights' | A nonnegative single or double vector of
observation weights, one per row of X. An observation’s box
constraint is n times BoxConstraint times its weight, the
weights scaled to sum to one; standardization uses weighted means and
standard deviations, and a row of zero or missing weight is left out.
The model’s W keeps the class of the weights, while every
computation runs in double. The default is uniform. |
'KernelFunction' | A character vector naming the kernel,
one of 'linear', the default, 'rbf', 'gaussian',
'polynomial' or 'sigmoid'. |
'PolynomialOrder' | A positive integer, the order of the polynomial kernel. The default is 3. It is ignored by every other kernel. |
'KernelScale' | A positive scalar dividing every
predictor before any kernel is applied, as MATLAB does, so that with
u and v the divided predictors the kernels are u'v,
exp (-||u - v||^2), (1 + u'v)^q and tanh (u'v + c),
c being 'KernelOffset'. The default is 1. |
'KernelOffset' | A non-negative scalar, the constant c of the sigmoid kernel, which MATLAB does not have. MATLAB adds it to every element of the Gram matrix, which leaves the fitted model unchanged, so it changes no other kernel here. The default is 0. |
'SVMtype' | A character vector selecting the
formulation, either 'eps_svr', the default, or 'nu_svr'.
MATLAB fits only the epsilon form; 'nu_svr' is an Octave
extension, in which Nu bounds the fraction of support vectors and
Epsilon is determined by the fit rather than given. |
'Nu' | A scalar in (0, 1] used by
'nu_svr'. The default is 0.5. |
'CacheSize' | A positive scalar, the kernel cache in megabytes. The default is 1000. |
'Tolerance' | A non-negative scalar, the tolerance of the termination criterion. The default is 1e-6. |
'Shrinking' | Either 0 or 1, whether to use the shrinking heuristic. The default is 1. |
The supported values for 'ResponseTransform' are:
| Value | Description |
|---|---|
'none' | x (no transformation) |
'identity' | x (no transformation) |
'exp' | exp (x) |
'log' | log (x) |
See also: fitrsvm, ClassificationSVM, RegressionNeuralNetwork
RegressionSVM: obj = discardSupportVectors (obj)
obj = discardSupportVectors (obj) empties
Alpha and SupportVectors, leaving Beta and
Bias to decide every prediction. A linear kernel needs
nothing else, so the returned model predicts what it predicted
before while carrying one vector in place of many.
The kernel must be linear. Under any other the support vectors are part of the decision function and cannot be dropped. Discarding twice is not an error and changes nothing.
See also: fitrsvm, RegressionSVM, CompactRegressionSVM
RegressionSVM: yFit = predict (obj, XC)
yFit = predict (obj, XC) returns a column
vector holding the predicted response for each row of XC.
RegressionSVM class object.
The transformation named by ResponseTransform is applied to the
model’s output before it is returned.
The new data may be a table, whose variables are matched to the predictors the model was fitted on by name and not by position: one the model was not fitted on is passed over, one it needs and cannot find is named, and a value holding a level is coded as that level was coded at fitting.
See also: RegressionSVM, fitrsvm
RegressionSVM: yFit = resubPredict (obj)
yFit = resubPredict (obj) returns a column vector
holding the predicted response for every observation the model was
trained on.
RegressionSVM class object.
See also: RegressionSVM, fitrsvm
RegressionSVM: L = loss (obj, X, Y)
RegressionSVM: L = loss (obj, Tbl, ResponseVarName)
RegressionSVM: L = loss (obj, Tbl)
RegressionSVM: L = loss (…, name, value)
L = loss (obj, X, Y) returns the
weighted mean squared error between the response Y and the
response the model predicts for X.
RegressionSVM class object.
X may also be a table Tbl, whose variables are matched to
the predictors the model was fitted on by name and not by position.
loss (obj, Tbl, ResponseVarName) takes the
response from the variable ResponseVarName names, and
loss (obj, Tbl) from the variable the model was
fitted on. The response may also be given beside the table as
Y.
L = loss (…, name, value) accepts the
following Name-Value pairs.
| Name | Value |
|---|---|
'LossFun' | 'mse', the default,
'epsiloninsensitive', or a function handle called as
lossfun (Y, yFit, W) returning a scalar.
The epsilon-insensitive loss charges nothing for an error inside
the tube, max (0, abs (Y - yFit) - Epsilon), which is
the quantity the fit itself minimizes. |
'Weights' | A numeric vector of observation weights with one entry per row of X. It defaults to a uniform weight. The weights are normalized to sum to one before the loss is formed. |
See also: RegressionSVM, fitrsvm
RegressionSVM: L = resubLoss (obj)
RegressionSVM: L = resubLoss (…, name, value)
L = resubLoss (obj) returns the weighted mean
squared error of the model on the data it was trained on. It accepts
the same Name-Value pairs as loss.
RegressionSVM class object.
See also: RegressionSVM, fitrsvm
RegressionSVM: CVMdl = crossval (obj)
RegressionSVM: CVMdl = crossval (…, name, value)
CVMdl = crossval (obj) returns a
RegressionPartitionedModel holding one refit of obj per
fold of a ten-fold partition, or of an n-fold one where the
model has fewer than ten observations.
RegressionSVM class object.
CVMdl = crossval (…, name, value)
accepts one, and only one, of the following Name-Value pairs.
| Name | Value |
|---|---|
'KFold' | An integer greater than 1, the number of folds. |
'Holdout' | A scalar in (0, 1), the fraction of observations held out for testing. |
'Leaveout' | 'on' or 'off', whether
to hold out one observation at a time. |
'CVPartition' | A cvpartition object over as
many observations as the model was trained on. |
See also: RegressionSVM, RegressionPartitionedModel, cvpartition
RegressionSVM: CMdl = compact (obj)
CMdl = compact (obj) returns a compact version of
the RegressionSVM object obj, which keeps the support
vectors and their coefficients but drops the training data, so it
predicts identically while carrying no observations.
See also: fitrsvm, RegressionSVM, CompactRegressionSVM
RegressionSVM: savemodel (obj, filename)
savemodel (obj, filename) saves every property of
the RegressionSVM object obj into filename in
binary format, so that it can be read back with loadmodel.
See also: loadmodel, RegressionSVM, fitrsvm