RegressionKernel
statistics: RegressionKernel
Gaussian kernel regression model for large data.
A RegressionKernel object maps the predictors into a randomized
feature space whose inner product approximates a Gaussian kernel, and then
fits a linear model there. A kernel regression is therefore as nonlinear
as a support vector machine with a Gaussian kernel, while costing what a
linear fit costs: nothing of size is ever formed.
The expansion is the random Fourier basis of Rahimi and Recht, drawn once
when the model is fitted and kept with it, so predict maps new data
through the same basis. MATLAB approximates the same kernel by the
Fastfood construction, which reaches the same distribution more cheaply;
the two are interchangeable in distribution but not draw by draw, and the
draws come from different generators in any case, so the predictions of a
model fitted here and one fitted in MATLAB differ even from the same seed.
What does not differ is what they estimate.
Like RegressionLinear the object holds no copy of the training
data. It does hold the basis and the coefficients, so it is bounded by
the number of expansion dimensions rather than by the number of
observations.
Create a RegressionKernel object with fitrkernel.
See also: fitrkernel, RegressionLinear, RegressionSVM
Source Code: RegressionKernel
The RegressionKernel class contains the following properties:
A nonnegative scalar for a support vector machine, and empty for a
least squares fit, which has no such band. It defaults to the
interquartile range of the response over 13.49, an estimate of its
standard deviation, or to 0.1 when that range is zero. This
property is read-only.
Fit fuel consumption through a randomized Gaussian kernel, which bends where a linear model cannot.
load carsmall X = [Acceleration, Displacement, Horsepower, Weight]; ok = ! any (isnan ([X, MPG]), 2); Mdl = RegressionKernel (X(ok,:), MPG(ok))
Mdl =
RegressionKernel
ResponseName: 'Y'
Learner: 'svm'
NumExpansionDimensions: 128
KernelScale: 1
Lambda: 0.0107527
BoxConstraint: 1
Epsilon: 0.926612
yFit = predict (Mdl, X(find (ok, 3),:))
yFit = 22.727 24.159 21.182
Standardizing matters more here than for a linear fit, since the kernel measures one distance across predictors of every scale.
load carsmall X = [Acceleration, Displacement, Horsepower, Weight]; ok = ! any (isnan ([X, MPG]), 2); plain = RegressionKernel (X(ok,:), MPG(ok)); scaled = RegressionKernel (X(ok,:), MPG(ok), 'Standardize', true); plainLoss = loss (plain, X(ok,:), MPG(ok))
plainLoss = 55.467
scaledLoss = loss (scaled, X(ok,:), MPG(ok))
scaledLoss = 22.986
A positive scalar. It is the reciprocal of the product of
Lambda and the number of observations, so setting either of
the two in the constructor fixes the other, and giving both is an
error. This property is read-only.
Fit fuel consumption through a randomized Gaussian kernel, which bends where a linear model cannot.
load carsmall X = [Acceleration, Displacement, Horsepower, Weight]; ok = ! any (isnan ([X, MPG]), 2); Mdl = RegressionKernel (X(ok,:), MPG(ok))
Mdl =
RegressionKernel
ResponseName: 'Y'
Learner: 'svm'
NumExpansionDimensions: 128
KernelScale: 1
Lambda: 0.0107527
BoxConstraint: 1
Epsilon: 0.926612
yFit = predict (Mdl, X(find (ok, 3),:))
yFit = 22.012 23.693 22.628
Standardizing matters more here than for a linear fit, since the kernel measures one distance across predictors of every scale.
load carsmall X = [Acceleration, Displacement, Horsepower, Weight]; ok = ! any (isnan ([X, MPG]), 2); plain = RegressionKernel (X(ok,:), MPG(ok)); scaled = RegressionKernel (X(ok,:), MPG(ok), 'Standardize', true); plainLoss = loss (plain, X(ok,:), MPG(ok))
plainLoss = 54.302
scaledLoss = loss (scaled, X(ok,:), MPG(ok))
scaledLoss = 21.030
A character vector, or the text of the function handle that was supplied. Assigning to it accepts either.
Fit fuel consumption through a randomized Gaussian kernel, which bends where a linear model cannot.
load carsmall X = [Acceleration, Displacement, Horsepower, Weight]; ok = ! any (isnan ([X, MPG]), 2); Mdl = RegressionKernel (X(ok,:), MPG(ok))
Mdl =
RegressionKernel
ResponseName: 'Y'
Learner: 'svm'
NumExpansionDimensions: 128
KernelScale: 1
Lambda: 0.0107527
BoxConstraint: 1
Epsilon: 0.926612
yFit = predict (Mdl, X(find (ok, 3),:))
yFit = 23.357 22.807 23.020
Standardizing matters more here than for a linear fit, since the kernel measures one distance across predictors of every scale.
load carsmall X = [Acceleration, Displacement, Horsepower, Weight]; ok = ! any (isnan ([X, MPG]), 2); plain = RegressionKernel (X(ok,:), MPG(ok)); scaled = RegressionKernel (X(ok,:), MPG(ok), 'Standardize', true); plainLoss = loss (plain, X(ok,:), MPG(ok))
plainLoss = 50.747
scaledLoss = loss (scaled, X(ok,:), MPG(ok))
scaledLoss = 19.494
A cell array of character vectors with one name per column of the
training data, defaulting to 'x1', 'x2' and so on.
This property is read-only.
Fit fuel consumption through a randomized Gaussian kernel, which bends where a linear model cannot.
load carsmall X = [Acceleration, Displacement, Horsepower, Weight]; ok = ! any (isnan ([X, MPG]), 2); Mdl = RegressionKernel (X(ok,:), MPG(ok))
Mdl =
RegressionKernel
ResponseName: 'Y'
Learner: 'svm'
NumExpansionDimensions: 128
KernelScale: 1
Lambda: 0.0107527
BoxConstraint: 1
Epsilon: 0.926612
yFit = predict (Mdl, X(find (ok, 3),:))
yFit = 22.758 22.459 23.625
Standardizing matters more here than for a linear fit, since the kernel measures one distance across predictors of every scale.
load carsmall X = [Acceleration, Displacement, Horsepower, Weight]; ok = ! any (isnan ([X, MPG]), 2); plain = RegressionKernel (X(ok,:), MPG(ok)); scaled = RegressionKernel (X(ok,:), MPG(ok), 'Standardize', true); plainLoss = loss (plain, X(ok,:), MPG(ok))
plainLoss = 53.877
scaledLoss = loss (scaled, X(ok,:), MPG(ok))
scaledLoss = 23.171
A row vector of column indices, empty when every predictor is numeric. This property is read-only.
Fit fuel consumption through a randomized Gaussian kernel, which bends where a linear model cannot.
load carsmall X = [Acceleration, Displacement, Horsepower, Weight]; ok = ! any (isnan ([X, MPG]), 2); Mdl = RegressionKernel (X(ok,:), MPG(ok))
Mdl =
RegressionKernel
ResponseName: 'Y'
Learner: 'svm'
NumExpansionDimensions: 128
KernelScale: 1
Lambda: 0.0107527
BoxConstraint: 1
Epsilon: 0.926612
yFit = predict (Mdl, X(find (ok, 3),:))
yFit = 22.036 23.379 23.732
Standardizing matters more here than for a linear fit, since the kernel measures one distance across predictors of every scale.
load carsmall X = [Acceleration, Displacement, Horsepower, Weight]; ok = ! any (isnan ([X, MPG]), 2); plain = RegressionKernel (X(ok,:), MPG(ok)); scaled = RegressionKernel (X(ok,:), MPG(ok), 'Standardize', true); plainLoss = loss (plain, X(ok,:), MPG(ok))
plainLoss = 53.559
scaledLoss = loss (scaled, X(ok,:), MPG(ok))
scaledLoss = 22.007
A character vector, defaulting to 'Y'. This property is
read-only.
Fit fuel consumption through a randomized Gaussian kernel, which bends where a linear model cannot.
load carsmall X = [Acceleration, Displacement, Horsepower, Weight]; ok = ! any (isnan ([X, MPG]), 2); Mdl = RegressionKernel (X(ok,:), MPG(ok))
Mdl =
RegressionKernel
ResponseName: 'Y'
Learner: 'svm'
NumExpansionDimensions: 128
KernelScale: 1
Lambda: 0.0107527
BoxConstraint: 1
Epsilon: 0.926612
yFit = predict (Mdl, X(find (ok, 3),:))
yFit = 24.129 23.062 22.997
Standardizing matters more here than for a linear fit, since the kernel measures one distance across predictors of every scale.
load carsmall X = [Acceleration, Displacement, Horsepower, Weight]; ok = ! any (isnan ([X, MPG]), 2); plain = RegressionKernel (X(ok,:), MPG(ok)); scaled = RegressionKernel (X(ok,:), MPG(ok), 'Standardize', true); plainLoss = loss (plain, X(ok,:), MPG(ok))
plainLoss = 52.508
scaledLoss = loss (scaled, X(ok,:), MPG(ok))
scaledLoss = 22.387
A cell array of character vectors. These name the original predictors, not the expansion dimensions, which have no names. This property is read-only.
Fit fuel consumption through a randomized Gaussian kernel, which bends where a linear model cannot.
load carsmall X = [Acceleration, Displacement, Horsepower, Weight]; ok = ! any (isnan ([X, MPG]), 2); Mdl = RegressionKernel (X(ok,:), MPG(ok))
Mdl =
RegressionKernel
ResponseName: 'Y'
Learner: 'svm'
NumExpansionDimensions: 128
KernelScale: 1
Lambda: 0.0107527
BoxConstraint: 1
Epsilon: 0.926612
yFit = predict (Mdl, X(find (ok, 3),:))
yFit = 22.426 23.365 21.519
Standardizing matters more here than for a linear fit, since the kernel measures one distance across predictors of every scale.
load carsmall X = [Acceleration, Displacement, Horsepower, Weight]; ok = ! any (isnan ([X, MPG]), 2); plain = RegressionKernel (X(ok,:), MPG(ok)); scaled = RegressionKernel (X(ok,:), MPG(ok), 'Standardize', true); plainLoss = loss (plain, X(ok,:), MPG(ok))
plainLoss = 49.676
scaledLoss = loss (scaled, X(ok,:), MPG(ok))
scaledLoss = 21.637
A positive integer scalar. It defaults to
2 .^ ceil (min (log2 (p) + 5, 15)) for p
predictors, so four predictors give 128 dimensions. More dimensions
approximate the kernel more closely and cost proportionally more.
This property is read-only.
Fit fuel consumption through a randomized Gaussian kernel, which bends where a linear model cannot.
load carsmall X = [Acceleration, Displacement, Horsepower, Weight]; ok = ! any (isnan ([X, MPG]), 2); Mdl = RegressionKernel (X(ok,:), MPG(ok))
Mdl =
RegressionKernel
ResponseName: 'Y'
Learner: 'svm'
NumExpansionDimensions: 128
KernelScale: 1
Lambda: 0.0107527
BoxConstraint: 1
Epsilon: 0.926612
yFit = predict (Mdl, X(find (ok, 3),:))
yFit = 23.018 22.489 23.321
Standardizing matters more here than for a linear fit, since the kernel measures one distance across predictors of every scale.
load carsmall X = [Acceleration, Displacement, Horsepower, Weight]; ok = ! any (isnan ([X, MPG]), 2); plain = RegressionKernel (X(ok,:), MPG(ok)); scaled = RegressionKernel (X(ok,:), MPG(ok), 'Standardize', true); plainLoss = loss (plain, X(ok,:), MPG(ok))
plainLoss = 53.578
scaledLoss = loss (scaled, X(ok,:), MPG(ok))
scaledLoss = 22.769
'epsiloninsensitive' for a support vector machine and
'mse' for a least squares fit. This property is read-only.
Fit fuel consumption through a randomized Gaussian kernel, which bends where a linear model cannot.
load carsmall X = [Acceleration, Displacement, Horsepower, Weight]; ok = ! any (isnan ([X, MPG]), 2); Mdl = RegressionKernel (X(ok,:), MPG(ok))
Mdl =
RegressionKernel
ResponseName: 'Y'
Learner: 'svm'
NumExpansionDimensions: 128
KernelScale: 1
Lambda: 0.0107527
BoxConstraint: 1
Epsilon: 0.926612
yFit = predict (Mdl, X(find (ok, 3),:))
yFit = 21.780 22.993 22.139
Standardizing matters more here than for a linear fit, since the kernel measures one distance across predictors of every scale.
load carsmall X = [Acceleration, Displacement, Horsepower, Weight]; ok = ! any (isnan ([X, MPG]), 2); plain = RegressionKernel (X(ok,:), MPG(ok)); scaled = RegressionKernel (X(ok,:), MPG(ok), 'Standardize', true); plainLoss = loss (plain, X(ok,:), MPG(ok))
plainLoss = 48.556
scaledLoss = loss (scaled, X(ok,:), MPG(ok))
scaledLoss = 20.058
A nonnegative scalar, the reciprocal of the product of
BoxConstraint and the number of observations. This property
is read-only.
Fit fuel consumption through a randomized Gaussian kernel, which bends where a linear model cannot.
load carsmall X = [Acceleration, Displacement, Horsepower, Weight]; ok = ! any (isnan ([X, MPG]), 2); Mdl = RegressionKernel (X(ok,:), MPG(ok))
Mdl =
RegressionKernel
ResponseName: 'Y'
Learner: 'svm'
NumExpansionDimensions: 128
KernelScale: 1
Lambda: 0.0107527
BoxConstraint: 1
Epsilon: 0.926612
yFit = predict (Mdl, X(find (ok, 3),:))
yFit = 23.416 23.821 21.862
Standardizing matters more here than for a linear fit, since the kernel measures one distance across predictors of every scale.
load carsmall X = [Acceleration, Displacement, Horsepower, Weight]; ok = ! any (isnan ([X, MPG]), 2); plain = RegressionKernel (X(ok,:), MPG(ok)); scaled = RegressionKernel (X(ok,:), MPG(ok), 'Standardize', true); plainLoss = loss (plain, X(ok,:), MPG(ok))
plainLoss = 55.310
scaledLoss = loss (scaled, X(ok,:), MPG(ok))
scaledLoss = 21.929
A structure holding every parameter of the fit, with the
'auto' values as they were given rather than as they were
resolved. This property is read-only.
Fit fuel consumption through a randomized Gaussian kernel, which bends where a linear model cannot.
load carsmall X = [Acceleration, Displacement, Horsepower, Weight]; ok = ! any (isnan ([X, MPG]), 2); Mdl = RegressionKernel (X(ok,:), MPG(ok))
Mdl =
RegressionKernel
ResponseName: 'Y'
Learner: 'svm'
NumExpansionDimensions: 128
KernelScale: 1
Lambda: 0.0107527
BoxConstraint: 1
Epsilon: 0.926612
yFit = predict (Mdl, X(find (ok, 3),:))
yFit = 23.422 22.644 22.804
Standardizing matters more here than for a linear fit, since the kernel measures one distance across predictors of every scale.
load carsmall X = [Acceleration, Displacement, Horsepower, Weight]; ok = ! any (isnan ([X, MPG]), 2); plain = RegressionKernel (X(ok,:), MPG(ok)); scaled = RegressionKernel (X(ok,:), MPG(ok), 'Standardize', true); plainLoss = loss (plain, X(ok,:), MPG(ok))
plainLoss = 52.004
scaledLoss = loss (scaled, X(ok,:), MPG(ok))
scaledLoss = 22.005
Always 'ridge (L2)': a kernel model fits in the expanded
space, where a lasso penalty has nothing to select. This property is
read-only.
Fit fuel consumption through a randomized Gaussian kernel, which bends where a linear model cannot.
load carsmall X = [Acceleration, Displacement, Horsepower, Weight]; ok = ! any (isnan ([X, MPG]), 2); Mdl = RegressionKernel (X(ok,:), MPG(ok))
Mdl =
RegressionKernel
ResponseName: 'Y'
Learner: 'svm'
NumExpansionDimensions: 128
KernelScale: 1
Lambda: 0.0107527
BoxConstraint: 1
Epsilon: 0.926612
yFit = predict (Mdl, X(find (ok, 3),:))
yFit = 23.000 23.145 22.676
Standardizing matters more here than for a linear fit, since the kernel measures one distance across predictors of every scale.
load carsmall X = [Acceleration, Displacement, Horsepower, Weight]; ok = ! any (isnan ([X, MPG]), 2); plain = RegressionKernel (X(ok,:), MPG(ok)); scaled = RegressionKernel (X(ok,:), MPG(ok), 'Standardize', true); plainLoss = loss (plain, X(ok,:), MPG(ok))
plainLoss = 53.450
scaledLoss = loss (scaled, X(ok,:), MPG(ok))
scaledLoss = 22.402
A positive scalar dividing every predictor before the expansion, so a larger scale makes the kernel wider and the fit smoother. This property is read-only.
Fit fuel consumption through a randomized Gaussian kernel, which bends where a linear model cannot.
load carsmall X = [Acceleration, Displacement, Horsepower, Weight]; ok = ! any (isnan ([X, MPG]), 2); Mdl = RegressionKernel (X(ok,:), MPG(ok))
Mdl =
RegressionKernel
ResponseName: 'Y'
Learner: 'svm'
NumExpansionDimensions: 128
KernelScale: 1
Lambda: 0.0107527
BoxConstraint: 1
Epsilon: 0.926612
yFit = predict (Mdl, X(find (ok, 3),:))
yFit = 23.135 22.648 22.633
Standardizing matters more here than for a linear fit, since the kernel measures one distance across predictors of every scale.
load carsmall X = [Acceleration, Displacement, Horsepower, Weight]; ok = ! any (isnan ([X, MPG]), 2); plain = RegressionKernel (X(ok,:), MPG(ok)); scaled = RegressionKernel (X(ok,:), MPG(ok), 'Standardize', true); plainLoss = loss (plain, X(ok,:), MPG(ok))
plainLoss = 54.153
scaledLoss = loss (scaled, X(ok,:), MPG(ok))
scaledLoss = 21.564
Either 'svm' or 'leastsquares'. This property is
read-only.
Fit fuel consumption through a randomized Gaussian kernel, which bends where a linear model cannot.
load carsmall X = [Acceleration, Displacement, Horsepower, Weight]; ok = ! any (isnan ([X, MPG]), 2); Mdl = RegressionKernel (X(ok,:), MPG(ok))
Mdl =
RegressionKernel
ResponseName: 'Y'
Learner: 'svm'
NumExpansionDimensions: 128
KernelScale: 1
Lambda: 0.0107527
BoxConstraint: 1
Epsilon: 0.926612
yFit = predict (Mdl, X(find (ok, 3),:))
yFit = 23.319 21.232 22.942
Standardizing matters more here than for a linear fit, since the kernel measures one distance across predictors of every scale.
load carsmall X = [Acceleration, Displacement, Horsepower, Weight]; ok = ! any (isnan ([X, MPG]), 2); plain = RegressionKernel (X(ok,:), MPG(ok)); scaled = RegressionKernel (X(ok,:), MPG(ok), 'Standardize', true); plainLoss = loss (plain, X(ok,:), MPG(ok))
plainLoss = 53.769
scaledLoss = loss (scaled, X(ok,:), MPG(ok))
scaledLoss = 23.937
A row vector with one element per predictor, or empty when the model was fitted without standardizing. This property is read-only.
Fit fuel consumption through a randomized Gaussian kernel, which bends where a linear model cannot.
load carsmall X = [Acceleration, Displacement, Horsepower, Weight]; ok = ! any (isnan ([X, MPG]), 2); Mdl = RegressionKernel (X(ok,:), MPG(ok))
Mdl =
RegressionKernel
ResponseName: 'Y'
Learner: 'svm'
NumExpansionDimensions: 128
KernelScale: 1
Lambda: 0.0107527
BoxConstraint: 1
Epsilon: 0.926612
yFit = predict (Mdl, X(find (ok, 3),:))
yFit = 23.696 23.530 22.987
Standardizing matters more here than for a linear fit, since the kernel measures one distance across predictors of every scale.
load carsmall X = [Acceleration, Displacement, Horsepower, Weight]; ok = ! any (isnan ([X, MPG]), 2); plain = RegressionKernel (X(ok,:), MPG(ok)); scaled = RegressionKernel (X(ok,:), MPG(ok), 'Standardize', true); plainLoss = loss (plain, X(ok,:), MPG(ok))
plainLoss = 55.160
scaledLoss = loss (scaled, X(ok,:), MPG(ok))
scaledLoss = 21.072
A row vector with one element per predictor, or empty when the model was fitted without standardizing. This property is read-only.
Fit fuel consumption through a randomized Gaussian kernel, which bends where a linear model cannot.
load carsmall X = [Acceleration, Displacement, Horsepower, Weight]; ok = ! any (isnan ([X, MPG]), 2); Mdl = RegressionKernel (X(ok,:), MPG(ok))
Mdl =
RegressionKernel
ResponseName: 'Y'
Learner: 'svm'
NumExpansionDimensions: 128
KernelScale: 1
Lambda: 0.0107527
BoxConstraint: 1
Epsilon: 0.926612
yFit = predict (Mdl, X(find (ok, 3),:))
yFit = 23.633 22.562 24.034
Standardizing matters more here than for a linear fit, since the kernel measures one distance across predictors of every scale.
load carsmall X = [Acceleration, Displacement, Horsepower, Weight]; ok = ! any (isnan ([X, MPG]), 2); plain = RegressionKernel (X(ok,:), MPG(ok)); scaled = RegressionKernel (X(ok,:), MPG(ok), 'Standardize', true); plainLoss = loss (plain, X(ok,:), MPG(ok))
plainLoss = 54.039
scaledLoss = loss (scaled, X(ok,:), MPG(ok))
scaledLoss = 22.848
The RegressionKernel class offers the following public methods:
RegressionKernel: obj = RegressionKernel (X, Y)
RegressionKernel: obj = RegressionKernel (…, name, value)
obj = RegressionKernel (X, Y) fits a support
vector machine in a randomized Gaussian kernel space to the
predictor matrix X and the continuous response
Y.
obj = RegressionKernel (…, name, value)
takes the following Name-Value pairs.
| Name | Value |
|---|---|
'Learner' | 'svm', the default, or
'leastsquares'. |
'Epsilon' | Half the width of the insensitive band,
a nonnegative scalar or 'auto', which is the interquartile
range of Y over 13.49. It applies to a support vector machine
alone. |
'NumExpansionDimensions' | 'auto', the
default, or a positive integer. |
'KernelScale' | 1 by default, a positive
scalar, or 'auto', which takes the median distance between the
observations. |
'Lambda' | 'auto', the default, which is the
reciprocal of the number of observations, or a nonnegative scalar. It
cannot be given beside 'BoxConstraint'. |
'BoxConstraint' | A positive scalar, 1 by
default. It applies to a support vector machine alone. |
'Standardize' | Whether to centre and scale the predictors, false by default. |
'BetaTolerance' | Relative tolerance on the
coefficients, 1e-4 by default. |
'GradientTolerance' | Absolute tolerance on the
gradient’s infinity norm, 1e-6 by default. |
'IterationLimit' | Largest number of iterations,
1000 by default. |
'HessianHistorySize' | Number of curvature pairs the
solver keeps, 15 by default. |
'BlockSize' | Memory the expansion may occupy, in
megabytes, 4e3 by default. |
'ResponseTransform' | A transformation applied to the predicted response, named or given as a function handle. |
'Weights' | One nonnegative weight per observation. |
'PredictorNames' | One name per predictor. |
'ResponseName' | A name for the response. |
'CategoricalPredictors' | Indices of the categorical predictors. |
The fit is always by limited-memory BFGS, the only solver MATLAB offers a kernel model, and always under a ridge penalty.
See also: fitrkernel, RegressionLinear
Fit fuel consumption through a randomized Gaussian kernel, which bends where a linear model cannot.
load carsmall X = [Acceleration, Displacement, Horsepower, Weight]; ok = ! any (isnan ([X, MPG]), 2); Mdl = RegressionKernel (X(ok,:), MPG(ok))
Mdl =
RegressionKernel
ResponseName: 'Y'
Learner: 'svm'
NumExpansionDimensions: 128
KernelScale: 1
Lambda: 0.0107527
BoxConstraint: 1
Epsilon: 0.926612
yFit = predict (Mdl, X(find (ok, 3),:))
yFit = 22.811 23.534 24.161
Standardizing matters more here than for a linear fit, since the kernel measures one distance across predictors of every scale.
load carsmall X = [Acceleration, Displacement, Horsepower, Weight]; ok = ! any (isnan ([X, MPG]), 2); plain = RegressionKernel (X(ok,:), MPG(ok)); scaled = RegressionKernel (X(ok,:), MPG(ok), 'Standardize', true); plainLoss = loss (plain, X(ok,:), MPG(ok))
plainLoss = 52.145
scaledLoss = loss (scaled, X(ok,:), MPG(ok))
scaledLoss = 21.056
RegressionKernel: yFit = predict (obj, XC)
yFit = predict (obj, XC) maps each row of
XC through the model’s own random basis and returns the
predicted response, with ResponseTransform applied.
Fit fuel consumption through a randomized Gaussian kernel, which bends where a linear model cannot.
load carsmall X = [Acceleration, Displacement, Horsepower, Weight]; ok = ! any (isnan ([X, MPG]), 2); Mdl = RegressionKernel (X(ok,:), MPG(ok))
Mdl =
RegressionKernel
ResponseName: 'Y'
Learner: 'svm'
NumExpansionDimensions: 128
KernelScale: 1
Lambda: 0.0107527
BoxConstraint: 1
Epsilon: 0.926612
yFit = predict (Mdl, X(find (ok, 3),:))
yFit = 23.278 22.362 24.992
Standardizing matters more here than for a linear fit, since the kernel measures one distance across predictors of every scale.
load carsmall X = [Acceleration, Displacement, Horsepower, Weight]; ok = ! any (isnan ([X, MPG]), 2); plain = RegressionKernel (X(ok,:), MPG(ok)); scaled = RegressionKernel (X(ok,:), MPG(ok), 'Standardize', true); plainLoss = loss (plain, X(ok,:), MPG(ok))
plainLoss = 52.631
scaledLoss = loss (scaled, X(ok,:), MPG(ok))
scaledLoss = 20.993
RegressionKernel: l = loss (obj, X, Y)
RegressionKernel: l = loss (…, name, value)
l = loss (obj, X, Y) returns the mean
squared error.
l = loss (…, name, value) takes
'LossFun', either 'mse' or
'epsiloninsensitive', and 'Weights'. The
epsilon-insensitive loss needs a band to be insensitive within, so it
is offered by a support vector machine alone.
Fit fuel consumption through a randomized Gaussian kernel, which bends where a linear model cannot.
load carsmall X = [Acceleration, Displacement, Horsepower, Weight]; ok = ! any (isnan ([X, MPG]), 2); Mdl = RegressionKernel (X(ok,:), MPG(ok))
Mdl =
RegressionKernel
ResponseName: 'Y'
Learner: 'svm'
NumExpansionDimensions: 128
KernelScale: 1
Lambda: 0.0107527
BoxConstraint: 1
Epsilon: 0.926612
yFit = predict (Mdl, X(find (ok, 3),:))
yFit = 22.756 23.566 23.276
Standardizing matters more here than for a linear fit, since the kernel measures one distance across predictors of every scale.
load carsmall X = [Acceleration, Displacement, Horsepower, Weight]; ok = ! any (isnan ([X, MPG]), 2); plain = RegressionKernel (X(ok,:), MPG(ok)); scaled = RegressionKernel (X(ok,:), MPG(ok), 'Standardize', true); plainLoss = loss (plain, X(ok,:), MPG(ok))
plainLoss = 53.219
scaledLoss = loss (scaled, X(ok,:), MPG(ok))
scaledLoss = 23.255
RegressionKernel: obj = resume (obj, X, Y)
RegressionKernel: obj = resume (…, name, value)
obj = resume (obj, X, Y) restarts the
optimization from the coefficients the model already carries, through
the basis it already holds. It takes 'BetaTolerance',
'GradientTolerance' and 'IterationLimit', each
defaulting to what the model was fitted with, and 'Weights'.
X and Y must be the data the model was fitted to; the
object keeps no copy of them, which is what makes it small. Neither
does it keep the observation weights, so a model fitted with
'Weights' must be given them again here or it will resume
against uniform ones. MATLAB behaves the same way: measured on
R2024a, resuming a weighted fit without passing the weights back
reaches the objective of the unweighted fit.
Fit fuel consumption through a randomized Gaussian kernel, which bends where a linear model cannot.
load carsmall X = [Acceleration, Displacement, Horsepower, Weight]; ok = ! any (isnan ([X, MPG]), 2); Mdl = RegressionKernel (X(ok,:), MPG(ok))
Mdl =
RegressionKernel
ResponseName: 'Y'
Learner: 'svm'
NumExpansionDimensions: 128
KernelScale: 1
Lambda: 0.0107527
BoxConstraint: 1
Epsilon: 0.926612
yFit = predict (Mdl, X(find (ok, 3),:))
yFit = 22.472 21.644 25.091
Standardizing matters more here than for a linear fit, since the kernel measures one distance across predictors of every scale.
load carsmall X = [Acceleration, Displacement, Horsepower, Weight]; ok = ! any (isnan ([X, MPG]), 2); plain = RegressionKernel (X(ok,:), MPG(ok)); scaled = RegressionKernel (X(ok,:), MPG(ok), 'Standardize', true); plainLoss = loss (plain, X(ok,:), MPG(ok))
plainLoss = 52.104
scaledLoss = loss (scaled, X(ok,:), MPG(ok))
scaledLoss = 23.556
RegressionKernel: savemodel (obj, filename)
savemodel (obj, filename) saves the model
obj into filename in a form loadmodel can read
back, the random basis included.
Fit fuel consumption through a randomized Gaussian kernel, which bends where a linear model cannot.
load carsmall X = [Acceleration, Displacement, Horsepower, Weight]; ok = ! any (isnan ([X, MPG]), 2); Mdl = RegressionKernel (X(ok,:), MPG(ok))
Mdl =
RegressionKernel
ResponseName: 'Y'
Learner: 'svm'
NumExpansionDimensions: 128
KernelScale: 1
Lambda: 0.0107527
BoxConstraint: 1
Epsilon: 0.926612
yFit = predict (Mdl, X(find (ok, 3),:))
yFit = 23.636 23.646 21.274
Standardizing matters more here than for a linear fit, since the kernel measures one distance across predictors of every scale.
load carsmall X = [Acceleration, Displacement, Horsepower, Weight]; ok = ! any (isnan ([X, MPG]), 2); plain = RegressionKernel (X(ok,:), MPG(ok)); scaled = RegressionKernel (X(ok,:), MPG(ok), 'Standardize', true); plainLoss = loss (plain, X(ok,:), MPG(ok))
plainLoss = 51.865
scaledLoss = loss (scaled, X(ok,:), MPG(ok))
scaledLoss = 25.308
load carsmall X = [Acceleration, Displacement, Horsepower, Weight]; ok = ! any (isnan ([X, MPG]), 2); Mdl = RegressionKernel (X(ok,:), MPG(ok))
Mdl =
RegressionKernel
ResponseName: 'Y'
Learner: 'svm'
NumExpansionDimensions: 128
KernelScale: 1
Lambda: 0.0107527
BoxConstraint: 1
Epsilon: 0.926612
yFit = predict (Mdl, X(find (ok, 3),:))
yFit = 23.150 23.023 23.461
load carsmall X = [Acceleration, Displacement, Horsepower, Weight]; ok = ! any (isnan ([X, MPG]), 2); plain = RegressionKernel (X(ok,:), MPG(ok)); scaled = RegressionKernel (X(ok,:), MPG(ok), 'Standardize', true); plainLoss = loss (plain, X(ok,:), MPG(ok))
plainLoss = 53.330
scaledLoss = loss (scaled, X(ok,:), MPG(ok))
scaledLoss = 25.265