Categories &

Functions List

Class Definition: RegressionKernel

statistics: RegressionKernel

Gaussian kernel regression model for large data.

A RegressionKernel object maps the predictors into a randomized feature space whose inner product approximates a Gaussian kernel, and then fits a linear model there. A kernel regression is therefore as nonlinear as a support vector machine with a Gaussian kernel, while costing what a linear fit costs: nothing of size NxN is ever formed.

The expansion is the random Fourier basis of Rahimi and Recht, drawn once when the model is fitted and kept with it, so predict maps new data through the same basis. MATLAB approximates the same kernel by the Fastfood construction, which reaches the same distribution more cheaply; the two are interchangeable in distribution but not draw by draw, and the draws come from different generators in any case, so the predictions of a model fitted here and one fitted in MATLAB differ even from the same seed. What does not differ is what they estimate.

Like RegressionLinear the object holds no copy of the training data. It does hold the basis and the coefficients, so it is bounded by the number of expansion dimensions rather than by the number of observations.

Create a RegressionKernel object with fitrkernel.

See also: fitrkernel, RegressionLinear, RegressionSVM

Source Code: RegressionKernel

The RegressionKernel class contains the following properties:

A nonnegative scalar for a support vector machine, and empty for a least squares fit, which has no such band. It defaults to the interquartile range of the response over 13.49, an estimate of its standard deviation, or to 0.1 when that range is zero. This property is read-only.

Fit fuel consumption through a randomized Gaussian kernel, which bends where a linear model cannot.

 load carsmall
 X = [Acceleration, Displacement, Horsepower, Weight];
 ok = ! any (isnan ([X, MPG]), 2);
 Mdl = RegressionKernel (X(ok,:), MPG(ok))
Mdl =

  RegressionKernel

              ResponseName: 'Y'
                   Learner: 'svm'
    NumExpansionDimensions: 128
               KernelScale: 1
                    Lambda: 0.0107527
             BoxConstraint: 1
                   Epsilon: 0.926612
 yFit = predict (Mdl, X(find (ok, 3),:))
yFit =

   23.480
   21.409
   23.007

Standardizing matters more here than for a linear fit, since the kernel measures one distance across predictors of every scale.

 load carsmall
 X = [Acceleration, Displacement, Horsepower, Weight];
 ok = ! any (isnan ([X, MPG]), 2);
 plain = RegressionKernel (X(ok,:), MPG(ok));
 scaled = RegressionKernel (X(ok,:), MPG(ok), 'Standardize', true);
 plainLoss = loss (plain, X(ok,:), MPG(ok))
plainLoss = 51.901
 scaledLoss = loss (scaled, X(ok,:), MPG(ok))
scaledLoss = 19.795

A positive scalar. It is the reciprocal of the product of Lambda and the number of observations, so setting either of the two in the constructor fixes the other, and giving both is an error. This property is read-only.

Fit fuel consumption through a randomized Gaussian kernel, which bends where a linear model cannot.

 load carsmall
 X = [Acceleration, Displacement, Horsepower, Weight];
 ok = ! any (isnan ([X, MPG]), 2);
 Mdl = RegressionKernel (X(ok,:), MPG(ok))
Mdl =

  RegressionKernel

              ResponseName: 'Y'
                   Learner: 'svm'
    NumExpansionDimensions: 128
               KernelScale: 1
                    Lambda: 0.0107527
             BoxConstraint: 1
                   Epsilon: 0.926612
 yFit = predict (Mdl, X(find (ok, 3),:))
yFit =

   22.496
   21.518
   23.252

Standardizing matters more here than for a linear fit, since the kernel measures one distance across predictors of every scale.

 load carsmall
 X = [Acceleration, Displacement, Horsepower, Weight];
 ok = ! any (isnan ([X, MPG]), 2);
 plain = RegressionKernel (X(ok,:), MPG(ok));
 scaled = RegressionKernel (X(ok,:), MPG(ok), 'Standardize', true);
 plainLoss = loss (plain, X(ok,:), MPG(ok))
plainLoss = 49.214
 scaledLoss = loss (scaled, X(ok,:), MPG(ok))
scaledLoss = 21.000

A character vector, or the text of the function handle that was supplied. Assigning to it accepts either.

Fit fuel consumption through a randomized Gaussian kernel, which bends where a linear model cannot.

 load carsmall
 X = [Acceleration, Displacement, Horsepower, Weight];
 ok = ! any (isnan ([X, MPG]), 2);
 Mdl = RegressionKernel (X(ok,:), MPG(ok))
Mdl =

  RegressionKernel

              ResponseName: 'Y'
                   Learner: 'svm'
    NumExpansionDimensions: 128
               KernelScale: 1
                    Lambda: 0.0107527
             BoxConstraint: 1
                   Epsilon: 0.926612
 yFit = predict (Mdl, X(find (ok, 3),:))
yFit =

   22.320
   23.148
   23.156

Standardizing matters more here than for a linear fit, since the kernel measures one distance across predictors of every scale.

 load carsmall
 X = [Acceleration, Displacement, Horsepower, Weight];
 ok = ! any (isnan ([X, MPG]), 2);
 plain = RegressionKernel (X(ok,:), MPG(ok));
 scaled = RegressionKernel (X(ok,:), MPG(ok), 'Standardize', true);
 plainLoss = loss (plain, X(ok,:), MPG(ok))
plainLoss = 53.184
 scaledLoss = loss (scaled, X(ok,:), MPG(ok))
scaledLoss = 23.370

A cell array of character vectors with one name per column of the training data, defaulting to 'x1', 'x2' and so on. This property is read-only.

Fit fuel consumption through a randomized Gaussian kernel, which bends where a linear model cannot.

 load carsmall
 X = [Acceleration, Displacement, Horsepower, Weight];
 ok = ! any (isnan ([X, MPG]), 2);
 Mdl = RegressionKernel (X(ok,:), MPG(ok))
Mdl =

  RegressionKernel

              ResponseName: 'Y'
                   Learner: 'svm'
    NumExpansionDimensions: 128
               KernelScale: 1
                    Lambda: 0.0107527
             BoxConstraint: 1
                   Epsilon: 0.926612
 yFit = predict (Mdl, X(find (ok, 3),:))
yFit =

   22.106
   22.908
   22.965

Standardizing matters more here than for a linear fit, since the kernel measures one distance across predictors of every scale.

 load carsmall
 X = [Acceleration, Displacement, Horsepower, Weight];
 ok = ! any (isnan ([X, MPG]), 2);
 plain = RegressionKernel (X(ok,:), MPG(ok));
 scaled = RegressionKernel (X(ok,:), MPG(ok), 'Standardize', true);
 plainLoss = loss (plain, X(ok,:), MPG(ok))
plainLoss = 54.654
 scaledLoss = loss (scaled, X(ok,:), MPG(ok))
scaledLoss = 26.526

A row vector of column indices, empty when every predictor is numeric. This property is read-only.

Fit fuel consumption through a randomized Gaussian kernel, which bends where a linear model cannot.

 load carsmall
 X = [Acceleration, Displacement, Horsepower, Weight];
 ok = ! any (isnan ([X, MPG]), 2);
 Mdl = RegressionKernel (X(ok,:), MPG(ok))
Mdl =

  RegressionKernel

              ResponseName: 'Y'
                   Learner: 'svm'
    NumExpansionDimensions: 128
               KernelScale: 1
                    Lambda: 0.0107527
             BoxConstraint: 1
                   Epsilon: 0.926612
 yFit = predict (Mdl, X(find (ok, 3),:))
yFit =

   22.568
   23.098
   24.415

Standardizing matters more here than for a linear fit, since the kernel measures one distance across predictors of every scale.

 load carsmall
 X = [Acceleration, Displacement, Horsepower, Weight];
 ok = ! any (isnan ([X, MPG]), 2);
 plain = RegressionKernel (X(ok,:), MPG(ok));
 scaled = RegressionKernel (X(ok,:), MPG(ok), 'Standardize', true);
 plainLoss = loss (plain, X(ok,:), MPG(ok))
plainLoss = 52.638
 scaledLoss = loss (scaled, X(ok,:), MPG(ok))
scaledLoss = 18.782

A character vector, defaulting to 'Y'. This property is read-only.

Fit fuel consumption through a randomized Gaussian kernel, which bends where a linear model cannot.

 load carsmall
 X = [Acceleration, Displacement, Horsepower, Weight];
 ok = ! any (isnan ([X, MPG]), 2);
 Mdl = RegressionKernel (X(ok,:), MPG(ok))
Mdl =

  RegressionKernel

              ResponseName: 'Y'
                   Learner: 'svm'
    NumExpansionDimensions: 128
               KernelScale: 1
                    Lambda: 0.0107527
             BoxConstraint: 1
                   Epsilon: 0.926612
 yFit = predict (Mdl, X(find (ok, 3),:))
yFit =

   21.282
   23.375
   22.292

Standardizing matters more here than for a linear fit, since the kernel measures one distance across predictors of every scale.

 load carsmall
 X = [Acceleration, Displacement, Horsepower, Weight];
 ok = ! any (isnan ([X, MPG]), 2);
 plain = RegressionKernel (X(ok,:), MPG(ok));
 scaled = RegressionKernel (X(ok,:), MPG(ok), 'Standardize', true);
 plainLoss = loss (plain, X(ok,:), MPG(ok))
plainLoss = 54.459
 scaledLoss = loss (scaled, X(ok,:), MPG(ok))
scaledLoss = 19.831

A cell array of character vectors. These name the original predictors, not the expansion dimensions, which have no names. This property is read-only.

Fit fuel consumption through a randomized Gaussian kernel, which bends where a linear model cannot.

 load carsmall
 X = [Acceleration, Displacement, Horsepower, Weight];
 ok = ! any (isnan ([X, MPG]), 2);
 Mdl = RegressionKernel (X(ok,:), MPG(ok))
Mdl =

  RegressionKernel

              ResponseName: 'Y'
                   Learner: 'svm'
    NumExpansionDimensions: 128
               KernelScale: 1
                    Lambda: 0.0107527
             BoxConstraint: 1
                   Epsilon: 0.926612
 yFit = predict (Mdl, X(find (ok, 3),:))
yFit =

   21.821
   23.219
   24.124

Standardizing matters more here than for a linear fit, since the kernel measures one distance across predictors of every scale.

 load carsmall
 X = [Acceleration, Displacement, Horsepower, Weight];
 ok = ! any (isnan ([X, MPG]), 2);
 plain = RegressionKernel (X(ok,:), MPG(ok));
 scaled = RegressionKernel (X(ok,:), MPG(ok), 'Standardize', true);
 plainLoss = loss (plain, X(ok,:), MPG(ok))
plainLoss = 53.794
 scaledLoss = loss (scaled, X(ok,:), MPG(ok))
scaledLoss = 22.914

A positive integer scalar. It defaults to 2 .^ ceil (min (log2 (p) + 5, 15)) for p predictors, so four predictors give 128 dimensions. More dimensions approximate the kernel more closely and cost proportionally more. This property is read-only.

Fit fuel consumption through a randomized Gaussian kernel, which bends where a linear model cannot.

 load carsmall
 X = [Acceleration, Displacement, Horsepower, Weight];
 ok = ! any (isnan ([X, MPG]), 2);
 Mdl = RegressionKernel (X(ok,:), MPG(ok))
Mdl =

  RegressionKernel

              ResponseName: 'Y'
                   Learner: 'svm'
    NumExpansionDimensions: 128
               KernelScale: 1
                    Lambda: 0.0107527
             BoxConstraint: 1
                   Epsilon: 0.926612
 yFit = predict (Mdl, X(find (ok, 3),:))
yFit =

   23.243
   22.873
   22.677

Standardizing matters more here than for a linear fit, since the kernel measures one distance across predictors of every scale.

 load carsmall
 X = [Acceleration, Displacement, Horsepower, Weight];
 ok = ! any (isnan ([X, MPG]), 2);
 plain = RegressionKernel (X(ok,:), MPG(ok));
 scaled = RegressionKernel (X(ok,:), MPG(ok), 'Standardize', true);
 plainLoss = loss (plain, X(ok,:), MPG(ok))
plainLoss = 52.916
 scaledLoss = loss (scaled, X(ok,:), MPG(ok))
scaledLoss = 21.089

'epsiloninsensitive' for a support vector machine and 'mse' for a least squares fit. This property is read-only.

Fit fuel consumption through a randomized Gaussian kernel, which bends where a linear model cannot.

 load carsmall
 X = [Acceleration, Displacement, Horsepower, Weight];
 ok = ! any (isnan ([X, MPG]), 2);
 Mdl = RegressionKernel (X(ok,:), MPG(ok))
Mdl =

  RegressionKernel

              ResponseName: 'Y'
                   Learner: 'svm'
    NumExpansionDimensions: 128
               KernelScale: 1
                    Lambda: 0.0107527
             BoxConstraint: 1
                   Epsilon: 0.926612
 yFit = predict (Mdl, X(find (ok, 3),:))
yFit =

   21.997
   22.694
   22.593

Standardizing matters more here than for a linear fit, since the kernel measures one distance across predictors of every scale.

 load carsmall
 X = [Acceleration, Displacement, Horsepower, Weight];
 ok = ! any (isnan ([X, MPG]), 2);
 plain = RegressionKernel (X(ok,:), MPG(ok));
 scaled = RegressionKernel (X(ok,:), MPG(ok), 'Standardize', true);
 plainLoss = loss (plain, X(ok,:), MPG(ok))
plainLoss = 55.543
 scaledLoss = loss (scaled, X(ok,:), MPG(ok))
scaledLoss = 23.648

A nonnegative scalar, the reciprocal of the product of BoxConstraint and the number of observations. This property is read-only.

Fit fuel consumption through a randomized Gaussian kernel, which bends where a linear model cannot.

 load carsmall
 X = [Acceleration, Displacement, Horsepower, Weight];
 ok = ! any (isnan ([X, MPG]), 2);
 Mdl = RegressionKernel (X(ok,:), MPG(ok))
Mdl =

  RegressionKernel

              ResponseName: 'Y'
                   Learner: 'svm'
    NumExpansionDimensions: 128
               KernelScale: 1
                    Lambda: 0.0107527
             BoxConstraint: 1
                   Epsilon: 0.926612
 yFit = predict (Mdl, X(find (ok, 3),:))
yFit =

   21.591
   22.307
   22.691

Standardizing matters more here than for a linear fit, since the kernel measures one distance across predictors of every scale.

 load carsmall
 X = [Acceleration, Displacement, Horsepower, Weight];
 ok = ! any (isnan ([X, MPG]), 2);
 plain = RegressionKernel (X(ok,:), MPG(ok));
 scaled = RegressionKernel (X(ok,:), MPG(ok), 'Standardize', true);
 plainLoss = loss (plain, X(ok,:), MPG(ok))
plainLoss = 53.853
 scaledLoss = loss (scaled, X(ok,:), MPG(ok))
scaledLoss = 26.150

A structure holding every parameter of the fit, with the 'auto' values as they were given rather than as they were resolved. This property is read-only.

Fit fuel consumption through a randomized Gaussian kernel, which bends where a linear model cannot.

 load carsmall
 X = [Acceleration, Displacement, Horsepower, Weight];
 ok = ! any (isnan ([X, MPG]), 2);
 Mdl = RegressionKernel (X(ok,:), MPG(ok))
Mdl =

  RegressionKernel

              ResponseName: 'Y'
                   Learner: 'svm'
    NumExpansionDimensions: 128
               KernelScale: 1
                    Lambda: 0.0107527
             BoxConstraint: 1
                   Epsilon: 0.926612
 yFit = predict (Mdl, X(find (ok, 3),:))
yFit =

   24.141
   22.958
   24.771

Standardizing matters more here than for a linear fit, since the kernel measures one distance across predictors of every scale.

 load carsmall
 X = [Acceleration, Displacement, Horsepower, Weight];
 ok = ! any (isnan ([X, MPG]), 2);
 plain = RegressionKernel (X(ok,:), MPG(ok));
 scaled = RegressionKernel (X(ok,:), MPG(ok), 'Standardize', true);
 plainLoss = loss (plain, X(ok,:), MPG(ok))
plainLoss = 52.413
 scaledLoss = loss (scaled, X(ok,:), MPG(ok))
scaledLoss = 20.475

Always 'ridge (L2)': a kernel model fits in the expanded space, where a lasso penalty has nothing to select. This property is read-only.

Fit fuel consumption through a randomized Gaussian kernel, which bends where a linear model cannot.

 load carsmall
 X = [Acceleration, Displacement, Horsepower, Weight];
 ok = ! any (isnan ([X, MPG]), 2);
 Mdl = RegressionKernel (X(ok,:), MPG(ok))
Mdl =

  RegressionKernel

              ResponseName: 'Y'
                   Learner: 'svm'
    NumExpansionDimensions: 128
               KernelScale: 1
                    Lambda: 0.0107527
             BoxConstraint: 1
                   Epsilon: 0.926612
 yFit = predict (Mdl, X(find (ok, 3),:))
yFit =

   22.712
   23.463
   23.169

Standardizing matters more here than for a linear fit, since the kernel measures one distance across predictors of every scale.

 load carsmall
 X = [Acceleration, Displacement, Horsepower, Weight];
 ok = ! any (isnan ([X, MPG]), 2);
 plain = RegressionKernel (X(ok,:), MPG(ok));
 scaled = RegressionKernel (X(ok,:), MPG(ok), 'Standardize', true);
 plainLoss = loss (plain, X(ok,:), MPG(ok))
plainLoss = 52.577
 scaledLoss = loss (scaled, X(ok,:), MPG(ok))
scaledLoss = 20.664

A positive scalar dividing every predictor before the expansion, so a larger scale makes the kernel wider and the fit smoother. This property is read-only.

Fit fuel consumption through a randomized Gaussian kernel, which bends where a linear model cannot.

 load carsmall
 X = [Acceleration, Displacement, Horsepower, Weight];
 ok = ! any (isnan ([X, MPG]), 2);
 Mdl = RegressionKernel (X(ok,:), MPG(ok))
Mdl =

  RegressionKernel

              ResponseName: 'Y'
                   Learner: 'svm'
    NumExpansionDimensions: 128
               KernelScale: 1
                    Lambda: 0.0107527
             BoxConstraint: 1
                   Epsilon: 0.926612
 yFit = predict (Mdl, X(find (ok, 3),:))
yFit =

   22.192
   21.910
   23.144

Standardizing matters more here than for a linear fit, since the kernel measures one distance across predictors of every scale.

 load carsmall
 X = [Acceleration, Displacement, Horsepower, Weight];
 ok = ! any (isnan ([X, MPG]), 2);
 plain = RegressionKernel (X(ok,:), MPG(ok));
 scaled = RegressionKernel (X(ok,:), MPG(ok), 'Standardize', true);
 plainLoss = loss (plain, X(ok,:), MPG(ok))
plainLoss = 53.588
 scaledLoss = loss (scaled, X(ok,:), MPG(ok))
scaledLoss = 23.104

Either 'svm' or 'leastsquares'. This property is read-only.

Fit fuel consumption through a randomized Gaussian kernel, which bends where a linear model cannot.

 load carsmall
 X = [Acceleration, Displacement, Horsepower, Weight];
 ok = ! any (isnan ([X, MPG]), 2);
 Mdl = RegressionKernel (X(ok,:), MPG(ok))
Mdl =

  RegressionKernel

              ResponseName: 'Y'
                   Learner: 'svm'
    NumExpansionDimensions: 128
               KernelScale: 1
                    Lambda: 0.0107527
             BoxConstraint: 1
                   Epsilon: 0.926612
 yFit = predict (Mdl, X(find (ok, 3),:))
yFit =

   22.500
   23.300
   23.381

Standardizing matters more here than for a linear fit, since the kernel measures one distance across predictors of every scale.

 load carsmall
 X = [Acceleration, Displacement, Horsepower, Weight];
 ok = ! any (isnan ([X, MPG]), 2);
 plain = RegressionKernel (X(ok,:), MPG(ok));
 scaled = RegressionKernel (X(ok,:), MPG(ok), 'Standardize', true);
 plainLoss = loss (plain, X(ok,:), MPG(ok))
plainLoss = 55.237
 scaledLoss = loss (scaled, X(ok,:), MPG(ok))
scaledLoss = 21.541

A row vector with one element per predictor, or empty when the model was fitted without standardizing. This property is read-only.

Fit fuel consumption through a randomized Gaussian kernel, which bends where a linear model cannot.

 load carsmall
 X = [Acceleration, Displacement, Horsepower, Weight];
 ok = ! any (isnan ([X, MPG]), 2);
 Mdl = RegressionKernel (X(ok,:), MPG(ok))
Mdl =

  RegressionKernel

              ResponseName: 'Y'
                   Learner: 'svm'
    NumExpansionDimensions: 128
               KernelScale: 1
                    Lambda: 0.0107527
             BoxConstraint: 1
                   Epsilon: 0.926612
 yFit = predict (Mdl, X(find (ok, 3),:))
yFit =

   22.612
   23.729
   21.930

Standardizing matters more here than for a linear fit, since the kernel measures one distance across predictors of every scale.

 load carsmall
 X = [Acceleration, Displacement, Horsepower, Weight];
 ok = ! any (isnan ([X, MPG]), 2);
 plain = RegressionKernel (X(ok,:), MPG(ok));
 scaled = RegressionKernel (X(ok,:), MPG(ok), 'Standardize', true);
 plainLoss = loss (plain, X(ok,:), MPG(ok))
plainLoss = 53.368
 scaledLoss = loss (scaled, X(ok,:), MPG(ok))
scaledLoss = 22.813

A row vector with one element per predictor, or empty when the model was fitted without standardizing. This property is read-only.

Fit fuel consumption through a randomized Gaussian kernel, which bends where a linear model cannot.

 load carsmall
 X = [Acceleration, Displacement, Horsepower, Weight];
 ok = ! any (isnan ([X, MPG]), 2);
 Mdl = RegressionKernel (X(ok,:), MPG(ok))
Mdl =

  RegressionKernel

              ResponseName: 'Y'
                   Learner: 'svm'
    NumExpansionDimensions: 128
               KernelScale: 1
                    Lambda: 0.0107527
             BoxConstraint: 1
                   Epsilon: 0.926612
 yFit = predict (Mdl, X(find (ok, 3),:))
yFit =

   23.498
   22.989
   23.542

Standardizing matters more here than for a linear fit, since the kernel measures one distance across predictors of every scale.

 load carsmall
 X = [Acceleration, Displacement, Horsepower, Weight];
 ok = ! any (isnan ([X, MPG]), 2);
 plain = RegressionKernel (X(ok,:), MPG(ok));
 scaled = RegressionKernel (X(ok,:), MPG(ok), 'Standardize', true);
 plainLoss = loss (plain, X(ok,:), MPG(ok))
plainLoss = 53.662
 scaledLoss = loss (scaled, X(ok,:), MPG(ok))
scaledLoss = 21.043

The RegressionKernel class offers the following public methods:

RegressionKernel: obj = RegressionKernel (X, Y)
RegressionKernel: obj = RegressionKernel (…, name, value)

obj = RegressionKernel (X, Y) fits a support vector machine in a randomized Gaussian kernel space to the NxP predictor matrix X and the Nx1 continuous response Y.

obj = RegressionKernel (…, name, value) takes the following Name-Value pairs.

NameValue
'Learner''svm', the default, or 'leastsquares'.
'Epsilon'Half the width of the insensitive band, a nonnegative scalar or 'auto', which is the interquartile range of Y over 13.49. It applies to a support vector machine alone.
'NumExpansionDimensions''auto', the default, or a positive integer.
'KernelScale'1 by default, a positive scalar, or 'auto', which takes the median distance between the observations.
'Lambda''auto', the default, which is the reciprocal of the number of observations, or a nonnegative scalar. It cannot be given beside 'BoxConstraint'.
'BoxConstraint'A positive scalar, 1 by default. It applies to a support vector machine alone.
'Standardize'Whether to centre and scale the predictors, false by default.
'BetaTolerance'Relative tolerance on the coefficients, 1e-4 by default.
'GradientTolerance'Absolute tolerance on the gradient’s infinity norm, 1e-6 by default.
'IterationLimit'Largest number of iterations, 1000 by default.
'HessianHistorySize'Number of curvature pairs the solver keeps, 15 by default.
'BlockSize'Memory the expansion may occupy, in megabytes, 4e3 by default.
'ResponseTransform'A transformation applied to the predicted response, named or given as a function handle.
'Weights'One nonnegative weight per observation.
'PredictorNames'One name per predictor.
'ResponseName'A name for the response.
'CategoricalPredictors'Indices of the categorical predictors.

The fit is always by limited-memory BFGS, the only solver MATLAB offers a kernel model, and always under a ridge penalty.

See also: fitrkernel, RegressionLinear

Fit fuel consumption through a randomized Gaussian kernel, which bends where a linear model cannot.

 load carsmall
 X = [Acceleration, Displacement, Horsepower, Weight];
 ok = ! any (isnan ([X, MPG]), 2);
 Mdl = RegressionKernel (X(ok,:), MPG(ok))
Mdl =

  RegressionKernel

              ResponseName: 'Y'
                   Learner: 'svm'
    NumExpansionDimensions: 128
               KernelScale: 1
                    Lambda: 0.0107527
             BoxConstraint: 1
                   Epsilon: 0.926612
 yFit = predict (Mdl, X(find (ok, 3),:))
yFit =

   22.756
   22.099
   22.283

Standardizing matters more here than for a linear fit, since the kernel measures one distance across predictors of every scale.

 load carsmall
 X = [Acceleration, Displacement, Horsepower, Weight];
 ok = ! any (isnan ([X, MPG]), 2);
 plain = RegressionKernel (X(ok,:), MPG(ok));
 scaled = RegressionKernel (X(ok,:), MPG(ok), 'Standardize', true);
 plainLoss = loss (plain, X(ok,:), MPG(ok))
plainLoss = 53.303
 scaledLoss = loss (scaled, X(ok,:), MPG(ok))
scaledLoss = 21.428
RegressionKernel: yFit = predict (obj, XC)

yFit = predict (obj, XC) maps each row of XC through the model’s own random basis and returns the predicted response, with ResponseTransform applied.

Fit fuel consumption through a randomized Gaussian kernel, which bends where a linear model cannot.

 load carsmall
 X = [Acceleration, Displacement, Horsepower, Weight];
 ok = ! any (isnan ([X, MPG]), 2);
 Mdl = RegressionKernel (X(ok,:), MPG(ok))
Mdl =

  RegressionKernel

              ResponseName: 'Y'
                   Learner: 'svm'
    NumExpansionDimensions: 128
               KernelScale: 1
                    Lambda: 0.0107527
             BoxConstraint: 1
                   Epsilon: 0.926612
 yFit = predict (Mdl, X(find (ok, 3),:))
yFit =

   22.144
   24.380
   23.412

Standardizing matters more here than for a linear fit, since the kernel measures one distance across predictors of every scale.

 load carsmall
 X = [Acceleration, Displacement, Horsepower, Weight];
 ok = ! any (isnan ([X, MPG]), 2);
 plain = RegressionKernel (X(ok,:), MPG(ok));
 scaled = RegressionKernel (X(ok,:), MPG(ok), 'Standardize', true);
 plainLoss = loss (plain, X(ok,:), MPG(ok))
plainLoss = 52.974
 scaledLoss = loss (scaled, X(ok,:), MPG(ok))
scaledLoss = 20.683
RegressionKernel: l = loss (obj, X, Y)
RegressionKernel: l = loss (…, name, value)

l = loss (obj, X, Y) returns the mean squared error.

l = loss (…, name, value) takes 'LossFun', either 'mse' or 'epsiloninsensitive', and 'Weights'. The epsilon-insensitive loss needs a band to be insensitive within, so it is offered by a support vector machine alone.

Fit fuel consumption through a randomized Gaussian kernel, which bends where a linear model cannot.

 load carsmall
 X = [Acceleration, Displacement, Horsepower, Weight];
 ok = ! any (isnan ([X, MPG]), 2);
 Mdl = RegressionKernel (X(ok,:), MPG(ok))
Mdl =

  RegressionKernel

              ResponseName: 'Y'
                   Learner: 'svm'
    NumExpansionDimensions: 128
               KernelScale: 1
                    Lambda: 0.0107527
             BoxConstraint: 1
                   Epsilon: 0.926612
 yFit = predict (Mdl, X(find (ok, 3),:))
yFit =

   23.326
   22.824
   22.939

Standardizing matters more here than for a linear fit, since the kernel measures one distance across predictors of every scale.

 load carsmall
 X = [Acceleration, Displacement, Horsepower, Weight];
 ok = ! any (isnan ([X, MPG]), 2);
 plain = RegressionKernel (X(ok,:), MPG(ok));
 scaled = RegressionKernel (X(ok,:), MPG(ok), 'Standardize', true);
 plainLoss = loss (plain, X(ok,:), MPG(ok))
plainLoss = 53.217
 scaledLoss = loss (scaled, X(ok,:), MPG(ok))
scaledLoss = 20.625
RegressionKernel: obj = resume (obj, X, Y)
RegressionKernel: obj = resume (…, name, value)

obj = resume (obj, X, Y) restarts the optimization from the coefficients the model already carries, through the basis it already holds. It takes 'BetaTolerance', 'GradientTolerance' and 'IterationLimit', each defaulting to what the model was fitted with, and 'Weights'.

X and Y must be the data the model was fitted to; the object keeps no copy of them, which is what makes it small. Neither does it keep the observation weights, so a model fitted with 'Weights' must be given them again here or it will resume against uniform ones. MATLAB behaves the same way: measured on R2024a, resuming a weighted fit without passing the weights back reaches the objective of the unweighted fit.

Fit fuel consumption through a randomized Gaussian kernel, which bends where a linear model cannot.

 load carsmall
 X = [Acceleration, Displacement, Horsepower, Weight];
 ok = ! any (isnan ([X, MPG]), 2);
 Mdl = RegressionKernel (X(ok,:), MPG(ok))
Mdl =

  RegressionKernel

              ResponseName: 'Y'
                   Learner: 'svm'
    NumExpansionDimensions: 128
               KernelScale: 1
                    Lambda: 0.0107527
             BoxConstraint: 1
                   Epsilon: 0.926612
 yFit = predict (Mdl, X(find (ok, 3),:))
yFit =

   22.656
   22.614
   22.403

Standardizing matters more here than for a linear fit, since the kernel measures one distance across predictors of every scale.

 load carsmall
 X = [Acceleration, Displacement, Horsepower, Weight];
 ok = ! any (isnan ([X, MPG]), 2);
 plain = RegressionKernel (X(ok,:), MPG(ok));
 scaled = RegressionKernel (X(ok,:), MPG(ok), 'Standardize', true);
 plainLoss = loss (plain, X(ok,:), MPG(ok))
plainLoss = 52.288
 scaledLoss = loss (scaled, X(ok,:), MPG(ok))
scaledLoss = 23.597
RegressionKernel: savemodel (obj, filename)

savemodel (obj, filename) saves the model obj into filename in a form loadmodel can read back, the random basis included.

Fit fuel consumption through a randomized Gaussian kernel, which bends where a linear model cannot.

 load carsmall
 X = [Acceleration, Displacement, Horsepower, Weight];
 ok = ! any (isnan ([X, MPG]), 2);
 Mdl = RegressionKernel (X(ok,:), MPG(ok))
Mdl =

  RegressionKernel

              ResponseName: 'Y'
                   Learner: 'svm'
    NumExpansionDimensions: 128
               KernelScale: 1
                    Lambda: 0.0107527
             BoxConstraint: 1
                   Epsilon: 0.926612
 yFit = predict (Mdl, X(find (ok, 3),:))
yFit =

   22.628
   22.856
   23.980

Standardizing matters more here than for a linear fit, since the kernel measures one distance across predictors of every scale.

 load carsmall
 X = [Acceleration, Displacement, Horsepower, Weight];
 ok = ! any (isnan ([X, MPG]), 2);
 plain = RegressionKernel (X(ok,:), MPG(ok));
 scaled = RegressionKernel (X(ok,:), MPG(ok), 'Standardize', true);
 plainLoss = loss (plain, X(ok,:), MPG(ok))
plainLoss = 54.954
 scaledLoss = loss (scaled, X(ok,:), MPG(ok))
scaledLoss = 19.513

Examples 

 load carsmall
 X = [Acceleration, Displacement, Horsepower, Weight];
 ok = ! any (isnan ([X, MPG]), 2);
 Mdl = RegressionKernel (X(ok,:), MPG(ok))
Mdl =

  RegressionKernel

              ResponseName: 'Y'
                   Learner: 'svm'
    NumExpansionDimensions: 128
               KernelScale: 1
                    Lambda: 0.0107527
             BoxConstraint: 1
                   Epsilon: 0.926612
 yFit = predict (Mdl, X(find (ok, 3),:))
yFit =

   23.226
   23.526
   22.033
 load carsmall
 X = [Acceleration, Displacement, Horsepower, Weight];
 ok = ! any (isnan ([X, MPG]), 2);
 plain = RegressionKernel (X(ok,:), MPG(ok));
 scaled = RegressionKernel (X(ok,:), MPG(ok), 'Standardize', true);
 plainLoss = loss (plain, X(ok,:), MPG(ok))
plainLoss = 53.507
 scaledLoss = loss (scaled, X(ok,:), MPG(ok))
scaledLoss = 19.656