Categories &

Functions List

Function Reference: ridge

statistics: b = ridge (y, X, k)
statistics: b = ridge (y, X, k, scaled)

Ridge regression.

b = ridge (y, X, k) returns the vector of coefficient estimates by applying ridge regression from the predictor matrix X to the response vector y. Each value of b is the coefficient for the respective ridge parameter given k. By default, b is calculated after centering and scaling the predictors to have a zero mean and standard deviation 1.

b = ridge (y, X, k, scaled) performs the regression with the specified scaling of the coefficient estimates b. When scaled = 0, the function restores the coefficients to the scale of the original data thus is more useful for making predictions. When scaled = 1, the coefficient estimates correspond to the scaled centered data.

  • y must be an N×1 numeric vector with the response data.
  • X must be an N×p numeric matrix with the predictor data.
  • k must be a numeric vector with the ridge parameters.
  • scaled must be a numeric scalar indicating whether the coefficient estimates in b are restored to the scale of the original data. By default, scaled = 1.

Further information about Ridge regression can be found at https://en.wikipedia.org/wiki/Ridge_regression

See also: lasso, stepwisefit, regress

Source Code: ridge

Example: 1

Perform ridge regression for a range of ridge parameters and observe how the coefficient estimates change based on the acetylene dataset.

 load acetylene
 X = [x1, x2, x3];
 x1x2 = x1 .* x2;
 x1x3 = x1 .* x3;
 x2x3 = x2 .* x3;
 D = [x1, x2, x3, x1x2, x1x3, x2x3];
 k = 0:1e-5:5e-3;
 b = ridge (y, D, k);
 figure
 plot (k, b, 'LineWidth', 2)
 ylim ([-100, 100])
 grid on
 xlabel ('Ridge Parameter')
 ylabel ('Standardized Coefficient')
 title ('Ridge Trace')
 legend ('x1', 'x2', 'x3', 'x1x2', 'x1x3', 'x2x3')
plotted figure

Example: 2 

 load carbig
 X = [Acceleration Weight Displacement Horsepower];
 y = MPG;
 n = length (y);
 rand ('seed',1); % For reproducibility
 c = cvpartition (n,'HoldOut',0.3);
 idxTrain = training(c,1);
 idxTest = ! idxTrain;
 idxTrain = training(c,1);
 idxTest = ! idxTrain;
 k = 5;
 b = ridge (y(idxTrain),X(idxTrain,:),k,0);

Predict MPG values for the test data using the model.

 yhat = b(1) + X(idxTest,:)*b(2:end);
 scatter (y(idxTest),yhat)
 hold on
 plot (y(idxTest),y(idxTest),'r')
 xlabel ('Actual MPG')
 ylabel ('Predicted MPG')
 hold off
plotted figure