lassoglm
statistics: B = lassoglm (X, y)
statistics: B = lassoglm (X, y, distr)
statistics: B = lassoglm (X, y, distr, Name, Value)
statistics: [B, FitInfo] = lassoglm (…)
Lasso and elastic-net regularized generalized linear model regression.
B = lassoglm (X, y, distr) returns fitted
least-squares regression coefficients for a generalized linear model of the
response y on the predictor data X, penalized by the lasso (L1)
or elastic-net penalty. X is an n-by-p numeric matrix of
p predictors at each of n observations, and y is a
numeric vector of n responses. For the 'normal',
'binomial' and 'poisson' distributions y may also be
a logical vector, true read as 1, or a categorical vector with
exactly two categories, the second read as 1 and undefined elements as
missing; for 'binomial' the value 1 is the success. For the
'binomial' distribution y may also be an n-by-2 matrix
whose first column holds the number of successes and whose second holds the
number of trials; a row with NaN in either column or with no trials
is removed. distr names the distribution of
the response: 'normal' (default), 'binomial',
'poisson', 'gamma', or 'inverse gaussian'. The
canonical link function of the chosen distribution is used unless overridden
by the 'Link' option.
B is a p-by-L matrix, where L is the number of
regularization ('Lambda') values used; column k holds the
coefficients for the k-th value of lambda, in order of
ascending lambda.
[B, FitInfo] = lassoglm (…) also returns a structure
FitInfo with information about the fitted models:
Intercept | a 1-by-L vector of intercept terms |
Lambda | the 1-by-L vector of lambda values, in ascending order |
Alpha | the elastic-net mixing value used |
DF | the number of nonzero coefficients in each column of B |
Deviance | the deviance of the fitted model at each lambda; when cross-validation is requested this is instead the cross-validated mean deviance |
When cross-validation is requested (see 'CV' below), FitInfo
additionally contains SE (standard error of the cross-validated
deviance), LambdaMinDeviance and IndexMinDeviance (the
lambda with minimum cross-validated deviance and its index), and
Lambda1SE and Index1SE (the largest lambda within one
standard error of that minimum).
lassoglm accepts the following Name/Value pairs:
| Name | Value |
|---|---|
'Alpha' | the elastic-net mixing parameter, a scalar in
(0, 1]. 'Alpha' = 1 is the lasso penalty (default);
values towards 0 approach ridge regression. |
'Lambda' | a vector of non-negative lambda values. |
'Standardize' | a logical value (default true)
specifying whether the predictors are standardized before fitting. |
'Weights' | a real, non-negative vector of class single or
double, with one element per row of X. A NaN weight marks its
observation as missing. |
'BinomialSize' | for the 'binomial' distribution,
the number of trials, a scalar or a vector with one element per row of
X; y holds the number of successes. Default is 1 (Bernoulli
responses). It describes the same model as the two-column y, whose
trials are used when both are given. A row whose number of trials is
NaN or 0 is removed. 'BinomialSize' is an Octave
extension, and 'Size' is accepted as an alias for it. |
'Link' | the link function to use instead of the family’s
canonical link. Accepts any link name understood by glmfit (e.g.
'log', 'probit') or a numeric exponent for a power link. |
'Offset' | a numeric vector, one value per observation, added as a fixed term to the linear predictor (not penalized or fitted). |
'RelTol' | convergence tolerance for the coordinate descent. |
'MaxIter' | maximum number of iterations, a non-negative integer. A value of 0 returns the starting point, every coefficient 0, as MATLAB does. |
'DFmax' | maximum number of nonzero coefficients. |
'Intercept' | a logical value (default true) whether
to fit an intercept term. |
'PredictorNames' | a cell array of predictor names. |
'CV' | the number of folds K for K-fold
cross-validation, or a cvpartition object. The fold assignment is
random, so the selected lambda values are not reproducible without a
fixed random seed. |
'MCReps' | the number of Monte-Carlo repetitions of the cross-validation (default 1). |
Observations with NaN values in X, y or
'Weights' are removed before fitting. At least two observations
must remain and, if 'Weights' is given, at least two of them must
have a non-zero weight.
A response at the edge of its range on every observation, all failures or
all successes for 'binomial' and all zeros for 'poisson',
has no finite fit. B is then zero, the intercept places the fitted
mean at the limit of its range, and the default Lambda values are all
zero, as in MATLAB. MATLAB’s intercept there is about 102.5 in magnitude.
See also: lasso, glmfit, glmval, cvpartition
Source Code: lassoglm
Logistic regression with a lasso penalty on a small binary dataset.
X = [0.1, 1.2; 0.4, 0.7; 1.1, 0.2; 1.5, 1.9; 0.3, 0.5; 1.8, 1.1]; y = [0; 0; 1; 1; 0; 1]; [B, FitInfo] = lassoglm (X, y, 'binomial', 'Lambda', [0.01, 0.1])
B =
8.9966 3.4160
0 0
FitInfo =
scalar structure containing the fields:
Intercept =
-6.9378 -2.8237
Lambda =
0.010000 0.100000
Alpha = 1
DF =
1 1
Deviance =
0.2073 1.8102
PredictorNames = {}(0x0)
UseCovariance = 0
Poisson (count) regression with an elastic-net penalty.
X = [0.1, 1.2; 0.4, 0.7; 1.1, 0.2; 1.5, 1.9; 0.3, 0.5; 1.8, 1.1; 0.9, 0.3];
y = [1; 0; 2; 3; 1; 4; 2];
[B, FitInfo] = lassoglm (X, y, 'poisson', 'Lambda', [0.05, 0.2], ...
'Alpha', 0.6)
B =
1.0302 0.9078
0 0
FitInfo =
scalar structure containing the fields:
Intercept =
-0.4647 -0.3171
Lambda =
0.050000 0.200000
Alpha = 0.6000
DF =
1 1
Deviance =
2.1378 2.2415
PredictorNames = {}(0x0)
UseCovariance = 0
Choosing the penalty by cross-validation. Passing a 'CV' fold count makes lassoglm cross-validate the deviance along the lambda path. It then reports the lambda with the lowest mean deviance (LambdaMinDeviance) and the largest lambda within one standard error of it (Lambda1SE) -- the sparser "one-standard-error" model that is often preferred.
Here only the first two of eight predictors truly drive the response, so a good fit should keep few nonzero coefficients. The fold assignment is random; the seed below just makes the printed numbers reproducible.
rng (42);
X = randn (60, 8);
beta = [1.5; -2; zeros(6, 1)];
y = double (rand (60, 1) < 1 ./ (1 + exp (- X * beta)));
[B, FitInfo] = lassoglm (X, y, 'binomial', 'CV', 5);
printf ('LambdaMinDeviance = %.4f (%d nonzero)\n', ...
FitInfo.LambdaMinDeviance, FitInfo.DF(FitInfo.IndexMinDeviance));
LambdaMinDeviance = 0.0525 (6 nonzero)
printf ('Lambda1SE = %.4f (%d nonzero)\n', ...
FitInfo.Lambda1SE, FitInfo.DF(FitInfo.Index1SE));
Lambda1SE = 0.1007 (2 nonzero)
Coefficients of the one-standard-error model.
coef_1SE = B(:, FitInfo.Index1SE)
coef_1SE =
0.2269
-0.7506
0
0
0
0
0
0