Categories &

Functions List

Function Reference: lassoglm

statistics: B = lassoglm (X, y)
statistics: B = lassoglm (X, y, distr)
statistics: B = lassoglm (X, y, distr, Name, Value)
statistics: [B, FitInfo] = lassoglm (…)

Lasso and elastic-net regularized generalized linear model regression.

B = lassoglm (X, y, distr) returns fitted least-squares regression coefficients for a generalized linear model of the response y on the predictor data X, penalized by the lasso (L1) or elastic-net penalty. X is an n-by-p numeric matrix of p predictors at each of n observations, and y is a numeric vector of n responses. distr names the distribution of the response: 'normal' (default), 'binomial', 'poisson', 'gamma', or 'inverse gaussian'. The canonical link function of the chosen distribution is used unless overridden by the 'Link' option.

B is a p-by-L matrix, where L is the number of regularization ('Lambda') values used; column k holds the coefficients for the k-th value of lambda, in order of ascending lambda.

[B, FitInfo] = lassoglm (…) also returns a structure FitInfo with information about the fitted models:

Intercepta 1-by-L vector of intercept terms
Lambdathe 1-by-L vector of lambda values, in ascending order
Alphathe elastic-net mixing value used
DFthe number of nonzero coefficients in each column of B
Deviancethe deviance of the fitted model at each lambda; when cross-validation is requested this is instead the cross-validated mean deviance

Source Code: lassoglm

When cross-validation is requested (see 'CV' below), FitInfo additionally contains SE (standard error of the cross-validated deviance), LambdaMinDeviance and IndexMinDeviance (the lambda with minimum cross-validated deviance and its index), and Lambda1SE and Index1SE (the largest lambda within one standard error of that minimum).

lassoglm accepts the following Name/Value pairs:

NameValue
'Alpha'the elastic-net mixing parameter, a scalar in (0, 1]. 'Alpha' = 1 is the lasso penalty (default); values towards 0 approach ridge regression.
'Lambda'a vector of non-negative lambda values.
'Standardize'a logical value (default true) specifying whether the predictors are standardized before fitting.
'Weights'a vector of non-negative observation weights.
'Size'for the 'binomial' distribution, the number of trials (a scalar or a per-observation vector); y holds the number of successes. Default is 1 (Bernoulli responses).
'Link'the link function to use instead of the family’s canonical link. Accepts any link name understood by glmfit (e.g. 'log', 'probit') or a numeric exponent for a power link.
'Offset'a numeric vector, one value per observation, added as a fixed term to the linear predictor (not penalized or fitted).
'RelTol'convergence tolerance for the coordinate descent.
'MaxIter'maximum number of iterations.
'DFmax'maximum number of nonzero coefficients.
'Intercept'a logical value (default true) whether to fit an intercept term.
'PredictorNames'a cell array of predictor names.
'CV'the number of folds K for K-fold cross-validation, or a cvpartition object. The fold assignment is random, so the selected lambda values are not reproducible without a fixed random seed.
'MCReps'the number of Monte-Carlo repetitions of the cross-validation (default 1).

Source Code: lassoglm

See also: lasso, glmfit, glmval, cvpartition

Source Code: lassoglm

Logistic regression with a lasso penalty on a small binary dataset.

 X = [0.1, 1.2; 0.4, 0.7; 1.1, 0.2; 1.5, 1.9; 0.3, 0.5; 1.8, 1.1];
 y = [0; 0; 1; 1; 0; 1];
 [B, FitInfo] = lassoglm (X, y, 'binomial', 'Lambda', [0.01, 0.1])
B =

   8.9966   3.4160
        0        0

FitInfo =

  scalar structure containing the fields:

    Intercept =

      -6.9378  -2.8237

    Lambda =

       0.010000   0.100000

    Alpha = 1
    DF =

       1   1

    Deviance =

       0.2073   1.8102

    PredictorNames = {}(0x0)
    UseCovariance = 0

Poisson (count) regression with an elastic-net penalty.

 X = [0.1, 1.2; 0.4, 0.7; 1.1, 0.2; 1.5, 1.9; 0.3, 0.5; 1.8, 1.1; 0.9, 0.3];
 y = [1; 0; 2; 3; 1; 4; 2];
 [B, FitInfo] = lassoglm (X, y, 'poisson', 'Lambda', [0.05, 0.2], ...
                          'Alpha', 0.6)
B =

   1.0302   0.9078
        0        0

FitInfo =

  scalar structure containing the fields:

    Intercept =

      -0.4647  -0.3171

    Lambda =

       0.050000   0.200000

    Alpha = 0.6000
    DF =

       1   1

    Deviance =

       2.1378   2.2415

    PredictorNames = {}(0x0)
    UseCovariance = 0

Choosing the penalty by cross-validation. Passing a 'CV' fold count makes lassoglm cross-validate the deviance along the lambda path. It then reports the lambda with the lowest mean deviance (LambdaMinDeviance) and the largest lambda within one standard error of it (Lambda1SE) -- the sparser "one-standard-error" model that is often preferred.

Here only the first two of eight predictors truly drive the response, so a good fit should keep few nonzero coefficients. The fold assignment is random; the seed below just makes the printed numbers reproducible.

 rand ('seed', 42);
 randn ('seed', 42);
 X = randn (60, 8);
 beta = [1.5; -2; zeros(6, 1)];
 y = double (rand (60, 1) < 1 ./ (1 + exp (- X * beta)));
 [B, FitInfo] = lassoglm (X, y, 'binomial', 'CV', 5);
 printf ('LambdaMinDeviance = %.4f (%d nonzero)\n', ...
         FitInfo.LambdaMinDeviance, FitInfo.DF(FitInfo.IndexMinDeviance));
LambdaMinDeviance = 0.0426 (2 nonzero)
 printf ('Lambda1SE         = %.4f (%d nonzero)\n', ...
         FitInfo.Lambda1SE, FitInfo.DF(FitInfo.Index1SE));
Lambda1SE         = 0.0896 (2 nonzero)

Coefficients of the one-standard-error model.

 coef_1SE = B(:, FitInfo.Index1SE)
coef_1SE =

   1.1220
  -0.7655
        0
        0
        0
        0
        0
        0