stepwiselm
statistics: mdl = stepwiselm (tbl)
statistics: mdl = stepwiselm (tbl, ResponseVarName)
statistics: mdl = stepwiselm (tbl, y)
statistics: mdl = stepwiselm (X, y)
statistics: mdl = stepwiselm (…, InitialModel)
statistics: mdl = stepwiselm (…, Name, Value, …)
Fit a linear regression model using stepwise regression and return a
LinearModel object.
stepwiselm starts from an initial model and repeatedly searches for
a term to add to, or remove from, the current model, based on the value of
the 'Criterion' option, until no single addition or removal
improves the model any further.
mdl = stepwiselm (tbl) fits a stepwise model using the
variables in the table (or dataset) tbl, starting from a constant
model. By default, the last variable in tbl is used as the response
and all other variables are candidate predictors. Variables that are
categorical arrays, cell arrays of character vectors, or logical
arrays are automatically treated as categorical predictors.
mdl = stepwiselm (tbl, ResponseVarName) uses the
variable named ResponseVarName in tbl as the response, and all
remaining variables in tbl as candidate predictors.
mdl = stepwiselm (tbl, y) uses the variables in
tbl as candidate predictors and the external numeric vector y
as the response.
mdl = stepwiselm (X, y) fits a stepwise model of
the response y to the predictor data X, an numeric
or logical matrix. By default, the predictors are named 'x1',
'x2', …, 'xP' and the response is named
'y'.
mdl = stepwiselm (…, InitialModel) additionally
specifies the model to start the stepwise search from, using any of the
input combinations shown above. InitialModel can be any of the
following, and the same set of values can also be used for the
'Lower' and 'Upper' options below, which bound the
smallest and largest set of terms stepwiselm is allowed to reach.
| Value | Description |
|---|---|
'constant' | Model contains only an intercept term. This
is the default InitialModel and default 'Lower' bound. |
'linear' | Model contains an intercept and one term for each predictor variable. |
'interactions' | Model contains an intercept, all
linear terms, and all pairwise products of distinct predictor variables
(no squared terms). This is the default 'Upper' bound. |
'purequadratic' | Model contains an intercept, all linear terms, and all squared terms. |
'quadratic' | Model contains an intercept, all linear terms, all pairwise products of distinct predictor variables, and all squared terms. |
'polyijk' | Model is a polynomial with maximum degree
in the first predictor, in the second, and so on, given
as a run of single-digit numerals, one per predictor (e.g.
'poly21' for two predictors). The model contains interaction
terms, but the degree of each interaction term never exceeds the largest
of the specified per-predictor degrees. |
| terms matrix | A or numeric matrix,
where is the number of terms and is the number of
predictor variables, following the same convention as fitlm’s
terms matrix. When InitialModel is given as a terms matrix, the
'PredictorVars' option may not also be used. |
| Wilkinson formula | A character vector of the form
'y ~ terms'. When a formula is combined with 'ResponseVar'
or 'PredictorVars', the formula’s response and predictor terms
must agree with those options, or stepwiselm errors. |
Source Code: stepwiselm
mdl = stepwiselm (…, Name, Value, …)
specifies additional options using one or more Name-Value pair
arguments.
| Name | Value |
|---|---|
'Criterion' | Criterion used to decide whether a term is
added or removed at each step. One of 'sse' (default),
'aic', 'bic', 'rsquared', or 'adjrsquared'.
For 'sse', the p-value of an F-test comparing the model with and
without the candidate term is used; for the others, the raw change in the
named quantity is used directly against 'PEnter'/'PRemove'. |
'PEnter' | Threshold to add a term. Defaults depend on
'Criterion': 0.05 for 'sse', 0 for
'aic'/'bic', 0.1 for 'rsquared', 0
for 'adjrsquared'. |
'PRemove' | Threshold to remove a term. Defaults depend
on 'Criterion': 0.10 for 'sse', 0.01 for
'aic'/'bic', 0.05 for 'rsquared',
-0.05 for 'adjrsquared'. |
'NSteps' | Maximum number of add/remove steps to take,
given as a nonnegative integer. Default is unlimited. 'NSteps'
set to 0 returns the initial model unchanged. |
'Lower' | Model specification (in the same form as
InitialModel, above) describing terms that may never be removed from
the model. Terms in 'Lower' are protected from removal, but are
not automatically added if absent from InitialModel. Default is
'constant'. |
'Upper' | Model specification (in the same form as
InitialModel, above) describing the largest set of terms
stepwiselm may add. Default is 'interactions'. |
'Verbose' | Controls how much progress information is
printed while stepping. 0 suppresses all output, 1
(default) prints the action taken at each step, 2 additionally
prints the p-value or criterion value considered for every candidate term
examined at each step. |
'Intercept' | A logical scalar indicating whether the
initial model includes a constant (intercept) term. Only applies when
InitialModel is a character vector model name (or omitted); ignored
when InitialModel is a terms matrix or formula. Default is
true. |
'Weights' | A numeric vector of nonnegative observation weights, one per observation. Default is a vector of ones. |
'Exclude' | A numeric or logical vector specifying observations to exclude from the fit. |
'CategoricalVars' | Specifies which predictor variables
are treated as categorical, given as a vector of column indices, a logical
vector, or a cell array of variable names (table input only). Each
categorical predictor with distinct categories is expanded into
indicator variables, and stepwiselm always adds or
removes that entire group of indicator variables together, in a single
step, treating the categorical predictor as one term. |
'VarNames' | A cell array of character vectors naming the predictor and response variables, response last. Only applies when X and y are supplied directly, not table input. |
'ResponseVar' | A character vector naming the response
variable, overriding the response that would otherwise be inferred (the
last table variable, or 'y' for matrix input). |
'PredictorVars' | A cell array of character vectors naming which variables in tbl to consider as candidate predictors. By default, all variables in tbl other than the response are used. May not be combined with a terms-matrix InitialModel, and must agree with any formula-based InitialModel. |
Source Code: stepwiselm
The Steps property
The returned model’s Steps property records the trace, as a
structure with seven fields. Start, Lower, and Upper
are LinearFormula objects for the starting model and the two bounds;
Criterion is the criterion as it was asked for; PEnter and
PRemove are the thresholds it ran under; and History is a
table with one row per step.
History always carries Action ('Start',
'Add', or 'Remove'), TermName, Terms (the
terms matrix after the step, over the model’s variables), DF (the
coefficient count after the step), and delDF (the change in it,
negative for a removal). The remaining columns follow the
criterion, which is why a history is read by name and not by position:
| Criterion | Further columns |
|---|---|
'SSE' | FStat and pValue. |
'AIC', 'BIC', 'Rsquared',
'AdjRsquared' one column named for the criterion, holding its
value after the step, the starting model included. |
Source Code: stepwiselm
The first row is the starting model, named by its right-hand side.
step appends to the history it inherits rather than beginning a new
one, and inherits Criterion, Lower, PEnter, and
PRemove from it; Steps.Start is then the model stepped from.
At each step, stepwiselm examines every term not currently in the
model but within the 'Upper' bound, and every term currently in
the model but not protected by the 'Lower' bound. If any term
outside the model would improve it by at least 'PEnter', the best
such term is added; otherwise, if any term inside the model falls short of
'PRemove', the worst such term is removed. The process repeats
until neither an addition nor a removal improves the model, or until
'NSteps' steps have been taken.
stepwiselm never adds a higher-order term unless all of its
lower-order marginal terms are already in the model (e.g. it will not add
x1:x2^2 unless both x1 and x2^2 are already present),
and correspondingly never removes a lower-order term that a higher-order
term still in the model depends on. At every step, if a term in the
current model is found to be exactly redundant (linearly dependent on the
other terms already in the model), it is removed immediately regardless of
the 'Criterion' value.
Because the final model depends on the initial model and the order in
which terms are considered, stepwiselm finds a locally, but not
necessarily globally, optimal model.
Robust fitting cannot be combined with stepwise regression; do not pass
'RobustOpts' to stepwiselm.
mdl is returned as a LinearModel object. See also the
step method of LinearModel, which performs a single bounded
round of stepwise search starting from an already-fitted model.
See also: LinearModel, fitlm
Source Code: stepwiselm
Stepwise search from a constant model, with a custom entry threshold. Twenty apartments are described by their size, floor number, and distance from the city center, along with their monthly rent. Starting from an empty model, stepwiselm adds one predictor at a time as long as it improves the fit by at least PEnter, then checks whether anything already in the model should come back out.
Size = [45 50 38 62 70 55 48 40 65 58 42 72 35 68 52 60 46 66 39 54]'; Floor = [3 5 2 8 10 4 1 6 9 7 3 12 2 11 5 6 4 10 1 5]'; Distance = [12 8 15 5 3 9 18 10 4 7 14 2 20 3 8 6 11 4 17 9]'; Rent = 200 + 12*Size - 15*Distance + 2*Floor + 5*sin ((1:20)'/2); X = [Size, Floor, Distance];
Fit with a looser entry threshold, printing every candidate examined.
mdl = stepwiselm (X, Rent, 'PEnter', 0.06, 'Verbose', 2)
pValue for adding x1 is 1.09045e-15
pValue for adding x2 is 1.66523e-09
pValue for adding x3 is 7.06608e-12
1. Adding x1, FStat = 669.122, pValue = 1.09045e-15
pValue for adding x2 is 0.00295624
pValue for adding x3 is 2.35184e-18
2. Adding x3, FStat = 1642.47, pValue = 2.351838e-18
pValue for adding x2 is 0.106958
pValue for adding x1:x3 is 0.0130046
3. Adding x1:x3, FStat = 7.80575, pValue = 0.01300455
pValue for adding x2 is 0.647485
pValue for removing x1:x3 is 0.0130046
mdl =
Linear regression model:
y ~ 1 + x1*x3
Estimated Coefficients:
4x4 table
Estimate SE tStat pValue
__________ _________ ________ ___________
(Intercept) 208.615 11.4707 18.1868 4.12042e-12
x1 12.2923 0.168557 72.9266 1.28832e-21
x3 -14.221 0.72894 -19.5091 1.40323e-12
x1:x3 -0.0436036 0.0156068 -2.79388 0.0130046
Number of observations: 20, Error degrees of freedom: 16
Root Mean Squared Error: 3.22673
R-squared: 0.99982, Adjusted R-Squared: 0.999786
F-statistic vs. constant model: 29564.1, p-value = 3.73907e-30
Starting from a formula, with the response, predictors, and categorical variable all named explicitly. Eighteen coffee shops report their weekly ad spend, staff count, and sales season, along with weekly sales. The search starts already containing AdSpend, with Employees and the categorical Season left to consider adding.
AdSpend = [200 350 500 220 370 520 240 390 540 260 410 560 280 430 580 300 450 600]';
Employees = [4 5 6 4 5 6 4 5 6 4 5 6 4 5 6 4 5 6]';
Season = {'Low';'Mid';'Peak';'Low';'Mid';'Peak';'Low';'Mid';'Peak'; ...
'Low';'Mid';'Peak';'Low';'Mid';'Peak';'Low';'Mid';'Peak'};
SeasonEffect = [0;150;400;0;150;400;0;150;400;0;150;400;0;150;400;0;150;400];
Sales = 1000 + 0.8*AdSpend + 5*Employees + SeasonEffect + 6*sin ((1:18)');
T = table (AdSpend, Employees, Season, Sales, ...
'VariableNames', {'AdSpend','Employees','Season','Sales'});
Fit, treating Season as categorical and naming the response explicitly.
mdl = stepwiselm (T, 'Sales ~ 1 + AdSpend', 'ResponseVar', 'Sales', ...
'PredictorVars', {'AdSpend','Employees','Season'}, ...
'CategoricalVars', {'Season'}, 'Verbose', 1)
1. Adding Season, FStat = 1063.92, pValue = 5.097984e-16
mdl =
Linear regression model:
Sales ~ 1 + AdSpend + Season
Estimated Coefficients:
4x4 table
Estimate SE tStat pValue
________ _________ _______ ___________
(Intercept) 1028.5 8.42292 122.107 1.33966e-22
AdSpend 0.767339 0.0327862 23.4044 1.26455e-12
Season_Mid 159.538 5.63121 28.3311 9.1845e-14
Season_Peak 419.105 10.2112 41.0437 5.43235e-16
Number of observations: 18, Error degrees of freedom: 14
Root Mean Squared Error: 4.75117
R-squared: 0.999755, Adjusted R-Squared: 0.999702
F-statistic vs. constant model: 19007, p-value = 1.68686e-25
Bounding the search with terms matrices instead of model-name keywords. Twelve potted plants are given varying hours of sunlight and amounts of water, and their growth is measured. T_initial is a constant model and T_upper allows the two main effects plus their interaction, using the same terms-matrix convention as fitlm.
Sunlight = [2 4 6 8 3 5 7 9 2.5 4.5 6.5 8.5]'; Water = [100 150 200 250 120 170 220 270 110 160 210 260]'; Growth = 5 + 0.3*Sunlight + 0.06*Water + 0.4*sin ((1:12)'); X = [Sunlight, Water]; T_initial = [0 0 0]; T_upper = [0 0 0; 1 0 0; 0 1 0; 1 1 0];
Fit, printing the p-value considered for every candidate term.
mdl = stepwiselm (X, Growth, T_initial, 'Upper', T_upper, 'Verbose', 2)
pValue for adding x1 is 2.40324e-12
pValue for adding x2 is 7.76906e-13
1. Adding x2, FStat = 1986.77, pValue = 7.769055e-13
pValue for adding x1 is 0.994763
pValue for removing x2 is 7.76906e-13
mdl =
Linear regression model:
y ~ 1 + x2
Estimated Coefficients:
2x4 table
Estimate SE tStat pValue
_________ __________ _______ ___________
(Intercept) 4.52893 0.310039 14.6076 4.5094e-08
x2 0.0714427 0.00160282 44.5732 7.76906e-13
Number of observations: 12, Error degrees of freedom: 10
Root Mean Squared Error: 0.313677
R-squared: 0.994992, Adjusted R-Squared: 0.994491
F-statistic vs. constant model: 1986.77, p-value = 7.76906e-13
A categorical predictor is added or removed as one indicator group, never one indicator column at a time. Eighteen plots receive varying fertilizer amounts across three regions, and crop yield is recorded. The upper bound poly21 allows a constant, Fertilizer, Fertilizer^2, Region, and their interaction; stepwiselm treats the two indicator columns generated by the three-level Region as a single term throughout.
Fertilizer = repmat ([10;20;30;40;50;60], 3, 1);
Region = [repmat({'A'},6,1); repmat({'B'},6,1); repmat({'C'},6,1)];
RegionEffect = [zeros(6,1); 15*ones(6,1); 35*ones(6,1)];
Yield = 20 + 0.5*Fertilizer + RegionEffect + 1.5*sin ((1:18)');
T = table (Fertilizer, Region, Yield);
Fit, printing every candidate considered at each step.
mdl = stepwiselm (T, 'Yield ~ Fertilizer', 'Upper', 'poly21', 'Verbose', 2)
pValue for adding Fertilizer^2 is 0.98431
pValue for adding Region is 3.04051e-20
1. Adding Region, FStat = 4290.82, pValue = 3.040514e-20
pValue for adding Fertilizer^2 is 0.649576
pValue for adding Fertilizer:Region is 0.989742
pValue for removing Fertilizer is 4.38859e-17
pValue for removing Region is 3.04051e-20
mdl =
Linear regression model:
Yield ~ 1 + Fertilizer + Region
Estimated Coefficients:
4x4 table
Estimate SE tStat pValue
________ _________ _______ ___________
(Intercept) 21.8566 0.415768 52.5693 1.73528e-17
Fertilizer 0.446216 0.0090728 49.1817 4.38859e-17
Region_B 15.0203 0.379542 39.5747 9.01177e-16
Region_C 35.041 0.379542 92.3243 6.68219e-21
Number of observations: 18, Error degrees of freedom: 14
Root Mean Squared Error: 0.657387
R-squared: 0.998729, Adjusted R-Squared: 0.998457
F-statistic vs. constant model: 3666.83, p-value = 1.68315e-20
Capping the search with NSteps before it converges on its own. Twenty observations depend on three predictors, all with genuine effects. Left alone, stepwiselm would add all three; limiting NSteps to 1 stops the search after only the single best addition.
x1 = (1:20)'; x2 = mod ((0:19)', 5); x3 = mod ((0:19)', 3); y = 5 + 2*x1 + 1.5*x2 + 3*x3 + 2*sin ((1:20)'/1.7); X = [x1, x2, x3];
Fit, but stop after the very first step.
mdl = stepwiselm (X, y, 'Upper', 'linear', 'NSteps', 1, 'Verbose', 1)
1. Adding x1, FStat = 194.187, pValue = 4.390526e-11
mdl =
Linear regression model:
y ~ 1 + x1
Estimated Coefficients:
2x4 table
Estimate SE tStat pValue
________ ________ _______ ___________
(Intercept) 10.8182 1.72305 6.27851 6.40323e-06
x1 2.00439 0.143838 13.9351 4.39053e-11
Number of observations: 20, Error degrees of freedom: 18
Root Mean Squared Error: 3.70923
R-squared: 0.915169, Adjusted R-Squared: 0.910456
F-statistic vs. constant model: 194.187, p-value = 4.39053e-11
Raw matrix input with default x1, x2, ... naming, and custom entry and removal thresholds. Fifteen observations depend mainly on x1. PEnter and PRemove are set explicitly rather than relying on the defaults for the SSE criterion.
x1 = (1:15)'; x2 = mod ((0:14)', 4); y = 8 + 0.5*x1 + 0.3*x2 + 2.5*sin ((1:15)'/1.2); X = [x1, x2];
Fit with a wider entry threshold and a stricter removal threshold.
mdl = stepwiselm (X, y, 'PEnter', 0.2, 'PRemove', 0.3, 'Verbose', 1)
1. Adding x1, FStat = 10.7754, pValue = 0.005943484
mdl =
Linear regression model:
y ~ 1 + x1
Estimated Coefficients:
2x4 table
Estimate SE tStat pValue
________ ________ _______ ___________
(Intercept) 9.57932 0.981745 9.75744 2.39052e-07
x1 0.354446 0.107978 3.28259 0.00594348
Number of observations: 15, Error degrees of freedom: 13
Root Mean Squared Error: 1.80681
R-squared: 0.453217, Adjusted R-Squared: 0.411157
F-statistic vs. constant model: 10.7754, p-value = 0.00594348
The hald cement data, mirroring MATLAB's own reference example for stepwiselm. Four chemical percentages in cement (ingredients) are used to predict heat given off while hardening (heat). Starting from a constant model, stepwiselm adds three terms and then removes one of them again once it becomes redundant.
load hald
Fit with a looser entry threshold than the default.
mdl = stepwiselm (ingredients, heat, 'PEnter', 0.06, 'Verbose', 1)
1. Adding x4, FStat = 22.7985, pValue = 0.0005762318
2. Adding x1, FStat = 108.224, pValue = 1.105281e-06
3. Adding x2, FStat = 5.02586, pValue = 0.05168735
4. Removing x4, FStat = 1.86326, pValue = 0.2053954
mdl =
Linear regression model:
y ~ 1 + x1 + x2
Estimated Coefficients:
3x4 table
Estimate SE tStat pValue
________ _________ _______ ___________
(Intercept) 52.5773 2.28617 22.998 5.45657e-10
x1 1.46831 0.121301 12.1047 2.69221e-07
x2 0.66225 0.0458547 14.4424 5.02896e-08
Number of observations: 13, Error degrees of freedom: 10
Root Mean Squared Error: 2.40634
R-squared: 0.978678, Adjusted R-Squared: 0.974414
F-statistic vs. constant model: 229.504, p-value = 4.40658e-09
The carsmall data, mirroring MATLAB's own reference example for stepwiselm with a terms-matrix bound. Fuel economy (MPG) is predicted from acceleration and weight. T_initial is a constant model and T_upper allows both main effects plus their interaction.
load carsmall X = [Acceleration, Weight]; T_initial = [0 0 0]; T_upper = [0 0 0; 1 0 0; 0 1 0; 1 1 0];
Fit, printing the p-value considered for every candidate term.
mdl = stepwiselm (X, MPG, T_initial, 'Upper', T_upper, 'Verbose', 2)
pValue for adding x1 is 4.09725e-06
pValue for adding x2 is 1.64335e-28
1. Adding x2, FStat = 259.309, pValue = 1.643351e-28
pValue for adding x1 is 0.184927
pValue for removing x2 is 1.64335e-28
mdl =
Linear regression model:
y ~ 1 + x2
Estimated Coefficients:
2x4 table
Estimate SE tStat pValue
___________ ___________ ________ ___________
(Intercept) 49.2376 1.64114 30.0022 2.70152e-49
x2 -0.00861193 0.000534801 -16.1031 1.64335e-28
Number of observations: 94, Error degrees of freedom: 92
Root Mean Squared Error: 4.13449
R-squared: 0.738122, Adjusted R-Squared: 0.735276
F-statistic vs. constant model: 259.309, p-value = 1.64335e-28