RegressionNeuralNetwork
statistics: RegressionNeuralNetwork
Create a RegressionNeuralNetwork object containing a neural network
regression model.
obj = RegressionNeuralNetwork (X, Y) returns a
neural network regression model, obj, with X being the predictor
data and Y the continuous response of the observations in X.
The network is trained against the mean squared error, and its output layer
applies the identity, so a prediction is an unrestricted real number rather
than a score over classes. This is the only difference in the engine
between this class and ClassificationNeuralNetwork; everything else,
the layer sizes, the activations, the learning rate and the initialisation,
behaves identically.
obj = RegressionNeuralNetwork (…, name,
value) returns a model with additional options specified by
Name-Value pair arguments listed below.
| Name | Value |
|---|---|
'Standardize' | A logical scalar specifying whether the
predictor data should be centred and scaled before training. The same
transformation is applied by predict. The default is false. |
'PredictorNames' | A cell array of character vectors naming the predictors, in the order they appear in X. |
'ResponseName' | A character vector naming the response.
The default is 'Y'. |
'ResponseTransform' | A character vector naming one of the
supported transformations, or a function handle, applied to the predicted
response by predict and resubPredict. The default is
'none'. |
'LayerSizes' | A positive integer vector specifying the number of units in each fully connected hidden layer. The default is 10, one hidden layer of ten units. |
'Activations' | A character vector or cell array of
character vectors specifying the activation of the hidden layers. The
supported functions are 'linear', 'sigmoid',
'relu', 'tanh', 'lrelu', 'prelu',
'elu', 'gelu' and 'none'. The default is
'relu'. |
'OutputLayerActivation' | A character vector specifying
the activation of the output layer. The default is 'none', the
identity, which is what a regression output calls for. The supported
values are the same as for 'Activations'. |
'LearningRate' | A positive scalar specifying the learning
rate for gradient descent. The default is 0.003. A larger rate can drive
every unit of a hidden layer negative, after which a rectifier passes no
gradient and the network stops training.
Applies only when 'Solver' is 'sgd'. |
'Solver' | A character vector naming the solver that
trains the network, either 'lbfgs' or 'sgd'. The
default is 'lbfgs', which minimizes the loss over the whole
training set at once by limited-memory BFGS, as MATLAB does. It takes
no learning rate, stops on the three tolerances below, and reaches a
lower training loss in fewer passes over the data, though each of its
iterations costs several passes where an epoch costs one.
'sgd' visits the samples one at a time and steps down the
gradient of each, running for 'IterationLimit' epochs; it was
the default before version 1.9.0. |
'GradientTolerance' | A nonnegative scalar. Training
stops once the gradient’s infinity norm falls to or below it, which is
the quantity MATLAB tests too. The default is 1e-6. Applies
only when 'Solver' is 'lbfgs'. |
'StepTolerance' | A nonnegative scalar. Training
stops once the step’s infinity norm falls to or below it, which is the
quantity MATLAB tests too. The default is 1e-6. Applies only
when 'Solver' is 'lbfgs'. |
'LossTolerance' | A real scalar. Training stops once
the training loss falls to or below it. The test is on the loss
itself and not on its change, matching MATLAB; pass -Inf to
switch it off. The default is 1e-6. Applies only when
'Solver' is 'lbfgs'. |
'IterationLimit' | A positive integer specifying the
maximum number of training iterations. The default is 1000.
Under 'sgd' this counts epochs, under
'lbfgs' solver iterations. |
'DisplayInfo' | A logical scalar specifying whether to
print information during training. The default is false. |
Source Code: RegressionNeuralNetwork
The supported values for 'ResponseTransform' are:
| Value | Description |
|---|---|
'none' | (no transformation) |
'identity' | (no transformation) |
'exp' | |
'log' |
Source Code: RegressionNeuralNetwork
See also: fitrnet, ClassificationNeuralNetwork, fcnntrain, fcnnpredict
Source Code: RegressionNeuralNetwork
The RegressionNeuralNetwork class contains the following properties:
An numeric matrix, as it was supplied to the constructor, before any rows carrying missing values were dropped. This property is read-only.
An numeric vector, as it was supplied to the constructor. This property is read-only.
A positive integer scalar, counting only the rows that survived the removal of missing values. This property is read-only.
A logical column vector with the same length as the observations in the
original predictor data X, true for each row that was used for
fitting the RegressionNeuralNetwork model. It is empty, [],
when every observation was used, so a non-empty value means that rows
holding missing values were dropped. This property is read-only.
A positive integer scalar, the number of columns of X. This
property is read-only.
A cell array of character vectors, one per column of X. This
property is read-only.
A character vector. This property is read-only.
A row vector with one entry per predictor, used for standardization. Empty when the predictor data were not standardized. This property is read-only.
A row vector with one entry per predictor, used for standardization. Empty when the predictor data were not standardized. This property is read-only.
A row vector of positive integers, one per hidden layer. It does not include the output layer, whose width is the number of responses. This property is read-only.
A character vector, applying to every hidden layer, or a cell array of character vectors with one entry per hidden layer. This property is read-only.
A character vector. The default, 'none', applies the identity,
so a prediction is an unrestricted real number. This property is
read-only.
A positive scalar value defining the learning rate used by the gradient descent algorithm during training. This property is read-only.
A positive integer scalar. This property is read-only.
A structure holding the fit as it was asked for: LayerSizes,
Activations, OutputLayerActivation,
LayerWeightsInitializers, Solver,
LearningRate, IterationLimit,
GradientTolerance, LossTolerance,
StepTolerance, DisplayInfo, StandardizeData,
and the Version, Method and Type tags.
What came out of the fit is elsewhere: the LayerWeights and
LayerBiases properties hold the network, TrainingHistory
the series and ConvergenceInfo where it stopped.
LayerWeightsInitializers names the scheme each layer’s weights
were drawn with, the output layer last: 'he' for a rectifying
activation and 'glorot' for a symmetric one. It is a report,
not a setting, the engine choosing per layer from the activation and
offering no way to override it.
OutputLayerActivation, Solver and LearningRate
are this package’s own; MATLAB has no counterpart for them. The fields
it reports that this class does not accept as arguments
(Lambda, the validation set and its patience and frequency,
InitialStepSize and the two initializer settings) are absent.
This property is read-only.
A structure with the fields Time, the seconds training took, and
TrainingLoss, the mean squared error of the network at the end of
each iteration. This property is read-only.
Under 'lbfgs' the structure carries Gradient and
Step, the two quantities the solver measured to decide it had
converged, and ConvergenceCriterion, naming the test that
stopped it. It carries no Accuracy: MATLAB reports none, and
measuring it would cost a pass over the whole training set at every
iteration.
A logical scalar. This property is read-only.
A character vector, either 'Gradient Descent' for the
stochastic solver or 'LBFGS' for the full-batch one.
This property is read-only.
A cell array with one entry per layer, the output layer included.
LayerWeights{i} has one row per unit of layer and one
column per input to that layer. This property is read-only.
A cell array with one entry per layer, the output layer included.
LayerBiases{i} is a column with one entry per unit of layer
. This property is read-only.
A table with the variables Iteration and
TrainingLoss, one row per training iteration. This property is
read-only.
The columns follow the solver. Under 'sgd' they are
Iteration and TrainingLoss, with TrainingAccuracy
for a classifier. Under 'lbfgs' they are Iteration,
TrainingLoss, Gradient and Step, as MATLAB’s are.
A numeric column vector with one entry per training observation. It defaults to a uniform weight for every observation. This property is read-only.
A numeric vector of column indices into X naming the predictors
treated as categorical, and empty when none is. This property is
read-only.
A cell array of character vectors. It matches PredictorNames
unless a categorical predictor was expanded into indicator variables.
This property is read-only.
A cell array with one entry per predictor, holding that predictor’s bin edges where the learner discretized it before fitting. It is empty here and stays empty: this learner fits the predictors as they are, and MATLAB’s reports an empty cell for it as well.
This property is read-only.
Always empty. It is declared for MATLAB compatibility, where it holds what an automatic search over the hyperparameters found. This class fits the parameters it is given and runs no such search, so there is nothing to report. This property is read-only.
A function handle, applied by predict and resubPredict to
the network’s output. It defaults to the identity and may be set after
construction, either to a handle or to the name of a supported
transformation.
The RegressionNeuralNetwork class offers the following public methods:
RegressionNeuralNetwork: obj = RegressionNeuralNetwork (X, Y)
RegressionNeuralNetwork: obj = RegressionNeuralNetwork (…, name, value)
See the class documentation for the accepted Name-Value pairs.
See also: fitrnet, RegressionNeuralNetwork
RegressionNeuralNetwork: yFit = predict (obj, XC)
yFit = predict (obj, XC) returns a column
vector holding the predicted response for each row of XC, using
the network stored in obj.
RegressionNeuralNetwork class object.
The transformation named by ResponseTransform is applied to the
network’s output before it is returned.
See also: RegressionNeuralNetwork, fitrnet
RegressionNeuralNetwork: yFit = resubPredict (obj)
yFit = resubPredict (obj) returns a column vector
holding the predicted response for every observation the model was
trained on, that is the rows of obj.X selected by
obj.RowsUsed.
RegressionNeuralNetwork class object.
See also: RegressionNeuralNetwork, fitrnet
RegressionNeuralNetwork: L = loss (obj, X, Y)
RegressionNeuralNetwork: L = loss (…, name, value)
L = loss (obj, X, Y) returns the
weighted mean squared error between the response Y and the
response the model predicts for X.
RegressionNeuralNetwork class object.
L = loss (…, name, value) accepts the
following Name-Value pairs.
| Name | Value |
|---|---|
'LossFun' | 'mse', the default, or a function
handle called as lossfun (Y, yFit, W)
and returning a scalar. |
'Weights' | A numeric vector of observation weights with one entry per row of X. It defaults to a uniform weight. The weights are normalized to sum to one before the loss is formed, so scaling them all by the same factor leaves the loss unchanged. |
See also: RegressionNeuralNetwork, fitrnet
RegressionNeuralNetwork: L = resubLoss (obj)
RegressionNeuralNetwork: L = resubLoss (…, name, value)
L = resubLoss (obj) returns the weighted mean
squared error of the model on the data it was trained on. It accepts
the same Name-Value pairs as loss.
RegressionNeuralNetwork class object.
See also: RegressionNeuralNetwork, fitrnet
RegressionNeuralNetwork: CVMdl = crossval (obj)
RegressionNeuralNetwork: CVMdl = crossval (…, name, value)
CVMdl = crossval (obj) returns a
RegressionPartitionedModel holding one refit of obj per
fold of a ten-fold partition, or of an -fold one where the
model has fewer than ten observations.
RegressionNeuralNetwork class object.
CVMdl = crossval (…, name, value)
accepts one, and only one, of the following Name-Value pairs.
| Name | Value |
|---|---|
'KFold' | An integer greater than 1, the number of folds. |
'Holdout' | A scalar in , the fraction of observations held out for testing. |
'Leaveout' | 'on' or 'off', whether
to hold out one observation at a time. |
'CVPartition' | A cvpartition object over as
many observations as the model was trained on. |
See also: RegressionNeuralNetwork, RegressionPartitionedModel, cvpartition
RegressionNeuralNetwork: CMdl = compact (obj)
CMdl = compact (obj) returns a compact version of
the RegressionNeuralNetwork object obj, which keeps the
trained network but drops the training data, so it predicts identically
while carrying no observations.
See also: fitrnet, RegressionNeuralNetwork, CompactRegressionNeuralNetwork
RegressionNeuralNetwork: savemodel (obj, filename)
savemodel (obj, filename) saves every property of
the RegressionNeuralNetwork object obj into
filename in binary format, so that it can be read back with
loadmodel.
See also: loadmodel, RegressionNeuralNetwork, fitrnet