Categories &

Functions List

Function Reference: fitrnet

statistics: Mdl = fitrnet (X, Y)
statistics: Mdl = fitrnet (…, name, value)

Fit a neural network regression model.

Mdl = fitrnet (X, Y) returns a neural network regression model, Mdl, with X being the predictor data and Y the continuous response of the observations in X.

  • X must be an NxP numeric matrix of predictor data, where rows correspond to observations and columns to features or variables.
  • Y must be an Nx1 numeric vector holding the response of the corresponding predictor data in X. Y must have the same number of rows as X.

The network is trained against the mean squared error and its output layer applies the identity, so a prediction is an unrestricted real number. Use fitcnet where the response names a class rather than a quantity.

Mdl = fitrnet (…, name, value) returns a neural network regression model with additional options specified by Name-Value pair arguments listed below.

Model Parameters

NameValue
'Standardize'A logical scalar indicating whether the data in X should be centred and scaled before training. The same transformation is applied by predict. The default is false.
'PredictorNames'A cell array of character vectors specifying the predictor variable names, in the order they appear in X.
'ResponseName'A character vector specifying the name of the response variable. The default is 'Y'.
'ResponseTransform'A character vector naming one of 'none', 'identity', 'exp' or 'log', or a function handle of one argument, applied to the predicted response by predict and resubPredict. The default is 'none'.
'LayerSizes'A vector of positive integers defining the number of units in each fully connected hidden layer. The default value is 10, a single hidden layer of ten units.
'LearningRate'A positive scalar value that defines the learning rate during the gradient descent. Default value is 0.003. A larger rate can drive every unit of a hidden layer negative, after which a rectifier passes no gradient and the network stops training. Applies only when 'Solver' is 'sgd'.
'Solver'A character vector naming the solver that trains the network, either 'lbfgs' or 'sgd'. The default is 'lbfgs', which minimizes the loss over the whole training set at once by limited-memory BFGS, as MATLAB does. It takes no learning rate, stops on the three tolerances below, and reaches a lower training loss in fewer passes over the data, though each of its iterations costs several passes where an epoch costs one. 'sgd' visits the samples one at a time and steps down the gradient of each, running for 'IterationLimit' epochs; it was the default before version 1.9.0.
'GradientTolerance'A nonnegative scalar. Training stops once the gradient’s infinity norm falls to or below it, which is the quantity MATLAB tests too. The default is 1e-6. Applies only when 'Solver' is 'lbfgs'.
'StepTolerance'A nonnegative scalar. Training stops once the step’s infinity norm falls to or below it, which is the quantity MATLAB tests too. The default is 1e-6. Applies only when 'Solver' is 'lbfgs'.
'LossTolerance'A real scalar. Training stops once the training loss falls to or below it. The test is on the loss itself and not on its change, matching MATLAB; pass -Inf to switch it off. The default is 1e-6. Applies only when 'Solver' is 'lbfgs'.
'Activations'A character vector or a cellstr vector specifying the activation functions for the hidden layers of the neural network, excluding the output layer. The available activation functions are 'linear', 'none', 'sigmoid', 'relu', 'tanh', 'lrelu', 'prelu', 'elu' and 'gelu'. The default value is 'relu'.
'OutputLayerActivation'A character vector specifying the activation function for the output layer. The available functions are the same as for 'Activations'. The default value is 'none', the identity, which is what a regression output calls for; anything else bounds the prediction to that function’s range.
'IterationLimit'A positive integer scalar specifying the maximum number of training iterations. The default value is 1000. Under 'sgd' this counts epochs, under 'lbfgs' solver iterations.
'DisplayInfo'A logical scalar indicating whether to print information during training. Default is false.

Source Code: fitrnet

The weights of each layer are drawn from a uniform range whose half-width is set by that layer’s activation, and the scheme cannot be chosen: a rectifying activation ('relu', 'lrelu', 'prelu', 'elu', 'gelu') takes the He range sqrt (6 / fan_in), because it passes only half of its input, and the remaining activations take the Glorot range sqrt (6 / (fan_in + fan_out)), which accounts for the backward pass as well. A network whose layers do not share an activation is therefore built with both schemes. What each layer was given is reported by the LayerWeightsInitializers field of the fitted model’s ModelParameters.

See also: RegressionNeuralNetwork, fitcnet, fcnntrain, fcnnpredict

Source Code: fitrnet

  1. Predict fuel economy from engine power and weight
 load carsmall
 X = [Horsepower, Weight];
 Mdl = fitrnet (X, MPG, 'Standardize', true, 'IterationLimit', 500);

Rows carrying a missing value were dropped, so ask about the ones used

 used = Mdl.RowsUsed;
 yFit = predict (Mdl, X(used,:));
 plot (MPG(used), yFit, 'o', [5, 45], [5, 45], 'k-');
 axis equal;
 xlabel ('Observed MPG');
 ylabel ('Predicted MPG');
 title (sprintf ('Neural network fit, RMSE %.2f', sqrt (resubLoss (Mdl))));
plotted figure

  1. Watching the fit converge
 load carsmall
 Mdl = fitrnet ([Horsepower, Weight], MPG, 'Standardize', true, ...
                'IterationLimit', 400);

TrainingHistory records the mean squared error at every iteration

 h = Mdl.TrainingHistory;
 semilogy (h.Iteration, h.TrainingLoss, 'linewidth', 1.5);
 xlabel ('Iteration');
 ylabel ('Training MSE');
 title ('The loss recorded is the network''s own, not a running average');
plotted figure

  1. Standardizing matters when the predictors differ in scale

Horsepower runs to a few hundred and Weight to a few thousand, so the heavier column dominates the first layer until both are put on one scale.

 load carsmall
 X = [Horsepower, Weight];
 raw = fitrnet (X, MPG, 'IterationLimit', 400);
 std_ = fitrnet (X, MPG, 'Standardize', true, 'IterationLimit', 400);
 printf ('RMSE, raw predictors : %.2f\n', sqrt (resubLoss (raw)));
RMSE, raw predictors : 4.07
 printf ('RMSE, standardized   : %.2f\n', sqrt (resubLoss (std_)));
RMSE, standardized   : 4.05
  1. A network recovers a curve a straight line cannot
 rand ('seed', 7);
 randn ('seed', 7);
 x = linspace (-3, 3, 120)';
 y = sin (x) + randn (120, 1) * 0.1;
 Mdl = fitrnet (x, y, 'LayerSizes', [16, 16], 'IterationLimit', 800);
 plot (x, y, 'o', 'markersize', 4);
 hold on;
 plot (x, predict (Mdl, x), 'r-', 'linewidth', 2);
 plot (x, [ones(120,1), x] * ([ones(120,1), x] \ y), 'k--', 'linewidth', 1.5);
 hold off;
 legend ({'data', 'neural network', 'least squares line'});
 title ('Two hidden layers of sixteen units');
plotted figure