Categories &

Functions List

Class Definition: ClassificationPartitionedModel

statistics: ClassificationPartitionedModel

Cross-validated classification model

The ClassificationPartitionedModel class stores cross-validated classification models trained on different partitions of the data. It can predict responses for observations not used for training using the kfoldPredict method.

Create a ClassificationPartitionedModel object by using the crossval function.

See also: crossval

Source Code: ClassificationPartitionedModel

The ClassificationPartitionedModel class contains the following properties:

An array of unique values of the response variable Y, which has the same data types as the data in Y. This property is read-only. ClassNames can have any of the following datatypes:

  • Cell array of character vectors
  • Character array
  • Logical vector
  • Numeric vector 
 load fisheriris
 x = meas;
 y = species;

Create a KNN classifier model

 obj = fitcknn (x, y, 'NumNeighbors', 5, 'Standardize', 1);

Create a partition for 5-fold cross-validation

 partition = cvpartition (y, 'KFold', 5);

Create the ClassificationPartitionedModel object

 cvModel = crossval (obj, 'cvPartition', partition)
cvModel =

  ClassificationPartitionedModel

      CrossValidatedModel: 'KNN'
             ResponseName: 'Y'
               ClassNames: {'setosa' 'versicolor' 'virginica'}
          NumObservations: 150
                    KFold: 5
           ScoreTransform: 'none'
 load fisheriris
 x = meas;
 y = species;

Create a KNN classifier model

 obj = fitcknn (x, y, 'NumNeighbors', 5, 'Standardize', 1);

Create the ClassificationPartitionedModel object

 cvModel = crossval (obj);

Predict the class labels for the observations not used for training

 [label, score, cost] = kfoldPredict (cvModel);
 fprintf ("Cross-validated accuracy = %1.2f%% (%d/%d)\n", ...
          sum (strcmp (label, y)) / numel (y) *100, ...
          sum (strcmp (label, y)), numel (y))
Cross-validated accuracy = 95.33% (143/150)

A square matrix specifying the cost of misclassification of a point. Cost(i,j) is the cost of classifying a point into class j if its true class is i (that is, the rows correspond to the true class and the columns correspond to the predicted class). The order of the rows and columns in Cost corresponds to the order of the classes in ClassNames. The number of rows and columns in Cost is the number of unique classes in the response. By default, Cost(i,j) = 1 if i != j, and Cost(i,j) = 0 if i = j. In other words, the cost is 0 for correct classification and 1 for incorrect classification.

Assigning Cost rebuilds it on every fold in Trained, so kfoldPredict and kfoldLoss answer under the new costs. It is refused on a cross-validated ClassificationSVM, whose costs enter the box constraint while it is being fitted: a model already fitted under one cost matrix cannot be made to describe another.

A cost may also be given as a struct with the fields ClassNames and ClassificationCosts, which names the order its own matrix is written in. That matrix is permuted into the order of ClassNames above, so a caller need not know which order the classes were sorted into. It must name every class.

A cost must be floating point, not sparse, not complex, non-negative and zero down its diagonal, and must hold no NaN or Inf. A single is widened to double. 

 load fisheriris
 x = meas;
 y = species;

Create a KNN classifier model

 obj = fitcknn (x, y, 'NumNeighbors', 5, 'Standardize', 1);

Create a partition for 5-fold cross-validation

 partition = cvpartition (y, 'KFold', 5);

Create the ClassificationPartitionedModel object

 cvModel = crossval (obj, 'cvPartition', partition)
cvModel =

  ClassificationPartitionedModel

      CrossValidatedModel: 'KNN'
             ResponseName: 'Y'
               ClassNames: {'setosa' 'versicolor' 'virginica'}
          NumObservations: 150
                    KFold: 5
           ScoreTransform: 'none'
 load fisheriris
 x = meas;
 y = species;

Create a KNN classifier model

 obj = fitcknn (x, y, 'NumNeighbors', 5, 'Standardize', 1);

Create the ClassificationPartitionedModel object

 cvModel = crossval (obj);

Predict the class labels for the observations not used for training

 [label, score, cost] = kfoldPredict (cvModel);
 fprintf ("Cross-validated accuracy = %1.2f%% (%d/%d)\n", ...
          sum (strcmp (label, y)) / numel (y) *100, ...
          sum (strcmp (label, y)), numel (y))
Cross-validated accuracy = 95.33% (143/150)

A numeric vector specifying the prior probabilities for each class. The order of the elements in Prior corresponds to the order of the classes in ClassNames.

It may be assigned only on a cross-validated ClassificationDiscriminant or ClassificationNaiveBayes, the two learners that score from the priors they are given rather than consuming them while they fit: the discriminant re-derives its coefficients from them and the naive Bayes weights its class densities by them. Every other learner cannot revisit them afterwards. Assigning it rebuilds the priors on every fold in Trained. 

 load fisheriris
 x = meas;
 y = species;

Create a KNN classifier model

 obj = fitcknn (x, y, 'NumNeighbors', 5, 'Standardize', 1);

Create a partition for 5-fold cross-validation

 partition = cvpartition (y, 'KFold', 5);

Create the ClassificationPartitionedModel object

 cvModel = crossval (obj, 'cvPartition', partition)
cvModel =

  ClassificationPartitionedModel

      CrossValidatedModel: 'KNN'
             ResponseName: 'Y'
               ClassNames: {'setosa' 'versicolor' 'virginica'}
          NumObservations: 150
                    KFold: 5
           ScoreTransform: 'none'
 load fisheriris
 x = meas;
 y = species;

Create a KNN classifier model

 obj = fitcknn (x, y, 'NumNeighbors', 5, 'Standardize', 1);

Create the ClassificationPartitionedModel object

 cvModel = crossval (obj);

Predict the class labels for the observations not used for training

 [label, score, cost] = kfoldPredict (cvModel);
 fprintf ("Cross-validated accuracy = %1.2f%% (%d/%d)\n", ...
          sum (strcmp (label, y)) / numel (y) *100, ...
          sum (strcmp (label, y)), numel (y))
Cross-validated accuracy = 95.33% (143/150)

Specified as a function handle for transforming the classification scores. 

 load fisheriris
 x = meas;
 y = species;

Create a KNN classifier model

 obj = fitcknn (x, y, 'NumNeighbors', 5, 'Standardize', 1);

Create a partition for 5-fold cross-validation

 partition = cvpartition (y, 'KFold', 5);

Create the ClassificationPartitionedModel object

 cvModel = crossval (obj, 'cvPartition', partition)
cvModel =

  ClassificationPartitionedModel

      CrossValidatedModel: 'KNN'
             ResponseName: 'Y'
               ClassNames: {'setosa' 'versicolor' 'virginica'}
          NumObservations: 150
                    KFold: 5
           ScoreTransform: 'none'
 load fisheriris
 x = meas;
 y = species;

Create a KNN classifier model

 obj = fitcknn (x, y, 'NumNeighbors', 5, 'Standardize', 1);

Create the ClassificationPartitionedModel object

 cvModel = crossval (obj);

Predict the class labels for the observations not used for training

 [label, score, cost] = kfoldPredict (cvModel);
 fprintf ("Cross-validated accuracy = %1.2f%% (%d/%d)\n", ...
          sum (strcmp (label, y)) / numel (y) *100, ...
          sum (strcmp (label, y)), numel (y))
Cross-validated accuracy = 92.67% (139/150)

A character vector holding the short name of the learner that was cross validated, as MATLAB reports it: 'Discriminant', 'GAM', 'KNN', 'NeuralNetwork' or 'SVM'. It is not the class name of that learner, and the regression side uses the same names. This property is read-only. 

 load fisheriris
 x = meas;
 y = species;

Create a KNN classifier model

 obj = fitcknn (x, y, 'NumNeighbors', 5, 'Standardize', 1);

Create a partition for 5-fold cross-validation

 partition = cvpartition (y, 'KFold', 5);

Create the ClassificationPartitionedModel object

 cvModel = crossval (obj, 'cvPartition', partition)
cvModel =

  ClassificationPartitionedModel

      CrossValidatedModel: 'KNN'
             ResponseName: 'Y'
               ClassNames: {'setosa' 'versicolor' 'virginica'}
          NumObservations: 150
                    KFold: 5
           ScoreTransform: 'none'
 load fisheriris
 x = meas;
 y = species;

Create a KNN classifier model

 obj = fitcknn (x, y, 'NumNeighbors', 5, 'Standardize', 1);

Create the ClassificationPartitionedModel object

 cvModel = crossval (obj);

Predict the class labels for the observations not used for training

 [label, score, cost] = kfoldPredict (cvModel);
 fprintf ("Cross-validated accuracy = %1.2f%% (%d/%d)\n", ...
          sum (strcmp (label, y)) / numel (y) *100, ...
          sum (strcmp (label, y)), numel (y))
Cross-validated accuracy = 95.33% (143/150)

A cell array of character vectors specifying the names of the predictor variables. The names are in the order in which they appear in the training dataset. This property is read-only. 

 load fisheriris
 x = meas;
 y = species;

Create a KNN classifier model

 obj = fitcknn (x, y, 'NumNeighbors', 5, 'Standardize', 1);

Create a partition for 5-fold cross-validation

 partition = cvpartition (y, 'KFold', 5);

Create the ClassificationPartitionedModel object

 cvModel = crossval (obj, 'cvPartition', partition)
cvModel =

  ClassificationPartitionedModel

      CrossValidatedModel: 'KNN'
             ResponseName: 'Y'
               ClassNames: {'setosa' 'versicolor' 'virginica'}
          NumObservations: 150
                    KFold: 5
           ScoreTransform: 'none'
 load fisheriris
 x = meas;
 y = species;

Create a KNN classifier model

 obj = fitcknn (x, y, 'NumNeighbors', 5, 'Standardize', 1);

Create the ClassificationPartitionedModel object

 cvModel = crossval (obj);

Predict the class labels for the observations not used for training

 [label, score, cost] = kfoldPredict (cvModel);
 fprintf ("Cross-validated accuracy = %1.2f%% (%d/%d)\n", ...
          sum (strcmp (label, y)) / numel (y) *100, ...
          sum (strcmp (label, y)), numel (y))
Cross-validated accuracy = 94.67% (142/150)

A vector of positive integers specifying the indices of categorical predictors. This property is read-only. 

 load fisheriris
 x = meas;
 y = species;

Create a KNN classifier model

 obj = fitcknn (x, y, 'NumNeighbors', 5, 'Standardize', 1);

Create a partition for 5-fold cross-validation

 partition = cvpartition (y, 'KFold', 5);

Create the ClassificationPartitionedModel object

 cvModel = crossval (obj, 'cvPartition', partition)
cvModel =

  ClassificationPartitionedModel

      CrossValidatedModel: 'KNN'
             ResponseName: 'Y'
               ClassNames: {'setosa' 'versicolor' 'virginica'}
          NumObservations: 150
                    KFold: 5
           ScoreTransform: 'none'
 load fisheriris
 x = meas;
 y = species;

Create a KNN classifier model

 obj = fitcknn (x, y, 'NumNeighbors', 5, 'Standardize', 1);

Create the ClassificationPartitionedModel object

 cvModel = crossval (obj);

Predict the class labels for the observations not used for training

 [label, score, cost] = kfoldPredict (cvModel);
 fprintf ("Cross-validated accuracy = %1.2f%% (%d/%d)\n", ...
          sum (strcmp (label, y)) / numel (y) *100, ...
          sum (strcmp (label, y)), numel (y))
Cross-validated accuracy = 94.67% (142/150)

A character vector specifying the name of the response variable Y. This property is read-only. 

 load fisheriris
 x = meas;
 y = species;

Create a KNN classifier model

 obj = fitcknn (x, y, 'NumNeighbors', 5, 'Standardize', 1);

Create a partition for 5-fold cross-validation

 partition = cvpartition (y, 'KFold', 5);

Create the ClassificationPartitionedModel object

 cvModel = crossval (obj, 'cvPartition', partition)
cvModel =

  ClassificationPartitionedModel

      CrossValidatedModel: 'KNN'
             ResponseName: 'Y'
               ClassNames: {'setosa' 'versicolor' 'virginica'}
          NumObservations: 150
                    KFold: 5
           ScoreTransform: 'none'
 load fisheriris
 x = meas;
 y = species;

Create a KNN classifier model

 obj = fitcknn (x, y, 'NumNeighbors', 5, 'Standardize', 1);

Create the ClassificationPartitionedModel object

 cvModel = crossval (obj);

Predict the class labels for the observations not used for training

 [label, score, cost] = kfoldPredict (cvModel);
 fprintf ("Cross-validated accuracy = %1.2f%% (%d/%d)\n", ...
          sum (strcmp (label, y)) / numel (y) *100, ...
          sum (strcmp (label, y)), numel (y))
Cross-validated accuracy = 94.00% (141/150)

A positive integer value specifying the number of observations in the training dataset used for training the cross-validated model. This property is read-only. 

 load fisheriris
 x = meas;
 y = species;

Create a KNN classifier model

 obj = fitcknn (x, y, 'NumNeighbors', 5, 'Standardize', 1);

Create a partition for 5-fold cross-validation

 partition = cvpartition (y, 'KFold', 5);

Create the ClassificationPartitionedModel object

 cvModel = crossval (obj, 'cvPartition', partition)
cvModel =

  ClassificationPartitionedModel

      CrossValidatedModel: 'KNN'
             ResponseName: 'Y'
               ClassNames: {'setosa' 'versicolor' 'virginica'}
          NumObservations: 150
                    KFold: 5
           ScoreTransform: 'none'
 load fisheriris
 x = meas;
 y = species;

Create a KNN classifier model

 obj = fitcknn (x, y, 'NumNeighbors', 5, 'Standardize', 1);

Create the ClassificationPartitionedModel object

 cvModel = crossval (obj);

Predict the class labels for the observations not used for training

 [label, score, cost] = kfoldPredict (cvModel);
 fprintf ("Cross-validated accuracy = %1.2f%% (%d/%d)\n", ...
          sum (strcmp (label, y)) / numel (y) *100, ...
          sum (strcmp (label, y)), numel (y))
Cross-validated accuracy = 94.67% (142/150)

A numeric matrix containing the unstandardized predictor data. Each column of X represents one predictor (variable), and each row represents one observation. This property is read-only. 

 load fisheriris
 x = meas;
 y = species;

Create a KNN classifier model

 obj = fitcknn (x, y, 'NumNeighbors', 5, 'Standardize', 1);

Create a partition for 5-fold cross-validation

 partition = cvpartition (y, 'KFold', 5);

Create the ClassificationPartitionedModel object

 cvModel = crossval (obj, 'cvPartition', partition)
cvModel =

  ClassificationPartitionedModel

      CrossValidatedModel: 'KNN'
             ResponseName: 'Y'
               ClassNames: {'setosa' 'versicolor' 'virginica'}
          NumObservations: 150
                    KFold: 5
           ScoreTransform: 'none'
 load fisheriris
 x = meas;
 y = species;

Create a KNN classifier model

 obj = fitcknn (x, y, 'NumNeighbors', 5, 'Standardize', 1);

Create the ClassificationPartitionedModel object

 cvModel = crossval (obj);

Predict the class labels for the observations not used for training

 [label, score, cost] = kfoldPredict (cvModel);
 fprintf ("Cross-validated accuracy = %1.2f%% (%d/%d)\n", ...
          sum (strcmp (label, y)) / numel (y) *100, ...
          sum (strcmp (label, y)), numel (y))
Cross-validated accuracy = 95.33% (143/150)

Specified as a logical or numeric column vector, or as a character array or a cell array of character vectors with the same number of rows as the predictor data. Each row in Y is the observed class label for the corresponding row in X. This property is read-only. 

 load fisheriris
 x = meas;
 y = species;

Create a KNN classifier model

 obj = fitcknn (x, y, 'NumNeighbors', 5, 'Standardize', 1);

Create a partition for 5-fold cross-validation

 partition = cvpartition (y, 'KFold', 5);

Create the ClassificationPartitionedModel object

 cvModel = crossval (obj, 'cvPartition', partition)
cvModel =

  ClassificationPartitionedModel

      CrossValidatedModel: 'KNN'
             ResponseName: 'Y'
               ClassNames: {'setosa' 'versicolor' 'virginica'}
          NumObservations: 150
                    KFold: 5
           ScoreTransform: 'none'
 load fisheriris
 x = meas;
 y = species;

Create a KNN classifier model

 obj = fitcknn (x, y, 'NumNeighbors', 5, 'Standardize', 1);

Create the ClassificationPartitionedModel object

 cvModel = crossval (obj);

Predict the class labels for the observations not used for training

 [label, score, cost] = kfoldPredict (cvModel);
 fprintf ("Cross-validated accuracy = %1.2f%% (%d/%d)\n", ...
          sum (strcmp (label, y)) / numel (y) *100, ...
          sum (strcmp (label, y)), numel (y))
Cross-validated accuracy = 94.00% (141/150)

A numeric column vector with one entry per observation, carried over from the model that was cross validated. This property is read-only. 

 load fisheriris
 x = meas;
 y = species;

Create a KNN classifier model

 obj = fitcknn (x, y, 'NumNeighbors', 5, 'Standardize', 1);

Create a partition for 5-fold cross-validation

 partition = cvpartition (y, 'KFold', 5);

Create the ClassificationPartitionedModel object

 cvModel = crossval (obj, 'cvPartition', partition)
cvModel =

  ClassificationPartitionedModel

      CrossValidatedModel: 'KNN'
             ResponseName: 'Y'
               ClassNames: {'setosa' 'versicolor' 'virginica'}
          NumObservations: 150
                    KFold: 5
           ScoreTransform: 'none'
 load fisheriris
 x = meas;
 y = species;

Create a KNN classifier model

 obj = fitcknn (x, y, 'NumNeighbors', 5, 'Standardize', 1);

Create the ClassificationPartitionedModel object

 cvModel = crossval (obj);

Predict the class labels for the observations not used for training

 [label, score, cost] = kfoldPredict (cvModel);
 fprintf ("Cross-validated accuracy = %1.2f%% (%d/%d)\n", ...
          sum (strcmp (label, y)) / numel (y) *100, ...
          sum (strcmp (label, y)), numel (y))
Cross-validated accuracy = 94.00% (141/150)

A structure holding the parameters the folds were fitted with, carried through from the learner that was cross validated, beside NLearn, the number of folds, and the Version, Method and Type tags of this class. The learner’s own tags are replaced rather than kept, so a cross-validated SVM reports Method as 'PartitionedModel' and not 'SVM'.

Deviation from MATLAB. MATLAB reports the parameter record of the cross-validation ensemble here rather than of the learner, so it says nothing at all about how the folds were fitted: of its eighteen fields only the fold count, its partitioner and a fit template carry anything, and the rest are boosting settings left inert. Nor can the parameters be reached through the folds, a compact model carrying none in MATLAB. This class reports the fit instead, which is strictly more than MATLAB offers, and everything MATLAB’s record does carry is published here as the KFold, Partition, X, Y, W and CrossValidatedModel properties.

This property is read-only. 

 load fisheriris
 x = meas;
 y = species;

Create a KNN classifier model

 obj = fitcknn (x, y, 'NumNeighbors', 5, 'Standardize', 1);

Create a partition for 5-fold cross-validation

 partition = cvpartition (y, 'KFold', 5);

Create the ClassificationPartitionedModel object

 cvModel = crossval (obj, 'cvPartition', partition)
cvModel =

  ClassificationPartitionedModel

      CrossValidatedModel: 'KNN'
             ResponseName: 'Y'
               ClassNames: {'setosa' 'versicolor' 'virginica'}
          NumObservations: 150
                    KFold: 5
           ScoreTransform: 'none'
 load fisheriris
 x = meas;
 y = species;

Create a KNN classifier model

 obj = fitcknn (x, y, 'NumNeighbors', 5, 'Standardize', 1);

Create the ClassificationPartitionedModel object

 cvModel = crossval (obj);

Predict the class labels for the observations not used for training

 [label, score, cost] = kfoldPredict (cvModel);
 fprintf ("Cross-validated accuracy = %1.2f%% (%d/%d)\n", ...
          sum (strcmp (label, y)) / numel (y) *100, ...
          sum (strcmp (label, y)), numel (y))
Cross-validated accuracy = 95.33% (143/150)

A cell array of models trained on each fold. Each cell contains a model trained on the minus-one fold of the data (all but one fold used for training and the remaining fold used for validation). This property is read-only. 

 load fisheriris
 x = meas;
 y = species;

Create a KNN classifier model

 obj = fitcknn (x, y, 'NumNeighbors', 5, 'Standardize', 1);

Create a partition for 5-fold cross-validation

 partition = cvpartition (y, 'KFold', 5);

Create the ClassificationPartitionedModel object

 cvModel = crossval (obj, 'cvPartition', partition)
cvModel =

  ClassificationPartitionedModel

      CrossValidatedModel: 'KNN'
             ResponseName: 'Y'
               ClassNames: {'setosa' 'versicolor' 'virginica'}
          NumObservations: 150
                    KFold: 5
           ScoreTransform: 'none'
 load fisheriris
 x = meas;
 y = species;

Create a KNN classifier model

 obj = fitcknn (x, y, 'NumNeighbors', 5, 'Standardize', 1);

Create the ClassificationPartitionedModel object

 cvModel = crossval (obj);

Predict the class labels for the observations not used for training

 [label, score, cost] = kfoldPredict (cvModel);
 fprintf ("Cross-validated accuracy = %1.2f%% (%d/%d)\n", ...
          sum (strcmp (label, y)) / numel (y) *100, ...
          sum (strcmp (label, y)), numel (y))
Cross-validated accuracy = 96.00% (144/150)

A positive integer value specifying the number of cross-validated folds. This property is read-only. 

 load fisheriris
 x = meas;
 y = species;

Create a KNN classifier model

 obj = fitcknn (x, y, 'NumNeighbors', 5, 'Standardize', 1);

Create a partition for 5-fold cross-validation

 partition = cvpartition (y, 'KFold', 5);

Create the ClassificationPartitionedModel object

 cvModel = crossval (obj, 'cvPartition', partition)
cvModel =

  ClassificationPartitionedModel

      CrossValidatedModel: 'KNN'
             ResponseName: 'Y'
               ClassNames: {'setosa' 'versicolor' 'virginica'}
          NumObservations: 150
                    KFold: 5
           ScoreTransform: 'none'
 load fisheriris
 x = meas;
 y = species;

Create a KNN classifier model

 obj = fitcknn (x, y, 'NumNeighbors', 5, 'Standardize', 1);

Create the ClassificationPartitionedModel object

 cvModel = crossval (obj);

Predict the class labels for the observations not used for training

 [label, score, cost] = kfoldPredict (cvModel);
 fprintf ("Cross-validated accuracy = %1.2f%% (%d/%d)\n", ...
          sum (strcmp (label, y)) / numel (y) *100, ...
          sum (strcmp (label, y)), numel (y))
Cross-validated accuracy = 94.67% (142/150)

A cvpartition object specifying the partition configuration used for cross-validation. This field stores the cvpartition instance that describes how the data was split into training and validation sets. This property is read-only. 

 load fisheriris
 x = meas;
 y = species;

Create a KNN classifier model

 obj = fitcknn (x, y, 'NumNeighbors', 5, 'Standardize', 1);

Create a partition for 5-fold cross-validation

 partition = cvpartition (y, 'KFold', 5);

Create the ClassificationPartitionedModel object

 cvModel = crossval (obj, 'cvPartition', partition)
cvModel =

  ClassificationPartitionedModel

      CrossValidatedModel: 'KNN'
             ResponseName: 'Y'
               ClassNames: {'setosa' 'versicolor' 'virginica'}
          NumObservations: 150
                    KFold: 5
           ScoreTransform: 'none'
 load fisheriris
 x = meas;
 y = species;

Create a KNN classifier model

 obj = fitcknn (x, y, 'NumNeighbors', 5, 'Standardize', 1);

Create the ClassificationPartitionedModel object

 cvModel = crossval (obj);

Predict the class labels for the observations not used for training

 [label, score, cost] = kfoldPredict (cvModel);
 fprintf ("Cross-validated accuracy = %1.2f%% (%d/%d)\n", ...
          sum (strcmp (label, y)) / numel (y) *100, ...
          sum (strcmp (label, y)), numel (y))
Cross-validated accuracy = 94.00% (141/150)

A cell array with one entry per predictor, holding that predictor’s bin edges where the learner discretized it before fitting. It is carried over from the model that was cross validated, and is empty whenever that model did no binning, which is every learner this package implements: MATLAB fills it only for its GAM, which bins because it is built from boosted trees where ours is built from splines.

This property is read-only. 

 load fisheriris
 x = meas;
 y = species;

Create a KNN classifier model

 obj = fitcknn (x, y, 'NumNeighbors', 5, 'Standardize', 1);

Create a partition for 5-fold cross-validation

 partition = cvpartition (y, 'KFold', 5);

Create the ClassificationPartitionedModel object

 cvModel = crossval (obj, 'cvPartition', partition)
cvModel =

  ClassificationPartitionedModel

      CrossValidatedModel: 'KNN'
             ResponseName: 'Y'
               ClassNames: {'setosa' 'versicolor' 'virginica'}
          NumObservations: 150
                    KFold: 5
           ScoreTransform: 'none'
 load fisheriris
 x = meas;
 y = species;

Create a KNN classifier model

 obj = fitcknn (x, y, 'NumNeighbors', 5, 'Standardize', 1);

Create the ClassificationPartitionedModel object

 cvModel = crossval (obj);

Predict the class labels for the observations not used for training

 [label, score, cost] = kfoldPredict (cvModel);
 fprintf ("Cross-validated accuracy = %1.2f%% (%d/%d)\n", ...
          sum (strcmp (label, y)) / numel (y) *100, ...
          sum (strcmp (label, y)), numel (y))
Cross-validated accuracy = 94.67% (142/150)

A scalar structure with fields PredictorTrees and InteractionTrees, each a row with one entry per fold, for a generalized additive model backing, and empty for every other.

It reports what each fold actually fitted, which the budget in ModelParameters does not: a phase stops early when it can no longer improve the fit, and the folds need not stop at the same place.

MATLAB carries this on its per-learner partitioned GAM classes, which this package deliberately does not have (see crossval), so like IsStandardDeviationFit it is declared here for every backing and left empty where it does not apply.

This property is read-only. 

 load fisheriris
 x = meas;
 y = species;

Create a KNN classifier model

 obj = fitcknn (x, y, 'NumNeighbors', 5, 'Standardize', 1);

Create a partition for 5-fold cross-validation

 partition = cvpartition (y, 'KFold', 5);

Create the ClassificationPartitionedModel object

 cvModel = crossval (obj, 'cvPartition', partition)
cvModel =

  ClassificationPartitionedModel

      CrossValidatedModel: 'KNN'
             ResponseName: 'Y'
               ClassNames: {'setosa' 'versicolor' 'virginica'}
          NumObservations: 150
                    KFold: 5
           ScoreTransform: 'none'
 load fisheriris
 x = meas;
 y = species;

Create a KNN classifier model

 obj = fitcknn (x, y, 'NumNeighbors', 5, 'Standardize', 1);

Create the ClassificationPartitionedModel object

 cvModel = crossval (obj);

Predict the class labels for the observations not used for training

 [label, score, cost] = kfoldPredict (cvModel);
 fprintf ("Cross-validated accuracy = %1.2f%% (%d/%d)\n", ...
          sum (strcmp (label, y)) / numel (y) *100, ...
          sum (strcmp (label, y)), numel (y))
Cross-validated accuracy = 95.33% (143/150)

The ClassificationPartitionedModel class offers the following public methods:

ClassificationPartitionedModel: this = ClassificationPartitionedModel (Mdl, Partition)

this = ClassificationPartitionedModel (Mdl, Partition) returns a ClassificationPartitionedModel object, with Mdl as the trained classification model object and Partition as the partitioning object obtained using cvpartition function.

See also: cvpartition 

 load fisheriris
 x = meas;
 y = species;

Create a KNN classifier model

 obj = fitcknn (x, y, 'NumNeighbors', 5, 'Standardize', 1);

Create a partition for 5-fold cross-validation

 partition = cvpartition (y, 'KFold', 5);

Create the ClassificationPartitionedModel object

 cvModel = crossval (obj, 'cvPartition', partition)
cvModel =

  ClassificationPartitionedModel

      CrossValidatedModel: 'KNN'
             ResponseName: 'Y'
               ClassNames: {'setosa' 'versicolor' 'virginica'}
          NumObservations: 150
                    KFold: 5
           ScoreTransform: 'none'
 load fisheriris
 x = meas;
 y = species;

Create a KNN classifier model

 obj = fitcknn (x, y, 'NumNeighbors', 5, 'Standardize', 1);

Create the ClassificationPartitionedModel object

 cvModel = crossval (obj);

Predict the class labels for the observations not used for training

 [label, score, cost] = kfoldPredict (cvModel);
 fprintf ("Cross-validated accuracy = %1.2f%% (%d/%d)\n", ...
          sum (strcmp (label, y)) / numel (y) *100, ...
          sum (strcmp (label, y)), numel (y))
Cross-validated accuracy = 93.33% (140/150)
ClassificationPartitionedModel: label = kfoldPredict (this)
ClassificationPartitionedModel: [label, score, cost] = kfoldPredict (this)

[label, Score, Cost] = kfoldPredict (this) returns the predicted class labels, classification scores, and classification costs for the data used to train the cross-validated model this.

this is a ClassificationPartitionedModel object. The function predicts the response for each observation that was held out during training in the cross-validation process.

An observation that no fold held out is not predicted at all: its scores and costs are NaN and its label is missing, an empty character vector for a cell array of strings and NaN for a numeric response. Under a 'Holdout' partition that is every observation outside the holdout set. This differs from MATLAB, which reports NaN scores for those rows as we do but labels every one of them with the first class, whatever their response: that label is the least-cost class of a row of NaN costs rather than a prediction any model made, and naming a class for an observation nothing scored would be wrong. A logical response has no missing value to give, so those rows stay false.

OutputDescription
labelPredicted class labels, returned as a vector or cell array. The type of label matches the type of Y in the original training data. Each element of label corresponds to the predicted class label for the corresponding row in X.
ScoreClassification scores, returned as a numeric matrix. Each row of Score corresponds to an observation, and each column corresponds to a class. The value in row i and column j is the classification score for class j for observation i.
CostClassification costs, returned as a numeric matrix. Each row of Cost corresponds to an observation, and each column corresponds to a class. The value in row i and column j is the classification cost for class j for observation i. This output is optional and only returned if requested.

See also: ClassificationKNN, ClassificationSVM, ClassificationPartitionedModel 

 load fisheriris
 x = meas;
 y = species;

Create a KNN classifier model

 obj = fitcknn (x, y, 'NumNeighbors', 5, 'Standardize', 1);

Create a partition for 5-fold cross-validation

 partition = cvpartition (y, 'KFold', 5);

Create the ClassificationPartitionedModel object

 cvModel = crossval (obj, 'cvPartition', partition)
cvModel =

  ClassificationPartitionedModel

      CrossValidatedModel: 'KNN'
             ResponseName: 'Y'
               ClassNames: {'setosa' 'versicolor' 'virginica'}
          NumObservations: 150
                    KFold: 5
           ScoreTransform: 'none'
 load fisheriris
 x = meas;
 y = species;

Create a KNN classifier model

 obj = fitcknn (x, y, 'NumNeighbors', 5, 'Standardize', 1);

Create the ClassificationPartitionedModel object

 cvModel = crossval (obj);

Predict the class labels for the observations not used for training

 [label, score, cost] = kfoldPredict (cvModel);
 fprintf ("Cross-validated accuracy = %1.2f%% (%d/%d)\n", ...
          sum (strcmp (label, y)) / numel (y) *100, ...
          sum (strcmp (label, y)), numel (y))
Cross-validated accuracy = 94.67% (142/150)
ClassificationPartitionedModel: L = kfoldLoss (obj)
ClassificationPartitionedModel: L = kfoldLoss (…, name, value)

L = kfoldLoss (obj) returns the fraction of observations the folds misclassify, each answered for by the fold’s model that did not see it, which is what kfoldPredict returns.

  • obj must be a ClassificationPartitionedModel object.

L = kfoldLoss (…, name, value) accepts the following Name-Value pairs.

NameValue
'LossFun''classiferror', the default, 'classifcost', 'mincost', or a function handle called as lossfun (C, S, W, Cost), where C is a logical matrix with one true per row marking the true class, S the scores, W the weights and Cost the misclassification cost.
'Mode''average', the default, which returns one number over the observations of every fold asked for, or 'individual', which returns one number per fold.
'Folds'A vector of fold indices to restrict the loss to. It defaults to every fold.

See also: ClassificationPartitionedModel, kfoldPredict 

 load fisheriris
 x = meas;
 y = species;

Create a KNN classifier model

 obj = fitcknn (x, y, 'NumNeighbors', 5, 'Standardize', 1);

Create a partition for 5-fold cross-validation

 partition = cvpartition (y, 'KFold', 5);

Create the ClassificationPartitionedModel object

 cvModel = crossval (obj, 'cvPartition', partition)
cvModel =

  ClassificationPartitionedModel

      CrossValidatedModel: 'KNN'
             ResponseName: 'Y'
               ClassNames: {'setosa' 'versicolor' 'virginica'}
          NumObservations: 150
                    KFold: 5
           ScoreTransform: 'none'
 load fisheriris
 x = meas;
 y = species;

Create a KNN classifier model

 obj = fitcknn (x, y, 'NumNeighbors', 5, 'Standardize', 1);

Create the ClassificationPartitionedModel object

 cvModel = crossval (obj);

Predict the class labels for the observations not used for training

 [label, score, cost] = kfoldPredict (cvModel);
 fprintf ("Cross-validated accuracy = %1.2f%% (%d/%d)\n", ...
          sum (strcmp (label, y)) / numel (y) *100, ...
          sum (strcmp (label, y)), numel (y))
Cross-validated accuracy = 94.67% (142/150)
ClassificationPartitionedModel: m = kfoldMargin (obj)

m = kfoldMargin (obj) returns an Nx1 vector holding, for every observation, the score its own fold’s model gave the true class less the largest score that model gave any other class. A larger margin is a more confident correct answer and a negative one is a misclassification. Every observation is scored by the fold that held it out, so no model answers for a row it was trained on.

  • obj must be a ClassificationPartitionedModel object.

Where the fold that held an observation out produced no score for it, the margin is NaN. This method takes no optional arguments, as MATLAB’s does not. 

 load fisheriris
 x = meas;
 y = species;

Create a KNN classifier model

 obj = fitcknn (x, y, 'NumNeighbors', 5, 'Standardize', 1);

Create a partition for 5-fold cross-validation

 partition = cvpartition (y, 'KFold', 5);

Create the ClassificationPartitionedModel object

 cvModel = crossval (obj, 'cvPartition', partition)
cvModel =

  ClassificationPartitionedModel

      CrossValidatedModel: 'KNN'
             ResponseName: 'Y'
               ClassNames: {'setosa' 'versicolor' 'virginica'}
          NumObservations: 150
                    KFold: 5
           ScoreTransform: 'none'
 load fisheriris
 x = meas;
 y = species;

Create a KNN classifier model

 obj = fitcknn (x, y, 'NumNeighbors', 5, 'Standardize', 1);

Create the ClassificationPartitionedModel object

 cvModel = crossval (obj);

Predict the class labels for the observations not used for training

 [label, score, cost] = kfoldPredict (cvModel);
 fprintf ("Cross-validated accuracy = %1.2f%% (%d/%d)\n", ...
          sum (strcmp (label, y)) / numel (y) *100, ...
          sum (strcmp (label, y)), numel (y))
Cross-validated accuracy = 94.00% (141/150)
ClassificationPartitionedModel: e = kfoldEdge (obj)
ClassificationPartitionedModel: e = kfoldEdge (…, name, value)

e = kfoldEdge (obj) returns the mean of the classification margins over every cross-validated observation, which is the mean of kfoldMargin (obj).

  • obj must be a ClassificationPartitionedModel object.

e = kfoldEdge (…, name, value) accepts the following Name-Value pairs.

NameValue
'Mode''average', the default, which returns one number over the observations of every fold asked for, or 'individual', which returns one number per fold.
'Folds'A vector of fold indices to restrict the edge to. It defaults to every fold.

The observations of a selection are weighted uniformly and normalized over that selection, so a subset of folds is an average rather than a sum, exactly as kfoldLoss does. 

 load fisheriris
 x = meas;
 y = species;

Create a KNN classifier model

 obj = fitcknn (x, y, 'NumNeighbors', 5, 'Standardize', 1);

Create a partition for 5-fold cross-validation

 partition = cvpartition (y, 'KFold', 5);

Create the ClassificationPartitionedModel object

 cvModel = crossval (obj, 'cvPartition', partition)
cvModel =

  ClassificationPartitionedModel

      CrossValidatedModel: 'KNN'
             ResponseName: 'Y'
               ClassNames: {'setosa' 'versicolor' 'virginica'}
          NumObservations: 150
                    KFold: 5
           ScoreTransform: 'none'
 load fisheriris
 x = meas;
 y = species;

Create a KNN classifier model

 obj = fitcknn (x, y, 'NumNeighbors', 5, 'Standardize', 1);

Create the ClassificationPartitionedModel object

 cvModel = crossval (obj);

Predict the class labels for the observations not used for training

 [label, score, cost] = kfoldPredict (cvModel);
 fprintf ("Cross-validated accuracy = %1.2f%% (%d/%d)\n", ...
          sum (strcmp (label, y)) / numel (y) *100, ...
          sum (strcmp (label, y)), numel (y))
Cross-validated accuracy = 94.67% (142/150)
ClassificationPartitionedModel: vals = kfoldfun (obj, fun)

vals = kfoldfun (obj, fun) calls fun once per fold and returns a K×M numeric matrix whose row k is what fun returned for fold k.

fun is a function handle taking seven inputs and returning a numeric vector of the same length every time it is called:

 
 testvals = fun (M, Xtrain, Ytrain, Wtrain, …
                Xtest, Ytest, Wtest)

M is the model the fold was fitted with, taken from obj.Trained{k}; Xtrain, Ytrain and Wtrain are the predictors, response and weights of the observations that fold was trained on, and Xtest, Ytest and Wtest those of the observations it held out.

See also: ClassificationPartitionedModel, kfoldPredict, kfoldLoss, kfoldMargin, kfoldEdge 

 load fisheriris
 x = meas;
 y = species;

Create a KNN classifier model

 obj = fitcknn (x, y, 'NumNeighbors', 5, 'Standardize', 1);

Create a partition for 5-fold cross-validation

 partition = cvpartition (y, 'KFold', 5);

Create the ClassificationPartitionedModel object

 cvModel = crossval (obj, 'cvPartition', partition)
cvModel =

  ClassificationPartitionedModel

      CrossValidatedModel: 'KNN'
             ResponseName: 'Y'
               ClassNames: {'setosa' 'versicolor' 'virginica'}
          NumObservations: 150
                    KFold: 5
           ScoreTransform: 'none'
 load fisheriris
 x = meas;
 y = species;

Create a KNN classifier model

 obj = fitcknn (x, y, 'NumNeighbors', 5, 'Standardize', 1);

Create the ClassificationPartitionedModel object

 cvModel = crossval (obj);

Predict the class labels for the observations not used for training

 [label, score, cost] = kfoldPredict (cvModel);
 fprintf ("Cross-validated accuracy = %1.2f%% (%d/%d)\n", ...
          sum (strcmp (label, y)) / numel (y) *100, ...
          sum (strcmp (label, y)), numel (y))
Cross-validated accuracy = 95.33% (143/150)

Examples 

 load fisheriris
 x = meas;
 y = species;

Create a KNN classifier model

 obj = fitcknn (x, y, 'NumNeighbors', 5, 'Standardize', 1);

Create a partition for 5-fold cross-validation

 partition = cvpartition (y, 'KFold', 5);

Create the ClassificationPartitionedModel object

 cvModel = crossval (obj, 'cvPartition', partition)
cvModel =

  ClassificationPartitionedModel

      CrossValidatedModel: 'KNN'
             ResponseName: 'Y'
               ClassNames: {'setosa' 'versicolor' 'virginica'}
          NumObservations: 150
                    KFold: 5
           ScoreTransform: 'none'
 load fisheriris
 x = meas;
 y = species;

Create a KNN classifier model

 obj = fitcknn (x, y, 'NumNeighbors', 5, 'Standardize', 1);

Create the ClassificationPartitionedModel object

 cvModel = crossval (obj);

Predict the class labels for the observations not used for training

 [label, score, cost] = kfoldPredict (cvModel);
 fprintf ("Cross-validated accuracy = %1.2f%% (%d/%d)\n", ...
          sum (strcmp (label, y)) / numel (y) *100, ...
          sum (strcmp (label, y)), numel (y))
Cross-validated accuracy = 93.33% (140/150)