Statistics: ClassificationSVM

Class Definition: `ClassificationSVM`

statistics: obj = ClassificationSVM (X, Y)
statistics: obj = ClassificationSVM (…, name, value)

Create a ClassificationSVM class object containing a Support Vector Machine classification model for one-class or two-class problems.

obj = ClassificationSVM (X, Y) returns a ClassificationSVM object, with X as the predictor data and Y containing the class labels of observations in X.

X must be a $N×P$ numeric matrix of input data where rows correspond to observations and columns correspond to features or variables. X will be used to train the SVM model.
Y is $N×1$ matrix or cell matrix containing the class labels of corresponding predictor data in X. Y can be either numeric, logical, or cell array of character vectors. It must have same numbers of rows as X.

obj = ClassificationSVM (…, name, value) returns a ClassificationSVM object with parameters specified by Name-Value pair arguments. Type help fitcsvm for more info.

A ClassificationSVM object, obj, stores the labelled training data and various parameters for the Support Vector machine classification model, which can be accessed in the following fields:

`Field`		`Description`
`X`		Unstandardized predictor data, specified as a numeric matrix. Each column of `X` represents one predictor (variable), and each row represents one observation.
`Y`		Class labels, specified as a logical or numeric vector, or cell array of character vectors. Each value in `Y` is the observed class label for the corresponding row in `X`.
`NumObservations`		Number of observations used in training the ClassificationSVM model, specified as a positive integer scalar. This number can be less than the number of rows in the training data because rows containing `NaN` values are not part of the fit.
`RowsUsed`		Rows of the original training data used in fitting the ClassificationSVM model, specified as a numerical vector. If you want to use this vector for indexing the training data in `X`, you have to convert it to a logical vector, i.e `X = obj.X(logical (obj.RowsUsed), :);`
`Standardize`		A boolean flag indicating whether the data in `X` have been standardized prior to training.
`Sigma`		Predictor standard deviations, specified as a numeric vector of the same length as the columns in `X`. If the predictor variables have not been standardized, then `"obj.Sigma"` is empty.
`Mu`		Predictor means, specified as a numeric vector of the same length as the columns in `X`. If the predictor variables have not been standardized, then `"obj.Mu"` is empty.
`NumPredictors`		The number of predictors (variables) in `X`.
`PredictorNames`		Predictor variable names, specified as a cell array of character vectors. The variable names are in the same order in which they appear in the training data `X`.
`ResponseName`		Response variable name, specified as a character vector.
`ClassNames`		Names of the classes in the class labels, `Y`, used for fitting the SVM model. `ClassNames` are of the same type as the class labels in `Y`.
`Prior`		Prior probabilities for each class, specified as a numeric vector. The order of the elements in `Prior` corresponds to the order of the classes in `ClassNames`.
`Cost`		Cost of the misclassification of a point, specified as a square matrix. `Cost(i,j)` is the cost of classifying a point into class `j` if its true class is `i` (that is, the rows correspond to the true class and the columns correspond to the predicted class). The order of the rows and columns in `Cost` corresponds to the order of the classes in `ClassNames`. The number of rows and columns in `Cost` is the number of unique classes in the response. By default, `Cost(i,j) = 1` if `i != j`, and `Cost(i,j) = 0` if `i = j`. In other words, the cost is 0 for correct classification and 1 for incorrect classification.
`ScoreTransform`		A function_handle which is used for transforming the SVM prediction score into a posterior probability. By default, it is `'none'`, in which case the `predict` and `resubPredict` methods return the prediction scores. Use the `fitPosterior` method to compute the appropriate `ScoreTransform`, in which case the `predict` and `resubPredict` methods return the posterior probabilities.
`ModelParameters`		A structure containing the parameters used to train the SVM model with the following fields: `SVMtype`, `BoxConstraint`, `CacheSize`, `KernelScale`, `KernelOffset`, `KernelFunction`, `PolynomialOrder`, `Nu`, `Tolerance`, and `Shrinking`. Type `help fitcsvm` for more info on their usage and default values.
`Model`		A structure containing the trained model in `'libsvm'` format.
`Alpha`		The coefficients of the trained SVM classifier specified as an $s×1$ numeric vector, where $s$ is the number of support vectors equal to `sum (obj.IsSupportVector)`. If the SVM classifier was trained with a `'linear'` kernel function, then `obj.Alpha` is left empty.
`Beta`		The linear predictor coefficients specified as an $s×1$ numeric vector, where $s$ is the number of support vectors equal to `sum (obj.IsSupportVector)`. If the SVM classifier was trained with a kernel function other than `'linear'`, then `obj.Beta` is left empty.
`Bias`		The bias term specified as a scalar.
`IsSupportVector`		Support vector indicator, specified as an $N×1$ logical vector that flags whether a corresponding observation in the predictor data matrix is a Support Vector. $N$ is the number of observations in the training data (see `NumObservations`).
`SupportVectorLabels`		The support vector class labels specified as an $s×1$ numeric vector, where $s$ is the number of support vectors equal to `sum (obj.IsSupportVector)`. A value of +1 in `SupportVectorLabels` indicates that the corresponding support vector belongs to the positive class `(ClassNames{2})`. A value of -1 indicates that the corresponding support vector belongs to the negative class `(ClassNames{1})`.
`SupportVectors`		The support vectors of the trained SVM classifier specified an $s×p$ numeric matrix, where $s$ is the number of support vectors equal to `sum (obj.IsSupportVector)`, and $p$ is the number of predictor variables in the predictor data.

See also: fitcsvm, svmtrain, svmpredict

Source Code: ClassificationSVM

Method: `compact`

ClassificationSVM: CVMdl = compact (obj)

Create a CompactClassificationSVM object.

CVMdl = compact (obj) creates a compact version of the ClassificationSVM object, obj.

Method: `crossval`

ClassificationSVM: CVMdl = crossval (obj)
ClassificationSVM: CVMdl = crossval (…, Name, Value)

Cross Validate a Support Vector Machine classification object.

CVMdl = crossval (obj) returns a cross-validated model object, CVMdl, from a trained model, obj, using 10-fold cross-validation by default.

CVMdl = crossval (obj, name, value) specifies additional name-value pair arguments to customize the cross-validation process.

`Name`		`Value`
`"KFold"`		Specify the number of folds to use in k-fold cross-validation. `"KFold", k`, where `k` is an integer greater than 1.
`"Holdout"`		Specify the fraction of the data to hold out for testing. `"Holdout", p`, where `p` is a scalar in the range $(0,1)$ .
`"Leaveout"`		Specify whether to perform leave-one-out cross-validation. `"Leaveout", Value`, where `Value` is ’on’ or ’off’.
`"CVPartition"`		Specify a `cvpartition` object used for cross-validation. `"CVPartition", cv`, where `isa (cv, "cvpartition")` = 1.

See also: fitcsvm, ClassificationSVM, cvpartition, ClassificationPartitionedModel

Method: `fitPosterior`

ClassificationSVM: Mdl = fitPosterior (obj)
ClassificationSVM: CVMdl = fitPosterior (obj, name, value)

Fit posterior probabilities to a Support Vector Machine model.

Mdl = fitPosterior (obj) returns the ClassificationSVM object, Mdl, from an already trained SVM model, obj, after fitting a posterior probabilities ScoreTransform.

CVMdl = fitPosterior (obj, name, value) returns the ClassificationPartitionedModel, CVMdl, from an already trained SVM model, obj, after fitting a posterior probabilities ScoreTransform. Use the additional name-value pair arguments to customize the cross-validation process.

`Name`		`Value`
`"KFold"`		Specify the number of folds to use in k-fold cross-validation. `'kfold', k` where `k` is an integer greater than 1.
`"Holdout"`		Specify the fraction of the data to hold out for testing. `'holdout', p` where `p` is a scalar in the range (0,1).
`"CVPartition"`		Specify the fraction of the data to hold out for testing. `'holdout', p` where `p` is a scalar in the range (0,1).
`"Leaveout"`		Specify whether to perform leave-one-out cross-validation. `'leaveout', Value` where `Value` is ’on’ or ’off’.

See also: cvpartition, fitcsvm, ClassificationSVM

Method: `loss`

ClassificationSVM: L = loss (obj, X, Y)
ClassificationSVM: L = loss (…, name, value)

Determine the classification error for a Support Vector Machine classifier.

L = loss (obj, X, Y) returns the predictive accuracy of support vector machine (SVM) classification models. Comparing the same type of loss across multiple models allows you to identify which model is more accurate, with a lower loss indicating superior predictive performance. It supports only binary classifier models.

obj must be a binary class ClassificationSVM object.
X must be an $M×P$ numeric matrix with the same number of features $P$ as the corresponding predictors of the SVM model in obj.
Y must be $M×1$ numeric vector containing the class labels corresponding to the predictor data in X. Y must have same number of rows as X.

L = loss (…, Name, Value) returns the aforementioned results with additional properties specified by Name-Value pair arguments listed below.

`Name`		`Value`
`"LossFun"`		Loss function, specified as a built-in loss function name. It accepts the following options: (Default is ’classiferror’) ’binodeviance’: Binomial deviance: The binomial deviance loss function is used to evaluate the performance of a binary classifier. It is calculated as: $L = \sum_j=1^n w_j \log \1 + \e×p [-2m_j]\$ ’classiferror’: Misclassification rate in decimal The classification error measures the fraction of misclassified instances out of the total instances. It is calculated as: $L = \frac1n \sum_j=1^n \mathbbI(m_j \leq 0)$ ’exponential’: Exponential loss: The exponential loss function is used to penalize misclassified instances exponentially. It is calculated as: $L = \sum_j=1^n w_j \e×p [-m_j]$ ’hinge’: Hinge loss: The hinge loss function is often used for maximum-margin classification, particularly for support vector machines. It is calculated as: $L = \sum_j=1^n w_j \ma× (0, 1 - m_j)$ ’logit’: Logistic loss: The logistic loss function, also known as log loss, measures the performance of a classification model where the prediction is a probability value. It is calculated as: $L = \sum_j=1^n w_j \log \1 + \e×p [-m_j]\$ ’quadratic’: Quadratic loss: The quadratic loss function penalizes the square of the margin. It is calculated as: $L = \sum_j=1^n w_j (1 - m_j)^2$
`"Weights"`		Specified as a numeric vector which weighs each observation (row) in X. The size of Weights must be equal to the number of rows in X. The default value is: ones(size(X,1),1)

See also: fitcsvm, ClassificationSVM

Method: `margin`

ClassificationSVM: m = margin (obj, X, Y)

Determine the classification margins for a Support Vector Machine classification object.

m = margin (obj, X, Y) returns the classification margins for the trained support vector machine (SVM) classifier obj using the sample data in X and the class labels in Y. It supports only binary classifier models. The classification margin is commonly defined as m = yf(x), where f(x) is the classification score and y is the true class label corresponding to x. A greater margin indicates a better model.

obj must be a binary class ClassificationSVM object.
X must be an $M×P$ numeric matrix with the same number of features $P$ as the corresponding predictors of the SVM model in obj.
Y must be $M×1$ numeric vector containing the class labels corresponding to the predictor data in X. Y must have same number of rows as X.

See also: fitcsvm, ClassificationSVM

Method: `predict`

ClassificationSVM: labels = predict (obj, XC)
ClassificationSVM: [labels, scores] = predict (obj, XC)

Classify new data points into categories using the Support Vector Machine classification object.

labels = predict (obj, XC) returns the vector of labels predicted for the corresponding instances in XC, using the trained Support Vector Machine classification model, obj. For one-class SVM model, +1 or -1 is returned.

obj must be a ClassificationSVM class object.
XC must be an $M×P$ numeric matrix with the same number of predictors $P$ as the corresponding predictors of the SVM model in obj.

[labels, scores] = predict (obj, XC also returns scores, which contains the desicion values for each each prediction. Alternatively, scores can contain the posterior probabilities if the ScoreTransform has been previously set using the fitPosterior method.

See also: fitcsvm, ClassificationSVM.fitPosterior

Method: `resubLoss`

ClassificationSVM: L = resubLoss (obj)
ClassificationSVM: L = resubLoss (…, name, value)

Compute the resubstitution classification loss for the trained Support Vector Machine classification object.

L = resubLoss (obj) returns the classification loss by resubstitution (L), or the in-sample classification loss, for the trained classification model obj using the training data stored in obj.X and the corresponding class labels stored in obj.Y.

obj must be a binary class ClassificationSVM object.

l = resubLoss (…, Name, Value) returns the aforementioned results with additional properties specified by Name-Value pair arguments listed below.

`Name`		`Value`
`"LossFun"`		Loss function, specified as a built-in loss function name. It accepts the following options: (Default is ’classiferror’) ’binodeviance’: Binomial deviance: The binomial deviance loss function is used to evaluate the performance of a binary classifier. It is calculated as: $L = \sum_j=1^n w_j \log \1 + \e×p [-2m_j]\$ ’classiferror’: Misclassification rate in decimal The classification error measures the fraction of misclassified instances out of the total instances. It is calculated as: $L = \frac1n \sum_j=1^n \mathbbI(m_j \leq 0)$ ’exponential’: Exponential loss: The exponential loss function is used to penalize misclassified instances exponentially. It is calculated as: $L = \sum_j=1^n w_j \e×p [-m_j]$ ’hinge’: Hinge loss: The hinge loss function is often used for maximum-margin classification, particularly for support vector machines. It is calculated as: $L = \sum_j=1^n w_j \ma× (0, 1 - m_j)$ ’logit’: Logistic loss: The logistic loss function, also known as log loss, measures the performance of a classification model where the prediction is a probability value. It is calculated as: $L = \sum_j=1^n w_j \log \1 + \e×p [-m_j]\$ ’quadratic’: Quadratic loss: The quadratic loss function penalizes the square of the margin. It is calculated as: $L = \sum_j=1^n w_j (1 - m_j)^2$
`"Weights"`		Specified as a numeric vector which weighs each observation (row) in X. The size of Weights must be equal to the number of rows in X. The default value is: ones(size(X,1),1)

See also: fitcsvm, ClassificationSVM

Method: `resubPredict`

ClassificationSVM: labels = resubPredict (obj)
ClassificationSVM: [labels, score] = resubPredict (obj)

Classify the training data using the trained Support Vector Machine classification object.

labels = resubPredict (obj) returns the vector of labels predicted for the corresponding instances in the training data, using the predictor data in obj.X and corresponding labels, obj.Y, stored in the Support Vector Machine classification model, obj. For one-class model, +1 or -1 is returned.

obj must be a ClassificationSVM class object.

[labels, scores] = resubPredict (obj also returns scores, which contains the desicion values for each each prediction. Alternatively, scores can contain the posterior probabilities if the ScoreTransform has been previously set using the fitPosterior method.

See also: fitcsvm, ClassificationSVM.fitPosterior

Method: `savemodel`

ClassificationSVM: savemodel (obj, filename)

Save a ClassificationSVM object.

savemodel (obj, filename) saves a ClassificationSVM object into a file defined by filename.

See also: loadmodel, fitcsvm, ClassificationSVM

Example: 1


 ## Create a Support Vector Machine classifier and determine margin for test
 ## data.
 load fisheriris
 rng(1);  ## For reproducibility

 ## Select indices of the non-setosa species
 inds = !strcmp(species, 'setosa');

  ## Select features and labels for non-setosa species
 X = meas(inds, 3:4);
 Y = grp2idx(species(inds));

 ##  Convert labels to +1 and -1
 unique_classes = unique(Y);
 Y(Y == unique_classes(1)) = -1;
 Y(Y == unique_classes(2)) = 1;

 ## Partition data for training and testing
 cv = cvpartition(Y, 'HoldOut', 0.15);
 X_train = X(training(cv), :);
 Y_train = Y(training(cv));
 X_test = X(test(cv), :);
 Y_test = Y(test(cv));

 ## Train the SVM model
 CVSVMModel = fitcsvm(X_train, Y_train);

 ## Calculate margins
 m = margin(CVSVMModel, X_test, Y_test);
 disp(m);

   1.1093
   7.3147
   3.3280
   2.4400
   5.1013
   4.2187
   5.9893
   6.4293
   6.4320
   9.5280
   0.6667
   5.5360
   0.2240
   4.2160
   1.9973

Example: 2


 ## Create a Support Vector Machine classifier and determine loss for test
 ## data.
 load fisheriris
 rng(1);  ## For reproducibility

  ## Select indices of the non-setosa species
 inds = !strcmp(species, 'setosa');

  ## Select features and labels for non-setosa species
 X = meas(inds, 3:4);
 Y = grp2idx(species(inds));

 ##  Convert labels to +1 and -1
 unique_classes = unique(Y);
 Y(Y == unique_classes(1)) = -1;
 Y(Y == unique_classes(2)) = 1;

 ## Randomly partition the data into training and testing sets
 cv = cvpartition(Y, 'HoldOut', 0.3); # 30% data for testing, 60% for training

 X_train = X(training(cv), :);
 Y_train = Y(training(cv));

 X_test = X(test(cv), :);
 Y_test = Y(test(cv));

 ## Train the SVM model
 SVMModel = fitcsvm(X_train, Y_train);

 ## Calculate loss

 L = loss(SVMModel,X_test,Y_test,'LossFun','binodeviance')
 L = loss(SVMModel,X_test,Y_test,'LossFun','classiferror')
 L = loss(SVMModel,X_test,Y_test,'LossFun','exponential')
 L = loss(SVMModel,X_test,Y_test,'LossFun','hinge')
 L = loss(SVMModel,X_test,Y_test,'LossFun','logit')
 L = loss(SVMModel,X_test,Y_test,'LossFun','quadratic')

L = 0.1500
L = 0.066667
L = 0.2811
L = 0.1667
L = 0.2089
L = 3.6107

Categories &

Functions List

CVpartition

CVpartition

Classification Classes

Classification Classes

Clustering

Clustering

Clustering Classes

Clustering Classes

Data Manipulation

Data Manipulation

Descriptive Statistics

Descriptive Statistics

Distribution Classes

Distribution Classes

Distribution Fitting

Distribution Fitting

Distribution Functions

Distribution Functions

Distribution Statistics

Distribution Statistics

Distribution Wrappers

Distribution Wrappers

Experimental Design

Experimental Design

Hypothesis Testing

Hypothesis Testing

I/O

I/O

Machine Learning

Machine Learning

Model Fitting

Model Fitting

Plotting

Plotting

Regression

Regression

Regression Classes

Regression Classes

Transforms

Transforms

Class Definition: ClassificationSVM

Method: compact

Method: crossval

Method: fitPosterior

Method: loss

Method: margin

Method: predict

Method: resubLoss

Method: resubPredict

Method: savemodel

Example: 1

Example: 2

Class Definition: `ClassificationSVM`

Method: `compact`

Method: `crossval`

Method: `fitPosterior`

Method: `loss`

Method: `margin`

Method: `predict`

Method: `resubLoss`

Method: `resubPredict`

Method: `savemodel`