Statistics: ClassificationSVM

Class Definition: `ClassificationSVM`

statistics: ClassificationSVM

Support Vector Machine classification

The ClassificationSVM class implements a Support Vector Machine classifier object for one-class or two-class problems, which can predict responses for new data using the predict method.

Support Vector Machine classification is a supervised learning method used for classification tasks. It works by finding the optimal hyperplane that separates classes in the feature space with the maximum margin. For non-linearly separable data, it uses kernel functions to map data to a higher-dimensional space where separation is possible.

Create a ClassificationSVM object by using the fitcsvm function or the class constructor.

See also: fitcsvm

Source Code: ClassificationSVM

Properties

A numeric matrix containing the unstandardized predictor data. Each column of X represents one predictor (variable), and each row represents one observation. This property is read-only.

Specified as a logical or numeric column vector, or as a character array or a cell array of character vectors with the same number of rows as the predictor data. Each row in Y is the observed class label for the corresponding row in X. This property is read-only.

A positive integer value specifying the number of observations in the training dataset used for training the ClassificationSVM model. This property is read-only.

A logical column vector with the same length as the observations in the original predictor data X specifying which rows have been used for fitting the ClassificationSVM model. This property is read-only.

A positive integer value specifying the number of predictors in the training dataset used for training the ClassificationSVM model. This property is read-only.

A cell array of character vectors specifying the names of the predictor variables. The names are in the order in which the appear in the training dataset. This property is read-only.

A character vector specifying the name of the response variable Y. This property is read-only.

An array of unique values of the response variable Y, which has the same data types as the data in Y. This property is read-only. ClassNames can have any of the following datatypes:

Cell array of character vectors
Character array
Logical vector
Numeric vector

Specified as a function handle for transforming the classification scores. Add or change the ScoreTransform property using dot notation as in:

obj.ScoreTransform = 'function_name'
obj.ScoreTransform = @function_handle

When specified as a character vector, it can be any of the following built-in functions. Nevertherless, the ScoreTransform property always stores their function handle equivalent.

`Value`		`Description`
`"doublelogit"`		$1 ./ (1 + e×p .^ (-2××))$
`"invlogit"`		$log (× ./ (1 -×))$
`"ismax"`		Sets the score for the class with the largest score to 1, and for all other classes to 0
`"logit"`		$1 ./ (1 + e×p .^ (-×))$
`"none"`		$×$ (no transformation)
`"identity"`		$×$ (no transformation)
`"sign"`		$-1 for× < 0, 0 for× = 0, 1 for× > 0$
`"symmetric"`		$2×× + 1$
`"symmetricismax"`		Sets the score for the class with the largest score to 1, and for all other classes to -1
`"symmetriclogit"`		$2 ./ (1 + e×p .^ (-×)) - 1$

A boolean flag indicating whether the data in X have been standardized prior to training. This property is read-only.

A numeric vector of the same length as the columns in X containing the standard deviations of predictor variables. If the predictor variables have not been standardized, then Sigma is empty. This property is read-only.

A numeric vector of the same length as the columns in X containing the means of predictor variables. If the predictor variables have not been standardized, then Mu is empty. This property is read-only.

A structure containing the parameters used to train the SVM model with the following fields: SVMtype, BoxConstraint, CacheSize, KernelScale, KernelOffset, KernelFunction, PolynomialOrder, Nu, Tolerance, and Shrinking. This property is read-only.

A structure containing the trained model in 'libsvm' format. This property is read-only.

The coefficients of the trained SVM classifier specified as an $s×1$ numeric vector, where $s$ is the number of support vectors equal to sum (obj.IsSupportVector). If the SVM classifier was trained with a kernel function other than 'linear', then Alpha is empty. This property is read-only.

The linear predictor coefficients specified as an $s×1$ numeric vector, where $s$ is the number of support vectors equal to sum (obj.IsSupportVector). If the SVM classifier was trained with a 'linear' kernel function, then Beta is empty. This property is read-only.

The bias term specified as a scalar. This property is read-only.

An $N×1$ logical vector that flags whether a corresponding observation in the predictor data matrix is a Support Vector. $N$ is the number of observations in the training data. This property is read-only.

The support vector class labels specified as an $s×1$ numeric vector, where $s$ is the number of support vectors equal to sum (obj.IsSupportVector). A value of +1 in SupportVectorLabels indicates that the corresponding support vector belongs to the positive class (ClassNames{2}). A value of -1 indicates that the corresponding support vector belongs to the negative class (ClassNames{1}). This property is read-only.

The support vectors of the trained SVM classifier specified an $s×p$ numeric matrix, where $s$ is the number of support vectors equal to sum (obj.IsSupportVector), and $p$ is the number of predictor variables in the predictor data. This property is read-only.

Methods

statistics: obj = ClassificationSVM (X, Y)
statistics: obj = ClassificationSVM (…, name, value)

obj = ClassificationSVM (X, Y) returns a ClassificationSVM object, with X as the predictor data and Y containing the class labels of observations in X.

X must be a $N×P$ numeric matrix of input data where rows correspond to observations and columns correspond to features or variables. X will be used to train the SVM model.
Y is $N×1$ matrix or cell matrix containing the class labels of corresponding predictor data in X. Y can be either numeric, logical, or cell array of character vectors. It must have same numbers of rows as X.

obj = ClassificationSVM (…, name, value) returns a ClassificationSVM object with parameters specified by the following name, value paired input arguments:

`Name`		`Value`
`'PredictorNames'`		A cell array of character vectors specifying the names of the predictors. The length of this array must match the number of columns in `X`.
`'ResponseName'`		A character vector specifying the name of the response variable.
`'ClassNames'`		Names of the classes in the class labels, `Y`, used for fitting the SVM model. `ClassNames` are of the same type as the class labels in `Y`.
`'ScoreTransform'`		A user-defined function handle or a character vector specifying one of the following builtin functions specifying the transformation applied to predicted classification scores. Supported values include `'doublelogit'`, `'invlogit'`, `'ismax'`, `'logit'`, `'none'`, `'identity'`, `'sign'`, `'symmetric'`, `'symmetricismax'`, and `'symmetriclogit'`.
`'Standardize'`		A logical scalar specifying whether to standardize the predictor variables. Default is `false`.
`'SVMtype'`		A character vector specifying the type of SVM to use. Supported values are `'c_svc'` (C-support vector classification), `'nu_svc'` (nu-support vector classification), and `'one_class_svm'` (one-class SVM).
`'KernelFunction'`		A character vector specifying the kernel function to use. Supported values are `'linear'`, `'rbf'` or `'gaussian'`, `'polynomial'`, and `'sigmoid'`.
`'PolynomialOrder'`		A positive integer specifying the order of the polynomial kernel function. Default is 3.
`'KernelScale'`		A positive scalar specifying the kernel scale parameter. Default is 1.
`'KernelOffset'`		A non-negative scalar specifying the kernel offset parameter. Default is 0.
`'BoxConstraint'`		A positive scalar specifying the box constraint parameter. Default is 1.
`'Nu'`		A positive scalar in the range (0,1] specifying the nu parameter for nu-SVM and one-class SVM. Default is 0.5.
`'CacheSize'`		A positive scalar specifying the cache size in MB. Default is 1000.
`'Tolerance'`		A positive scalar specifying the tolerance of termination criterion. Default is 1e-6.
`'Shrinking'`		Either 0 or 1 specifying whether to use the shrinking heuristics. Default is 1.
`'OutlierFraction'`		A positive scalar in the range [0,1) specifying the fraction of outliers for one-class SVM.

See also: fitcsvm

ClassificationSVM: label = predict (obj, XC)
ClassificationSVM: [label, score] = predict (obj, XC)

label = predict (obj, XC) returns the vector of labels predicted for the corresponding instances in XC, using the predictor data in obj.X and corresponding labels, obj.Y, stored in the ClassificationSVM model, obj. For one-class SVM model, +1 or -1 is returned.

obj must be a ClassificationSVM class object.
XC must be an $M×P$ numeric matrix with the same number of features $P$ as the corresponding predictors of the SVM model in obj.

[label, score] = predict (obj, XC) also returns score, which contains the decision values for each each prediction. Alternatively, score can contain the posterior probabilities if the ScoreTransform has been previously set using the fitPosterior method.

See also: ClassificationSVM, fitcsvm, ClassificationSVM.fitPosterior

ClassificationSVM: label = resubPredict (obj)
ClassificationSVM: [label, score] = resubPredict (obj)

label = resubPredict (obj) returns the vector of labels predicted for the corresponding instances in the training data, using the predictor data in obj.X and corresponding labels, obj.Y, stored in the Support Vector Machine classification model, obj. For one-class model, +1 or -1 is returned.

obj must be a ClassificationSVM class object.

[label, scores] = resubPredict (obj also returns scores, which contains the decision values for each each prediction. Alternatively, scores can contain the posterior probabilities if the ScoreTransform has been previously set using the fitPosterior method.

See also: fitcsvm, ClassificationSVM.fitPosterior

ClassificationSVM: m = margin (obj, X, Y)

m = margin (obj, X, Y) returns the classification margins for obj with data X and classification Y. m is a numeric vector of length size (X,1).

obj is a ClassificationSVM object trained on X and Y.
X must be a $N×P$ numeric matrix of input data where rows correspond to observations and columns correspond to features or variables.
Y is $N×1$ matrix or cell matrix containing the class labels of corresponding predictor data in X. Y must have same numbers of Rows as X.

The classification margin for each observation is the difference between the classification score for the true class and the maximal classification score for the false classes.

See also: fitcsvm, ClassificationSVM

ClassificationSVM: L = loss (obj, X, Y)
ClassificationSVM: L = loss (…, name, value)

L = loss (obj, X, Y) computes the loss, L, using the default loss function 'classiferror'.

obj is a ClassificationSVM object trained on X and Y.
X must be a $N×P$ numeric matrix of input data where rows correspond to observations and columns correspond to features or variables.
Y is $N×1$ matrix or cell matrix containing the class labels of corresponding predictor data in X. Y must have same numbers of Rows as X.

L = loss (…, name, value) allows additional options specified by name-value pairs:

`Name`		`Value`
`"LossFun"`		Specifies the loss function to use. Can be a function handle with four input arguments (C, S, W, Cost) which returns a scalar value or one of: ’binodeviance’, ’classifcost’, ’classiferror’, ’exponential’, ’hinge’, ’logit’,’mincost’, ’quadratic’. `C` is a logical matrix of size $N×K$ , where $N$ is the number of observations and $K$ is the number of classes. The element `C(i,j)` is true if the class label of the i-th observation is equal to the j-th class. `S` is a numeric matrix of size $N×K$ , where each element represents the classification score for the corresponding class. `W` is a numeric vector of length $N$ , representing the observation weights. `Cost` is a $K×K$ matrix representing the misclassification costs.
`"Weights"`		Specifies observation weights, must be a numeric vector of length equal to the number of rows in X. Default is `ones (size (X, 1))`. loss normalizes the weights so that observation weights in each class sum to the prior probability of that class. When you supply Weights, loss computes the weighted classification loss.

See also: ClassificationSVM

ClassificationSVM: L = resubLoss (obj)
ClassificationSVM: L = resubLoss (…, name, value)

L = resubLoss (obj) computes the resubstitution loss, L, using the default loss function 'classiferror'.

obj is a ClassificationSVM object trained on X and Y.

L = resubLoss (…, name, value) allows additional options specified by name-value pairs:

`Name`		`Value`
`"LossFun"`		Specifies the loss function to use. Can be a function handle with four input arguments (C, S, W, Cost) which returns a scalar value or one of: ’binodeviance’, ’classifcost’, ’classiferror’, ’exponential’, ’hinge’, ’logit’,’mincost’, ’quadratic’. `C` is a logical matrix of size $N×K$ , where $N$ is the number of observations and $K$ is the number of classes. The element `C(i,j)` is true if the class label of the i-th observation is equal to the j-th class. `S` is a numeric matrix of size $N×K$ , where each element represents the classification score for the corresponding class. `W` is a numeric vector of length $N$ , representing the observation weights. `Cost` is a $K×K$ matrix representing the misclassification costs.
`"Weights"`		Specifies observation weights, must be a numeric vector of length equal to the number of rows in X. Default is `ones (size (X, 1))`. loss normalizes the weights so that observation weights in each class sum to the prior probability of that class. When you supply Weights, loss computes the weighted classification loss.

See also: ClassificationSVM

ClassificationSVM: CVMdl = crossval (obj)
ClassificationSVM: CVMdl = crossval (…, name, value)

CVMdl = crossval (obj) returns a cross-validated model object, CVMdl, from a trained model, obj, using 10-fold cross-validation by default.

CVMdl = crossval (obj, name, value) specifies additional name-value pair arguments to customize the cross-validation process.

`Name`		`Value`
`"KFold"`		Specify the number of folds to use in k-fold cross-validation. `"KFold", k`, where `k` is an integer greater than 1.
`"Holdout"`		Specify the fraction of the data to hold out for testing. `"Holdout", p`, where `p` is a scalar in the range $(0,1)$ .
`"Leaveout"`		Specify whether to perform leave-one-out cross-validation. `"Leaveout", Value`, where `Value` is ’on’ or ’off’.
`"CVPartition"`		Specify a `cvpartition` object used for cross-validation. `"CVPartition", cv`, where `isa (cv, "cvpartition")` = 1.

ClassificationSVM: CVMdl = compact (obj)

CVMdl = compact (obj) creates a compact version of the ClassificationSVM object, obj.

ClassificationSVM: savemodel (obj, filename)

savemodel (obj, filename) saves each property of a ClassificationSVM object into an Octave binary file, the name of which is specified in filename, along with an extra variable, which defines the type classification object these variables constitute. Use loadmodel in order to load a classification object into Octave’s workspace.

See also: loadmodel, fitcsvm, ClassificationSVM

Examples

 load fisheriris
 rng(1);  ## For reproducibility

 ## Select indices of the non-setosa species
 inds = !strcmp(species, 'setosa');

  ## Select features and labels for non-setosa species
 X = meas(inds, 3:4);
 Y = grp2idx(species(inds));

 ##  Convert labels to +1 and -1
 unique_classes = unique(Y);
 Y(Y == unique_classes(1)) = -1;
 Y(Y == unique_classes(2)) = 1;

 ## Partition data for training and testing
 cv = cvpartition(Y, 'HoldOut', 0.15);
 X_train = X(training(cv), :);
 Y_train = Y(training(cv));
 X_test = X(test(cv), :);
 Y_test = Y(test(cv));

 ## Train the SVM model
 CVSVMModel = fitcsvm(X_train, Y_train);

 ## Calculate margins
 m = margin(CVSVMModel, X_test, Y_test);
 disp(m);

 load fisheriris
 rng(1);  ## For reproducibility

  ## Select indices of the non-setosa species
 inds = !strcmp(species, 'setosa');

  ## Select features and labels for non-setosa species
 X = meas(inds, 3:4);
 Y = grp2idx(species(inds));

 ##  Convert labels to +1 and -1
 unique_classes = unique(Y);
 Y(Y == unique_classes(1)) = -1;
 Y(Y == unique_classes(2)) = 1;

 ## Randomly partition the data into training and testing sets
 cv = cvpartition(Y, 'HoldOut', 0.3); # 30% data for testing, 60% for training

 X_train = X(training(cv), :);
 Y_train = Y(training(cv));

 X_test = X(test(cv), :);
 Y_test = Y(test(cv));

 ## Train the SVM model
 SVMModel = fitcsvm(X_train, Y_train);

 ## Calculate loss

 L = loss(SVMModel,X_test,Y_test,'LossFun','binodeviance')
 L = loss(SVMModel,X_test,Y_test,'LossFun','classiferror')
 L = loss(SVMModel,X_test,Y_test,'LossFun','exponential')
 L = loss(SVMModel,X_test,Y_test,'LossFun','hinge')
 L = loss(SVMModel,X_test,Y_test,'LossFun','logit')
 L = loss(SVMModel,X_test,Y_test,'LossFun','quadratic')

L = 0.1500
L = 0.066667
L = 0.2811
L = 0.1667
L = 0.2089
L = 3.6107

Categories &

Functions List

Clustering

Clustering

Classification Classes

Classification Classes

Clustering Classes

Clustering Classes

Regression Classes

Regression Classes

Data Manipulation

Data Manipulation

Descriptive Statistics

Descriptive Statistics

Distribution Classes

Distribution Classes

Distribution Fitting

Distribution Fitting

Distribution Functions

Distribution Functions

Distribution Statistics

Distribution Statistics

Distribution Wrappers

Distribution Wrappers

Experimental Design

Experimental Design

Machine Learning

Machine Learning

Model Fitting

Model Fitting

Hypothesis Testing

Hypothesis Testing

I/O

I/O

Plotting

Plotting

Regression

Regression

Transforms

Transforms

Class Definition: ClassificationSVM

statistics: ClassificationSVM

Properties

X

Y

NumObservations

RowsUsed

NumPredictors

PredictorNames

ResponseName

ClassNames

ScoreTransform

Standardize

Sigma

Mu

ModelParameters

Model

Alpha

Beta

Bias

IsSupportVector

SupportVectorLabels

SupportVectors

Methods

ClassificationSVM

statistics: obj = ClassificationSVM (X, Y)

statistics: obj = ClassificationSVM (…, name, value)

predict

ClassificationSVM: label = predict (obj, XC)

ClassificationSVM: [label, score] = predict (obj, XC)

resubPredict

ClassificationSVM: label = resubPredict (obj)

ClassificationSVM: [label, score] = resubPredict (obj)

margin

ClassificationSVM: m = margin (obj, X, Y)

loss

ClassificationSVM: L = loss (obj, X, Y)

ClassificationSVM: L = loss (…, name, value)

resubLoss

ClassificationSVM: L = resubLoss (obj)

Class Definition: `ClassificationSVM`

`X`

`Y`

`NumObservations`

`RowsUsed`

`NumPredictors`

`PredictorNames`

`ResponseName`

`ClassNames`

`ScoreTransform`

`Standardize`

`Sigma`

`Mu`

`ModelParameters`

`Model`

`Alpha`

`Beta`

`Bias`

`IsSupportVector`

`SupportVectorLabels`

`SupportVectors`

`ClassificationSVM`

`statistics: obj = ClassificationSVM (X, Y)`

`statistics: obj = ClassificationSVM (…, name, value)`

`predict`

`ClassificationSVM: label = predict (obj, XC)`

`ClassificationSVM: [label, score] = predict (obj, XC)`

`resubPredict`

`ClassificationSVM: label = resubPredict (obj)`

`ClassificationSVM: [label, score] = resubPredict (obj)`

`margin`

`ClassificationSVM: m = margin (obj, X, Y)`

`loss`

`ClassificationSVM: L = loss (obj, X, Y)`

`ClassificationSVM: L = loss (…, name, value)`

`resubLoss`

`ClassificationSVM: L = resubLoss (obj)`

`ClassificationSVM: L = resubLoss (…, name, value)`

`crossval`

`ClassificationSVM: CVMdl = crossval (obj)`

`ClassificationSVM: CVMdl = crossval (…, name, value)`

`compact`

`ClassificationSVM: CVMdl = compact (obj)`

`savemodel`

`ClassificationSVM: savemodel (obj, filename)`