Support Vector Machine (SVM)

Description

Support vector machines is a machine learning classifier algorithm used to classify the class membership of an unknown based on training data.

Simple Explanation

SVM showing a linear decision boundary

Fig. 1. A simple support vector machine marks a linear decision boundary (magenta line) between the x class (blue cross) and square class (green square).

During training, SVM learns from the data to draw a decision boundary that separates different classes. To ensure optimal separation of the different classes, the decision boundary created has the maximum distance between it and the training examples of different classes.

Kernels

SVM showing a non-linear decision boundary

Fig. 2. A support vector machine marks a non-linear decision boundary (magenta curve) between the x class (blue cross) and square class (green square) with the help of kernels.

The simplest of SVMs can only draw linear decision boundaries (Figure 1). But fortunately, with the help of kernel functions (add-on to SVM) and some mathematical tricks, the SVM can compute more complex non-linear decision boundaries such as the one shown in Figure 2. See parameters for more details.

Parameters

ParameterDescription
C penalty for misclassification
  • large value of C – SVM will try to classify more training data correctly at the cost of lowering the margin between decision boundary and data points
  • Small value of C – SVM will maximize the margin between decision boundary and data points possibly at the cost of placing some training data on the wrong side of decision boundary
Kernel MLGenius currently supports:
  • linear (no kernel)
  • polynomial
  • radial basis function
  • sigmoid
Degree (used in polynomial kernel)
higher degrees will generate more curvature in the decision boundary
Gamma identifies the radius of influence a single training example has
  • large value of gamma – individual training example has little influence outside of its immediate vicinity
  • small value of gamma – individual training example has large radius of influence
SVM type while SVM has been used to do more than just classification, MLGenius currently only supports the classifier version of SVM


MLGenius will help you determine the best parameters to use for your dataset

Pros and Cons

ProsCons
  • powerful classifier can learn from both linear and non-linear separable data
  • performs well even with small datasets and lots of features
  • may require some computing time to optimize the parameters
  • sensitive to outliers in small datasets