Support Vector Machine: An Interesting Overview In 2021

Ajay Ohri


Support Vector Machine (SVM) is becoming a more preferable option by many customers as the output produced by it has a higher amount of accuracy with less computation power. It is used mainly for classification tasks and regression problems. However, it is mostly used in classification objectives. An ideal SVM is the one where the data can be separated linearly, and there is a distinctive global minimum value. 

In this article let us look at:

  1. What is a Support Vector Machine?
  2. Difference between Logistic Regression and Support Vector Machine
  3. Hyperplanes and Support Vectors
  4. Large Margin Intuition
  5. Cost Function and Gradient Updates

1. What is a Support Vector Machine?

Its purpose is to look out for a hyperplane in N-dimensional space (N – being the number of available features) that accurately assorts the data points and hyperplanes are decision boundaries that facilitate classifying the data points. In the case of three dimensions, a plane with just two dimensions divides the third space into two parts, and thus it behaves as a hyperplane.

To split the two different categories of data, there are many possible options of hyperbplanes that can be chosen from. The target shall always be to seek out a plane that has the maximum margin, i.e., the maximum distance between data points of each category, and this is what completes the SVM working. Support Vectors (SV) are those data points that are closer to the hyperplane and influence the position and orientation of the hyperplane.

Advantages of SVM includes:

  • It works better when there is a clear margin of separation between classes.
  • It is more effective in high-dimensional spaces.
  • It is effective in cases where the number of dimensions is greater than the number of samples.
  • It is memory efficient.

Disadvantages of SVM includes:

  • Not suitable for large sets of data.
  • Not efficient in performance when target classes are overlapping.

2. Difference between logistic regression and Support Vector Machine

Logistic Regression:

  • It is an algorithm for solving classification problems.
  • Based on a statistical approach.
  • Vulnerable to overfitting.
  • Works with the already identified independent variables.

Support Vector Machine:

  • It is a model used for both classification and regression.
  • Based on the geometrical properties of the data.
  • The risk of overfitting is less.
  • Works well with unstructured and semi-structured data like texts and images.

3. Hyperplanes and support vectors

In the SVM classifier, it is easy to have a linear hyper-plane between two classes. But instead of adding the feature of the hyperplane, the SVM algorithm has a technique called the kernel trick. Kernel trick takes the data you give it and transforms it. In goes, some great features that you think are going to make a great classifier, and the output comes to your desired result. 

To build up an optimal hyperplane, SVM takes up a sequential training algorithm, thus reducing an error function to an acceptable limit. As per this error function, there can be four types of SVM:

  • Classification SVM Type-1 ( which means ‘C-SVM classification’)
  • Classification SVM Type-2 (which means ‘nu-SVM classification’)
  • Regression SVM Type-1 (which means ‘epsilon SVM regression’)
  • Regression SVM Type-2 (which means ‘nu-SVM regression’)

4. Large Margin Intuition

With the help of support vectors, we tend to maximize the margin of the SVM classifier. Deletion of the SV can amend the positions of the hyperplane and help us build our SVM.  A perfect SVM analysis should produce a hyperplane that totally split-up the vectors into two non-overlapping categories.

However, a perfect separation may not be attainable as it may result in a model with such a large number of cases that the model might not be able to classify correctly. In this situation, SVM finds the hyperplane that maximizes the margin and minimizes the misclassification. The SVM algorithm creates a line or a hyper-lapse which separates the data into classes. SVMs are known as large margin classifiers because the minimization of the cost function will result in large margins (i.e., the margin in the centre)

5. Cost Function and Gradient Updates

In the case of machine learning, cost functions are used to evaluate whether any model is performing poorly. The cost function is a measure of how wrong the model is in terms of its ability to estimate the relationship between two variables, say, a and b. Therefore, machine learning aims to find parameters, weights, or structures that minimize the cost function. 

The cost function is minimized with gradient descent which is an efficient optimization algorithm that attempts to find a local or global minimum of a function. SVM enables a model to learn the gradient or direction that the model should take to reduce errors [difference between actual variable (say b) and predicted b]. 


SVM is a supervised machine learning algorithm that can be used for regression problems and classification tasks. It uses kernel trick to change your data, and then based on these changes, it finds a suitable boundary between the possible outputs. SVM applications include protein fold and remote homology detection, speech recognition, text classification, facial expression classification, cancer diagnosis and prognosis, and stenography detection in digital images. Some other examples of SVM include its use in python, java code, etc.

There are no right or wrong ways of learning AI and ML technologies – the more, the better! These valuable resources can be the starting point for your journey on how to learn Artificial Intelligence and Machine Learning. Do pursuing AI and ML interest you? If you want to step into the world of emerging tech, you can accelerate your career with this Machine Learning And AI Courses by Jigsaw Academy.


Related Articles

Please wait while your application is being created.
Request Callback