Support Vector Machines

The objective is to find the widest street that separates the two classes.

Imagine a vector perpendicular to the median line to the street.

SVM-1.PNG

SVM-2.PNG

SVM-3.PNG

SVM-4.PNG

svm-5.PNG

SVM-6.PNG

SVM optimization doesn’t get stuck in local maximum, it has a convex base unlike Neural nets which could often get stuck in local maxima.

For linearly separable points, SVM works fine, but for nonlinearly separable points, you need to do a transformation to project these points to higher dimensional space, so that they can get linearly separable.

Conclusion:

  1. A general method that is convex and guaranteed to produce a global solution.
  2. Small sigma in gaussian kernel can cause overfitting because then classification is shrunk right around the sample points.
  3. For handwritten character recognition, linear kernel with n=2(nonlinear) works good.
Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s