Machine Learning - XII. Support Vector Machines (Week 7)

http://blog.csdn.net/pipisorry/article/details/44522881

机器学习Machine Learning - Andrew NG courses学习笔记

Support Vector Machines支持向量机

{SVM sometimes gives a cleaner and more powerful way of learning complex nonlinear functions}

Optimization Objective优化目标

Alternative view of logistic regression

Note:

1. 线条表示：蓝色线代表logistic regression， 紫红色代表SVM；for the SVM we‘re going to replace this blue line with purple and magenta line.It just do something pretty similar to logistic regression but it turns out that this will give the SVM computational advantage that will give us later on an easier optimization problem, that will be easier for stock trades and so on.

Cost function for SVM

Note:

1. The first is the term which is the cost that comes from the training set A and the second is this term, which is the regularization term B(without lambda).
2. by setting different values for this regularization parameter lambda.We could trade off the relative way between how much we want to fit the training set well,as minimizing A, versus how much we care about keeping the values of the parameters small.
3. use a different parameter C and we instead are going to minimize C times A plus B.C playing a role similar to 1 over lambda.
So for logistic regression if we send a very large value of lambda, that means to give B a very high weight.Here is that if we set C to be a very small value corresponds to giving B much larger weight than C than A.

Hypothesis for SVM

Note:

1. SVM doesn‘t output the probability. I predict 1, if theta transpose x is greater than or equal to 0.And so, having learned the parameters theta, this is the form of the hypothesis for the support vector machine.

intuition: 因为logstic reg的hypothesis是概率，加一个log就可以得到蓝色线条；而SVM中是紫红色线条不能由紫红线条 = log(H)推出H的表达式

Large Margin Intuition大边缘的直觉知识

Mathematics Behind Large Margin Classification (Optional)大边缘分类背后的数学

Kernels核

Using An SVM使用SVM

from:http://blog.csdn.net/pipisorry/article/details/44522881