SVM's support vectors decision function representation-CodePudding

I am currently using SVM for my project with 'rbf' kernel.

What i understand from the theory is that the decision function value for the support vectors must be either 1 or -1. (if i use clf.decision_function(x))

But i find some support vectors, the decision function value is even 0.76, -0.88, 0.93 and so on.. (its not even closer to 1 or -1 like 0.99 nor -0.99).

What is wrong in this scenario? Or is my understanding wrong?

CodePudding user response：

I guess there is no range limitation for the decision function value output in SVM.

The value of the decision function for those points, will be a high positive number for high-confidence positive decisions and have a low absolute value (near 0) for low-confidence decisions.

Source here

Code Example:

import numpy as np
from sklearn.svm import SVC

X = np.array([[-1, -1], [-2, -1], [0, 0], [0, 0], [1, 1], [2, 1]])
y = np.array([1, 1, 2, 2, 3, 3])

clf = SVC()
clf.fit(X, y)

print(clf.decision_function(X))
print(clf.predict(X))

Output:

# clf.decision_function(X)
array([[ 2.21034835,  0.96227609, -0.20427163],
       [ 2.22222707,  0.84702504, -0.17843569],
       [-0.16668475,  2.22222222,  0.83335142],
       [-0.16668475,  2.22222222,  0.83335142],
       [-0.20428472,  0.96227609,  2.21036024],
       [-0.17841683,  0.84702504,  2.22221737]])

# clf.predict(X)
array([1, 1, 2, 2, 3, 3])

CodePudding user response：

What SVM is interested is the sign of the decision. e.g., if the sign is negative, the point lies (say) left of the hyperplane. Similarly if the sign is positive, the point lies right of the hyperplane. The value determines how far is it from the hyperplane. Therefore -0.88 means the point is left of the hyperplane and having a distance 0.88. Near the point to the hyperplane, the chances of mis-classification can be considered higher.

Have a look here

To quote from scikit-learn:

the function values are proportional to the distance of the samples X to the separating hyperplane.