How to get the support vectors when using LinearSVC?-CodePudding

I am using GridSearchCV and would like to save the support vectors as follows:

np.save("support_vectors.npy", gs_cv.best_estimator_.named_steps['svm'].support_vectors_)

But I get this error:

  File "./improved_grid_search.py", line 500, in <module>
    np.save("support_vectors.npy", gs_cv.best_estimator_.named_steps['svm'].support_vectors_)
AttributeError: 'LinearSVC' object has no attribute 'support_vectors_'

I checked and indeed LinearSVC doesn't seem to have this attribute. Where then are they saved?

I am using the following Pipe:

search_spaces = []
if ovr:
        if 'linear' in kernel_lst:
            search_spaces.append({'svm': [OneVsRestClassifier(LinearSVC())], # optmized version of linear
                                  'svm__estimator__C': c})
        if 'poly' in kernel_lst:
            search_spaces.append({'svm': [OneVsRestClassifier(SVC(kernel='poly', cache_size=cache_size))],
                                  'svm__estimator__degree': degree,
                                  'svm__estimator__C': c})
        if 'rbf' in kernel_lst:
            search_spaces.append({'svm': [OneVsRestClassifier(SVC(kernel='rbf', cache_size=cache_size))],
                                  'svm__estimator__gamma': gamma,
                                  'svm__estimator__C': c})
else:
        if 'linear' in kernel_lst:
            search_spaces.append({'svm': [LinearSVC()], # optmized version of linear
                                  'svm__C': c})
        if 'poly' in kernel_lst:
            search_spaces.append({'svm': [SVC(kernel='poly', cache_size=cache_size)],
                                  'svm__degree': degree,
                                  'svm__C': c})
        if 'rbf' in kernel_lst:
            search_spaces.append({'svm': [SVC(kernel='rbf', cache_size=cache_size)],
                                  'svm__gamma': gamma,
                                  'svm__C': c})

svm_pipe = Pipeline([('svm', DummyClassifier())])

And grid search is done this way:

grid_cv_object = GridSearchCV(
        estimator = svm_pipe,
        param_grid = search_spaces,
        cv = cv_splits,
        scoring = make_scorer(matthews_corrcoef), # a callable returning single value, binary and multiclass labels are supported
        n_jobs = -1, # use all processors
        verbose = 10,
        refit = refit,
        return_train_score = True
    )

CodePudding user response：

LinearSVC optimizes a linear model (like LogisticRegression or Lasso) with a complexity penalty via some gradient-based method, without regard for support vectors. I'm afraid only SVC and NuSVC provide support_vectors_.

CodePudding user response：

As you have already discovered yourself, LinearSVC does not have a support_vectors_ attribute, only coef_ and intercept_ ones.

However, according to the documentation, LinearSVC is:

Similar to SVC with parameter kernel=’linear’, but implemented in terms of liblinear rather than libsvm, so it has more flexibility in the choice of penalties and loss functions and should scale better to large numbers of samples.

So, if you don't have performance issues, you can end up with your desired support_vectors_ simply by replacing LinearSVC() with the largely equivalent SVC(kernel='linear').