Tune Least Squares Support Vector Machine (LS-SVM) With Grid Search Optimization

If you follow @JuhaP's link, then you have a SciKit Learn compatible implementation:

class LSSVMRegression(BaseEstimator, RegressorMixin): """ An Least Squared Support Vector Machine (LS-SVM) regression class Attributes: - gamma : the hyper-parameter (float) - kernel: the kernel used (string: rbf, poly, lin) - kernel_: the actual kernel function - x : the data on which the LSSVM is trained (call it support vectors) - y : the targets for the training data - coef_ : coefficents of the support vectors - intercept_ : intercept term """ def __init__(self, gamma:float=1.0, kernel:str=None, c:float=1.0, d:float=2, sigma:float=1.0): self.gamma=gamma self.c=c self.d=d self.sigma=sigma if (kernel is None): self.kernel='rbf' else: self.kernel=kernel params=dict() if (kernel=='poly'): params['c']=c params['d']=d elif (kernel=='rbf'): params['sigma']=sigma self.kernel_=LSSVMRegression.__set_kernel(self.kernel,**params) self.x=None self.y=None self.coef_=None self.intercept_=None def fit(self,X:np.ndarray,y:np.ndarray): self.x=X self.y=y Omega=self.kernel_(self.x,self.x) Ones=np.array([[1]]*len(self.y)) A_dag = np.linalg.pinv(np.block([ [0, Ones.T ], [Ones, Omega + self.gamma**-1 * np.identity(len(self.y))] ])) B = np.concatenate((np.array([0]),self.y), axis=None) solution = np.dot(A_dag, B) self.intercept_ = solution[0] self.coef_ = solution[1:] def predict(self,X:np.ndarray)->np.ndarray: Ker = self.kernel_(X,self.x) Y=np.dot(self.coef_,Ker.T) +self.intercept_ return Y 

Now you may use SciKit Learn's built in Grid Search as implemented in GridSearchCV.

Credit: Danny E. P. Vanpoucke - Building your own SciKit Learn Regressor Class: LS-SVM (GitHub Repository).