Questions tagged [kernel-trick]
Kernel methods are used in machine learning to generalize linear techniques to nonlinear situations, especially SVMs, PCA, and GPs. Not to be confused with [kernel-smoothing], for kernel density estimation (KDE) and kernel regression.
751 questions
2 votes
1 answer
95 views
Multidimensional Scaling for mixed-variables
I am interested in the following problem; say I have a bunch of continuous and categorical variables. I wish to compute dissimilarities among my observations by considering kernel functions. In this ...
- 125
0 votes
0 answers
51 views
Can I use different kernel functions for different independent variables (features)? And should I?
I'm using kernel regression to model a non-linear relationship between several independent variables and a dependent variable. I understand kernel functions and bandwidth selection, but I’m wondering ...
- 135
2 votes
1 answer
120 views
Relation of RKHS to Feature Space in kernel learning
In standard SVM formulations, we typically look for a vector $w \in \mathbb{R}^D$ that defines a hyperplane in $\mathbb{R}^D$. The decision function is then of the form: $$ f(x) = \operatorname{sign}(\...
- 117
1 vote
0 answers
54 views
RKHS, Norms and Regularization [closed]
Note: Also asked on Math-Stack but without an answer (Math-Stack Question). I am trying to understand a paper about regularization in non-parametric regression and I am struggling to understand the ...
- 367
0 votes
0 answers
216 views
Prove matrix constructed based on gaussian RBF is PSD
I have a radial basis function $k(x, y) = \exp(-{(x-y)}^T M {(x-y)})$ where $M$ is a symmetric PSD matrix. I know that $k(\cdot)$ is a kernel itself: Prove that multiplication with positive ...
- 101
3 votes
2 answers
228 views
In the contex of Kernel regression why do we define the feature map as equal to the Kernel $\varphi(x)=k(\cdot ,x)$?
I have a notational confusion I am trying to clear up. In the context of Kernel regression the following relationship between the kernel and the feature map is defined: Consider a positive-definite ...
- 1,475
1 vote
0 answers
113 views
What is the best way to use Gaussian Processes to approximate highly non-stationary functions?
Gaussian process regression has trouble approximating functions with "kinks". So, what is the most widely used method to deal with this problem? I have found many proposed methods, including ...
- 43
0 votes
1 answer
125 views
SVM Kernel to compare histograms as input vectors
In lecture 7 of CS229 by Andrew Ng he mentions at the very end a specific Kernel that allows an SVM to "classify" how similar two histograms are, such as the demographics of 2 countries. He ...
- 33