I don't really understand how to approach this problem. I know that $w=\sum_{n=1}^{N}{a_ny_nx_n}$ and $y_n(w^T\cdot x_n+b)=1. $ So I can solve for $b$ from that equation but I can't figure out how to apply equation(1) that I'm supposed to use.
$\begingroup$ $\endgroup$
Add a comment |
1 Answer
$\begingroup$ $\endgroup$
1 From $$y^{(n)}(w^T\cdot x^{(n)} + b)=1,$$
since $y^{(n)}$ is binary, we have $$w^T\cdot x^{(n)} + b = y^{(n)}$$
That is $$b=y^{(n)}-w^T\cdot x^{(n)}.\tag{2}$$
Now, let's examine
\begin{align}&\sum_{m \in S} \alpha_m y^{(m)}\langle x^{(n)}, x^{(m)}\rangle\\&=\sum_{m \in S} \alpha_m y^{(m)}\langle x^{(n)}, x^{(m)}\rangle + \sum_{m \notin S} 0\cdot y^{(m)}\langle x^{(n)}, x^{(m)}\rangle \\ &=\sum_{m \in S} \alpha_m y^{(m)}\langle x^{(n)}, x^{(m)}\rangle + \sum_{m \notin S} \alpha_m\cdot y^{(m)}\langle x^{(n)}, x^{(m)}\rangle\\ &=\sum_{m=1}^N \alpha_m y^{(m)}\langle x^{(n)}, x^{(m)}\rangle \\ &=\langle \sum_{m=1}^N \alpha_my^{(m)}x^{(m)}, x^{(n)}\rangle\\ &= w^Tx^{(n)} \tag{3}\end{align}
Using $(2)$ and $(3)$, you should be able to obtain the conclusion.
- 1$\begingroup$ Nice derivation. One extra point: the averaging over ${\mathcal{M}}$ is done only for "numerical stability" (this is mentioned in 2006 Bishop's book Page 330), if the optimization process is exact (which is not), $b$ could be calculated using only one point from ${\mathcal{M}}$ without averaging. $\endgroup$Esmailian– Esmailian2019-05-03 14:35:01 +00:00Commented May 3, 2019 at 14:35