Skip to main content
deleted 2 characters in body
Source Link
Sycorax
  • 95.8k
  • 23
  • 246
  • 405

I am new to neural networks. I am studying back propagation and saw different references. for a layer $k$, some references state that the error $\delta_j^k$ for neuron $j$ at $k$th layer is

$$ \delta_j^k = \dfrac{\partial E}{\partial a_j^k} $$

while some other references state

$$ \delta_j^k = \dfrac{\partial E}{\partial z_j^k} $$ where $z^k = w^l a^{(l-1)} + b^k$. Andrew N.G.Ng in his courses introduced this as $$ \delta^k = (W^{(k+1)})^T \delta^{(k+1)} .* \sigma^{'}(z^{(k)}) $$ that made me confused. Which one is true?

I am new to neural networks. I am studying back propagation and saw different references. for a layer $k$, some references state that the error $\delta_j^k$ for neuron $j$ at $k$th layer is

$$ \delta_j^k = \dfrac{\partial E}{\partial a_j^k} $$

while some other references state

$$ \delta_j^k = \dfrac{\partial E}{\partial z_j^k} $$ where $z^k = w^l a^{(l-1)} + b^k$. Andrew N.G. in his courses introduced this as $$ \delta^k = (W^{(k+1)})^T \delta^{(k+1)} .* \sigma^{'}(z^{(k)}) $$ that made me confused. Which one is true?

I am new to neural networks. I am studying back propagation and saw different references. for a layer $k$, some references state that the error $\delta_j^k$ for neuron $j$ at $k$th layer is

$$ \delta_j^k = \dfrac{\partial E}{\partial a_j^k} $$

while some other references state

$$ \delta_j^k = \dfrac{\partial E}{\partial z_j^k} $$ where $z^k = w^l a^{(l-1)} + b^k$. Andrew Ng in his courses introduced this as $$ \delta^k = (W^{(k+1)})^T \delta^{(k+1)} .* \sigma^{'}(z^{(k)}) $$ that made me confused. Which one is true?

Source Link

Neural network, what is the error value for a layer?

I am new to neural networks. I am studying back propagation and saw different references. for a layer $k$, some references state that the error $\delta_j^k$ for neuron $j$ at $k$th layer is

$$ \delta_j^k = \dfrac{\partial E}{\partial a_j^k} $$

while some other references state

$$ \delta_j^k = \dfrac{\partial E}{\partial z_j^k} $$ where $z^k = w^l a^{(l-1)} + b^k$. Andrew N.G. in his courses introduced this as $$ \delta^k = (W^{(k+1)})^T \delta^{(k+1)} .* \sigma^{'}(z^{(k)}) $$ that made me confused. Which one is true?