$A= \left[ {\begin{array}{ccccc} 1 & 1 \\ 1 & -1 \\ -2 & 4 \end{array} } \right] $ and $b = \left[ {\begin{array}{cccc} 1 \\ 2 \\ 7 \end{array} } \right] $
I use the formula $p = A(A^{T}A)^{-1}A^{T}b$
$A^{T}A = \left[ {\begin{array}{ccccc} 6 & -8 \\ -8 & 18 \\ \end{array} } \right]$
$(A^{T}A)^{-1} = \frac{1}{44} \left[ {\begin{array}{ccccc} 18 & 8 \\ 8 & 6 \\ \end{array} } \right]$
$A^{T}b = \left[ {\begin{array}{ccccc} -11 \\ 27 \\ \end{array} } \right]$
Putting everything together,
$p = \frac{1}{44}\left[ {\begin{array}{ccccc} 1 & 1 \\ 1 & -1 \\ -2 & 4 \end{array} } \right] \left[ {\begin{array}{ccccc} 18 & 8 \\ 8 & 6 \\ \end{array} } \right] \left[ {\begin{array}{ccccc} -11 \\ 27 \\ \end{array} } \right]= \frac{1}{44}\left[ {\begin{array}{ccccc} 92 \\ -56 \\ 260 \end{array} } \right]$
Which is completely different from the book's answer $p=(3,0,6)$. Where did I go wrong? Also, if the question asked for a projection of $b$ onto the row space of $A$, would I take the transpose of $A$ first, say let $B = A^{T}$ then use the formula $p = B(B^{T}B)^{-1}B^{T}b$?