In order to understand this fact, see George Boolos, Logic Logic and Logic (1998), page 57 :
[Consider the formula : ]
(C) $\quad \exists X(\exists x Xx \land \forall x \forall y[Xx \land (x = 0 \lor x = y+1) \rightarrow x \ne y \land Xy])$.
[It] is a sentence that is true in all nonstandard models of arithmetic but false in the standard model [footnote : To see that (C) is true in any nonstandard model, take as $X$ the set of all nonstandard elements of the model. $X$ is nonempty, does not contain $0$, hence contains only successors, and contains the immediate predecessor of any of its members.]
To see that it is false in the standard model, suppose that there is some suitable set $X$ of natural numbers. $X$ must be nonempty: if its least member $x$ is $0$, let $y = 0$; otherwise $x = y + 1$ for some $y$. Since $x$ is least, $y$ is not in $X$, and "$Xy$" is false.
For non-standard models of first-order $\mathsf {PA}$, see George Boolos & John Burgess & Richard Jeffrey, Computability and Logic (5th ed - 2007), page 304 :
To sum up: the elements of the domain of any nonstandard model $\mathcal M$ of arithmetic are going to be linearly ordered by LESS THAN. This ordering will have an initial segment that is isomorphic to the usual ordering of natural numbers, followed by a sequence of blocks, each of which is isomorphic to the usual ordering of the integers (negative, zero, and positive). There is neither an earliest nor a latest block, and between any two blocks there lies a third. Thus the ordering of the blocks is what was called [...] a dense linear ordering without endpoints, and so, it is isomorphic to the usual ordering of the rational numbers.
Added
Assuming that we have convinced ourselves that the formula (C) is true every non-standard model but false in $\mathbb N$, if we assume that (C) is expressible in first-order logic, we can use it as the formula $\varphi$ of Hurkyl's answer and the conclusion follows.