Proving that a function is discontinuous

Take a point $x_0$ where $x_0^2 \neq 2x_0-1$ and consider two sequences approaching $x_0$, one consisting of only rational numbers, the other of only irrational numbers. To do it in terms of $\epsilon$ and $ \delta$, figure out the difference between $x_0^2$ and $2x_0 -1$ and make $\epsilon$ less than that.

You state that

I have to prove that some $\varepsilon > 0$ exists such that for every $\delta > 0$ if $|x-x_0|<\delta \to |f(x)-f(x_0)| >\ \varepsilon$

but this isn't the correct negation of the statement

$f$ is continuous.

To find the correct negation, start with the definition of continuity, which is

For every $x_0$ and every $\varepsilon > 0$ there exists a $\delta > 0$ such that if $|x-x_0| < \delta$ then $|f(x)-f(x_0)| < \varepsilon$.

Put more formally, this becomes

$\forall x_0\quad\forall \varepsilon > 0\quad\exists \delta > 0\quad\forall x\quad:\quad |x-x_0| < \delta \rightarrow |f(x)-f(x_0)|<\varepsilon.$

To negate that, we prefix the statement with a negation ($\lnot$), and then use the rules

• $\lnot\forall x\;\ldots$ is the same as $\exists x\;\lnot\ldots$
• $\lnot\exists x\;\ldots$ is the same as $\forall x\;\lnot\ldots$
• $\lnot(A \rightarrow B)$ is the same as $A \land \lnot B$ (i.e. the negation of $A$ implies $B$ is that there is a case where $A$ holds but $B$ doesn't).

Applying that to the definition of continuity yields

$\exists x_0\quad\exists \varepsilon > 0\quad\forall \delta > 0\quad\exists x\quad:\quad |x-x_0| < \delta\;\land\;|f(x)-f(x_0)|\geq\varepsilon$

So what you have to do is to first pick some $x_0$ and some $\varepsilon$. Then you have to show that for every $\delta > 0$ you can find some $x$ which satisfies $|x-x_0|<\delta$, but for which $|f(x)-f(x_0)| \geq \varepsilon$.

In other words, for some fixed $x_0$ and $\varepsilon$, you need to find $x$ which lie arbitrarily close to $x_0$, yet whose images under $f$ all lie at least $\varepsilon$ away from $f(x_0)$.

The problem with your statement is that you didn't correctly deal with the $\forall x$ in the definition of continuity, and failed to notice that it gets turned into an existantial quantifier in the negation. The way you stated the negation therefore seems to require that you find some $\delta$ such that for all $x$ closer than $\delta$ to $x_0$ the image under $f$ lies at least $\varepsilon$ away from $f(x_0)$.

But this is overly strict, and it fact impossible -- you can always pick $x=x_0$ (since then obviously $|x-x_0|=0 < \delta$ for every $\delta > 0$), and then $|f(x)-f(x_0)|=0<\varepsilon$ will always hold, regardless of whether $f$ is continuous or not.

In the correct negation, on the other hand, you see that you only need to find a single $x$ for every $\delta$. For your $f$, these $x$ will be irrational if $x_0$ is rational, and rational if $x_0$ is irrational. Note that you don't need to do both cases -- you can choose whatever value you like for $x_0$ (and also $\varepsilon$), because these quantities are under an existential quantifier in the negation of continuity.