Shortly:
Here what is going on, aTestFunction was created in aPackage` context, then you've removed it and since the current context is aPackage`Private` the definition of your function is created there.
More precisely:
New symbols are created in current $Context during read time, (1) the existance of aTestFunction between BeginPackage and Begin lines makes the aTestFunction an exported symbol (that means in aPackage` , because the symbol was not found and the current $Context is apackage` ).
After (2) Remove it's no longer there and (3) your definition is read in aPackage`Private` context. But aPackage`Private` is not going to be on $ContextPath or $Context after EndPackage[] so it is not found by ?.
I'm strongly encouraging to read a very nice answer of Szabolcs:
How symbol lookup actually works
BeginPackage["aPackage`"]; aTestFunction::usage = "aTestFunction usage"; (*1*) Begin["`Private`"]; Remove[aTestFunction]; (*2*) aTestFunction[arg_] := arg + 1; (*3*) Protect[aTestFunction]; End[]; EndPackage; ? aTestFunction Information::notfound: Symbol aTestFunction not found. >>
? aPackage`Private`aTestFunction 
