Shortly:
Here what is going on, aTestFunction was created in aPackage` context, then you've removed it and since the current context is aPackage`Private` the definition of your function is created there.
More precisely:
New symbols are created in current $Context during read time, (1) ::usage line makes aTestFunction an exported symbol. After (2) Remove it's no longer there and (3) your definition is read in Private` context. But aPackage`Private` is not going to be on $ContextPath or $Context after EndPackage[].
You can read more in very nice answer of Szabolcs: How symbol lookup actually works
BeginPackage["aPackage`"]; aTestFunction::usage = "aTestFunction usage"; (*1*) Begin["`Private`"]; Remove[aTestFunction]; (*2*) aTestFunction[arg_] := arg + 1; (*3*) Protect[aTestFunction]; End[]; EndPackage; ?aTestFunction Information::notfound: Symbol aTestFunction not found. >>
?aPackage`Private`aTestFunction 