Hi stats_noob: This isn't an answer because I don't have one. It's an interesting question but there's somewhat of a contradiction in your initial model:
You write, $$X_t = \mu \times t + \sigma \times B_t$$.
So, you are saying that $\mu$ is constant and $\sigma$ are constant Note that brownian motion is continuous and you do specify $t$. Now, when you subtract, you say that you get independent increments with mean $\mu =0$ and the $\sigma$ stays the same. But this is not necessarily the case. It's only true if you were subtracting at the frequency associated with $\mu$.
In other words, when you subtract, you need to subtract at the frequency with which $\mu$ was established.So my guess is that you are talking more about a discrete random walk where one subtracts at the same frequency as the random walk itself. With an actual brownian motion, if you subtract at some random frequency, you won't necessarily get independent increments because you don't know at what frequency to subtract. ( the only reason the subtraction-indepenent increments trick works is because BM is cumulative process but it's cumulative continuously ).
Now, if it is just a random walk where the frequency is known, then you can do what you did with the standard testing just as you would in the normal distribution case without time. But BM is trickier cause it's continuous so the subtraction is questionable-problematic. I think this notion of the cumulative nature of BM is missed often and it only hit me about 6 months to a year ago ago when someone asked something on cross-validated. I hope this helps some ?
P.S: I would change your last line to multiplication and take out the parentheses because it really is a multiplication if you knew the correct frequency to subtract. But you don't ( you are testing what $\mu$ is ) which is why I think you might be talking about a random walk ?
ADDENDUM: ML: 07-13-2025 #================================================================#
stats_noob: I was dealing with somewhat of a similar issue a few months ago so I think, if I add a few more basic equations, I can clarify what the issue with your original equation. Let me look at my old notes and I'll add to this sometimes this week. The idea ( which may not be very clear at the moment ) is that you really need to consider a random walk of the log price rather than a brownian motion. The brownian motion introduces issues because it really can be thought of a cumulative sum of infinitesimal normals which is going to be problematic when attempting to construct independent increments.
ADDENDUM: ML: 07-15-2025 #================================================================#
stats_noob: I was looking around to try to find an articulate explanation of what I've been thinking about and having difficulty saying. I think I hit paydirt with Tiberiu's answer ( Ignore the comment about variance ratios. that's not a true test of BM).
It's pretty clear from Tiberiu's answer that it's essentially not possible to deal with the BM directly. He wrote the answer that I wanted to write but couldn't !!!! He does a truly beautiful job. No way is that AI !!!!!
He also touches on the bottom about using the discrete version as a replacement but doesn't get into the fact that there's a nuisance parameter involved even in the discrete case. I'm going to look into that further because it seems problematic to me. Of course, feel free to look for it also. I would google something like: "test for ( constant) drift in the discrete random walk".
https://www.quora.com/Is-there-a-statistical-test-to-test-whether-a-time-series-behaves-like-a-Brownian-motion