My question is: Why do we do this?
Basically, because by restricting ourselves in this way, we make it easier ... for ourselves, and others who may need to read / modify the code in the future ... to understand the code, and the way that the various parts interact.
Most developers understand code by a mental process of abstraction; i.e. mentally drawing boundaries around bits of code, understanding each bit in isolation, and then understanding how each bit interacts with other bits. If any part of the code could potentially mess around with the "innards" of any other part of the code, then it makes it hard for the typical developer to understand what is going on.
This may not be a problem for you while you are writing the code, because you may be able to keep all of the complex interactions in your head while you create the code. But in a year or two's time, you will have forgotten a lot of the details. And other people never had the details in their heads to start with.
Why shouldn't I have one class modifying the global variables of another class directly?
Because it makes your code harder to understand; see above. The larger your codebase is, the more pronounced the problem will be.
Another point is that if you use over-use globals (statics actually) then you create problems if your code needs to be multi-threaded / reentrant, for unit testing, and if you need to reuse your code in other contexts.
And even if you shouldn't why are the protected, private, and public modifiers even necessary? It's as if programmers don't trust themselves not to do it, even though they are the ones writing the program.
It is not about trust. It is about expressing in the source code where the boundaries are.
If I write a class and declare a method or a field private, I know that I don't have to consider the problem of what happens if some other class calls it / accesses it / modifies it. If I'm reading someone elses code, I know that I can (initially) ignore the private parts when mapping the interactions and boundaries. The private and protected modifiers and package private just provide different granularities of boundary.
(Or maybe it is about trust; i.e. not trusting ourselves to remember where the abstraction boundaries in our design are / were.)