This is all about the roles of everyone involved. Whether you are browsing a website, or buying a muffin from that cafe you know, the result is the same.
The server isn't the more powerful device, it's the one "serving" requests. The client is just that, the one that makes the requests that need to be served. You are the client when you request a muffin, or a website, or in this case remote access.
As a client you ask "move the mouse here", "open this for me", and "what do you see right now?". Then the server serves you by sending what you asked for. This is the relationship in all RAT applications.