Going to answer my own question here.
Turns out it has to do with Turbo, which is a set of front-end JS tooling that is turned on by default in a Rails app.
One of the defaults is a pre-fetching feature -- it used to be called "Turbo Links" but now is just rolled into the larger Turbo framework which is part of the larger Hotwire framework :).
By default, when you rollover any links, Turbo will pre-fetch those URLs. The idea is that by the time a user chooses to click on a link, the pre-fetching will make full-page refreshes feel even snappier by not having to wait until the click. (Even the ~300ms natural human delay of hovering over a link and then clicking on it can make a noticeable difference depending on what's being pre-fetched).
What was happening? In my testing, I'd roll over the "Logout" button which would automatically fetch the logout call -- essentially logging me out even if I had not pressed the button.
To fix this, just disable Turbo's pre-fetch feature on the logout button specifically like so (if you're using the Rails link_to):
<%= link_to "Logout", logout_path, data: { turbo_prefetch: false } %>
or if you're using native HTML...
<a href="/logout" data-turbo-prefetch="false">Logout</a>
You can also apply the data-turbo-prefetch="false" attribute on any parent HTML element and any links inside that element will not prefetch (meaning you can do this on <body> to just disable the whole darn thing.
Hope this helps solve someone's problem one day.