As long as you have at least two Unity versions installed, theThe entry in the Unity Version column is a drop-down menu. Click on it and you can select one of your installed versions to which to upgrade your project. Remember to save a backup elsewhere first!
In older versions of the Unity Hub, this drop-down menu would only work if you had at least two versions of Unity installed (whether or not either of them matched the project version). But as of the last time I checked in 2020, this now works even if you have only a single version installed.
