A byte has been variously (at least) 1, 6, 7, 8, 9, 12, 18, 20 and possibly 36 bits, depending on what computer you are looking at. I am taking "byte" here to mean "smallest addressable unit of memory", rather than using any sort of text-centric interpretation.
The 20-bit bytes were extremely common in the "IAS machines", in the 50s. 6, 12, 18 (and maybe 36) were quite popular in a variety of architectures in the 60s, 70s and to some degree 80s.
In the end, having a nice correspondence between "powers of 2" and "bits in an addressable unit" seem to have won out.