This has nothing to do with int.to_bytes(), and everything to do with the bytes type itself.
I get a value of expint = b'\x876' instead of b'\x87\x36'
This complaint does not make any sense, because b'\x876' and b'\x87\x36' are the same thing:
>>> b'\x87\x36' b'\x876' >>> b'\x87\x36' == b'\x876' True
A bytes object's representation is not simply a hex dump. The grammar of byte-strings allows many other options, for historical reasons (i.e.: because in 2.x we used to pretend that they could represent text). The canonical representation (i.e. the one produced by Python when you print an instance) only uses \x style escapes as a last resort.
In ASCII, the byte with value 0x36 is mapped to the symbol 6. (Again: we used to pretend we could represent text this way. And if we only have to deal with English plus a small selection of European languages, and can choose which European language we're dealing with at any given time, we can sort-of get away with it, too.)