tac works with records and their separators, attached, by default after the corresponding record. This is somewhat counter-intuitive compared to other record-based tools (such as AWK) where separators are detached.
With -b, the records, with their newline attached, are as follows (in original order):
Hello\nNew\nWorld\n!\n
Output in reverse, this becomes
\n\n!\nWorld\nNewHello which corresponds to the output you see.
Without -b, the records, with their newline attached, are as follows:
Hello\nNew\nWorld\n!\n
Output in reverse, this becomes
!\nWorld\nNew\nHello\n