PS1="\${?#0}\$ "

It uses a special form of [parameter expansion][1], `${?#0}`, which means: "Remove the character zero if it is the first character of `${?}`, the exit code of the previous command."

You can also change the color of the prompt if the last exit code were not zero:

 PS1="\[\e[0;\$((\$?==0?0:91))m\]\$ \[\e[0m\]"

[![Prompt][4]][4]

That uses a [if-else ternary expression][2] `\$((\$?==0?0:91))` — with some escaping — that makes the color code `0;91m` (red, see [color codes][3]) if the last command exits with non-zero, or `0;0m` (your default color) otherwise.

 [1]: https://www.gnu.org/software/bash/manual/html_node/Shell-Parameter-Expansion.html
 [2]: https://www.gnu.org/software/bash/manual/html_node/Shell-Arithmetic.html
 [3]: https://unix.stackexchange.com/questions/124407/what-color-codes-can-i-use-in-my-ps1-prompt
 [4]: https://i.sstatic.net/tZEg3.png