Let’s step back and revisit the concept of DMZ.
When speaking of most things, especially technical things, we are always using some form of analogy. These analogies are handy for quick reference as long as we remain at a level and understanding appropriate to the analogy. Every analogy will break down if we probe deep enough.
We often say, “The DMZ,” or, “In the DMZ,” as if it is a thing or place but it’s not.
When connecting a device (generally a server or router) to the Internet, the initial question is what ports todo you want to Open/Allow.
The related question is, what do you want to do with attempts to communicate on ports other than those previously designated? If your answer is “Nothing or Block”, then you’re done, there is no DMZ.
The concept of a DMZ comes into play when you want to assign a server/handler for Other undesignated ports. The DMZ is a rule that says send these Other ports to my handler. Again, if you have no need or desire to handle Other ports, you have no need for a DMZ rule.
The analogy gets complicated, but not yet broken, when the Internet facing device is a router/firewall (yet another analogy). Here the initial port acceptance is (should be) accept no ports. To become useful, the Other port rules come into play. Designated Other ports such as 443 can be set to forward through to your 443 web server. This is not really DMZ as it is a designated port but many SoHo routers put this port forwarding control under a subcategory of DMZ.
DMZ is more undesignated ports. You could have a rule that says, “send all other ports to my special handler box”.
So some of your specifics:
“Should the reverse proxy be the only server in the DMZ”
The reverse proxy (often called a reverse server) is not a server at all, it’s a client. It has no static open ports forwarded from the router, only dynamic outbound initiated ports. No DMZ at all!
“Is it really unsafe to place a database, which has no port forwarded to, as long as the reverse proxy doesn't redirect anything to it, in the DMZ?”
It’s a non-sequitur. You have no DMZ and even if you had, the DMZ is a rule. With no forwarding rule to the database it’s not in a DMZ.
Here the DMZ analogy begins to break down. It has become common usage to refer to the network portion between two firewalls as a DMZ. It may or may not be as a function of the firewall rules.
If all you have is a database connected to your web server connected to your reverse proxy, proper port control should be all you need. No DMZ at all.
Realistically you may want another firewall between your database and the web server to try to protect against possible web server compromise, but this is unrelated to the concept of DMZ.