I suppose the attraction of application pages is that they are much closer to being a "normal" aspx page. You say you are building what is, in effect, an aspx application on top of SharePoint. If this is a custom application that you do not anticipate re-using, and it doesn't need to be modular or modifiable by users, then maybe application pages are the best solution for you. Application pages can also be very efficient at runtime.

In general, and particularly for third party application developers (and I think this is probably the group that Microsoft's second quoted recommendation is aimed at), then web parts and site pages are the way to go for many reasons. Here's a few: composition, re-use, end-user configureable, potential use in the sandbox, no touching the filesystem.