While we certainly don't reach your levels of volume, Physics.SE has a high percentage of discussion-in-the-comments cases. So I see where you're coming from on this.
Firstly, my personal philosophy on comments is the following: If it has little to no potential of improving the question, delete. This applies to chatty/useless comments, comments on posts where the author has declined fixing it according to the comment (this doesn't apply to "this post is wrong" comments, they are good guidance for future downvoters), and comments on old posts where the OP never came back. In the case of chatty comments, I sometimes give enough time for the OP to read them before deleting.
Alright. If mods are being inundiated by chatty comment flags, then one solution would be to treat all comment flags as similar to spam/offensive flags, without the attached rep penalty. So, 6 comment flags delete the comment. Then, show these comments in the 10k queue.
Incentivising comment flagging (making a successful comment flag count as 1/5th of a helpful flag for the badges) may also help get the comments cleaned up.
Now, discussions are a harder beast to tackle. To deal with these, I suggest a few things:
Firstly, mods need a tool that lets them migrate selected comments to chat with little hasslemods need a tool that lets them migrate selected comments to chat with little hassle. There have been cases when I've just been too lazy to do it(one has to quote all the comments in chat and then delete them) and I've asked the participants of the discussion to create a new chatroom if they wish to continue (with plans of coming back to clean up the comments once they finish). My guess is that the SO mods won't have the time to do this at all (as opposed to my laziness). If we want to battle these effectively, we need the tools for it.
Secondly, we should show the "move to chat" option quite early in a discussion, preferably after it has reached the threshold where there are at least two comments from at least two users each. This button will delete all the comments of the participants, and leave behind "a discussion was [moved to chat] link". If a comment is unequivocally a reply to a user who is participating in the discussion, it goes to chat as well.
It may be a good idea to scare people into moving to chat -- the move to chat link is summarily ignored by many. Something like "there's a chance this discussion will be deleted if it gets too long" may be enough. Even better, don't allow a discussion of more than X (X=10?) comments between two people. Disable the comment box and mention that it will be reenabled "if the side discussion is [moved to chat]"
Community flags could be tweaked to flag comment discussions of 10 comments instead of 20 (which could be easily dealt with if there were good tools for migrating discussions to chat).
For answering in the comments, a flag and a corresponding "convert to answer" mod option ought to be sufficient.
IMO, long discussions aren't that big a problem due to the collapsing. However, they do detract from more useful comments at times. And they add to clutter. It needs fixing, but wasting too much time doing so may not be the best idea.