The fact that one cannot reliably link to comments aside: direct linking would require listing all comments by themselves. Currently comments are grouped together, which keeps the list short. I like that.
(Your responses tab and your activity tab do link to comments directly, but only for comments on answers. I think that's actually a bug that could be solved by auto-expanding comments when explicitly browsing to an answer.)