The command line option -o (o standing for "Override-sort-field") also works on my Xubuntu machine and according to the Mac man page of top it should work on a Macintosh too. If I want to short by memory usage I usually use
top -o %MEM which sorts by the column %MEM. But I can use VIRT, RES or SHR too. On a Macintosh I would probably use mem or vsize.
I don't know why or how but this is pretty much different between Unix systems and even between Linux distributions. For example -o isn't even available on my Raspberry running Wheezy. It may be worth give it a try though.