It turns out this was a problem with ruby gems as a whole, not just rspec. And there's an emacs package that fixes the problem: rvm, as outlined hereas outlined here.
So, a step by step solution:
M-x package-install rvm M-x rvm-use-default You can also add (rvm-use-default) to your init file to enable the package from startup.