I ended up just installing the exec-path-from-shell package. I tried setting it myself but I figured installing the package was cleaner than having bunch of extra lines in my configs.