When accessing an authenticated webpage with cookies and am getting an SSL Error. This seems to be a mac-specific issue that many solve by installing the python certificates (as I tried in the terminal session pictured below), but an exception throws (Permission Denied) when I attempt at installing.
How should I fix this error?
Code:
... import urllib.request opener = urllib.request.build_opener() print("Cookies: ", headers["Cookie"]) opener.addheaders.append(('Cookie', headers["Cookie"])) f = opener.open("https://home.nest.com/home/"+url_addtl_data) print(f) Error Message:
Traceback (most recent call last): File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/urllib/request.py", line 1318, in do_open encode_chunked=req.has_header('Transfer-encoding')) File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/http/client.py", line 1239, in request self._send_request(method, url, body, headers, encode_chunked) File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/http/client.py", line 1285, in _send_request self.endheaders(body, encode_chunked=encode_chunked) File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/http/client.py", line 1234, in endheaders self._send_output(message_body, encode_chunked=encode_chunked) File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/http/client.py", line 1026, in _send_output self.send(msg) File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/http/client.py", line 964, in send self.connect() File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/http/client.py", line 1400, in connect server_hostname=server_hostname) File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/ssl.py", line 407, in wrap_socket _context=self, _session=session) File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/ssl.py", line 814, in __init__ self.do_handshake() File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/ssl.py", line 1068, in do_handshake self._sslobj.do_handshake() File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/ssl.py", line 689, in do_handshake self._sslobj.do_handshake() ssl.SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:777) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/Users/Ryan/Desktop/Dev/Projects/Py/Database_intro/venv/main.py", line 97, in <module> f = opener.open("https://home.nest.com/home/"+url_addtl_data) File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/urllib/request.py", line 526, in open response = self._open(req, data) File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/urllib/request.py", line 544, in _open '_open', req) File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/urllib/request.py", line 504, in _call_chain result = func(*args) File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/urllib/request.py", line 1361, in https_open context=self._context, check_hostname=self._check_hostname) File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/urllib/request.py", line 1320, in do_open raise URLError(err) urllib.error.URLError: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:777)> Solution attempted:
Last login: Sat May 11 03:44:45 on ttys000 Ryan-Cocuzzos-Laptop:~ Ryan$ /Applications/Python\ 3.6/Install\ Certificates.command ; exit; -- pip install --upgrade certifi Collecting certifi Using cached https://files.pythonhosted.org/packages/60/75/f692a584e85b7eaba0e03827b3d51f45f571c2e793dd731e598828d380aa/certifi-2019.3.9-py2.py3-none-any.whl Installing collected packages: certifi Exception: Traceback (most recent call last): File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/pip/basecommand.py", line 215, in main status = self.run(options, args) File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/pip/commands/install.py", line 342, in run prefix=options.prefix_path, File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/pip/req/req_set.py", line 784, in install **kwargs File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/pip/req/req_install.py", line 851, in install self.move_wheel_files(self.source_dir, root=root, prefix=prefix) File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/pip/req/req_install.py", line 1064, in move_wheel_files isolated=self.isolated, File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/pip/wheel.py", line 345, in move_wheel_files clobber(source, lib_dir, True) File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/pip/wheel.py", line 316, in clobber ensure_dir(destdir) File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/pip/utils/__init__.py", line 83, in ensure_dir os.makedirs(path) File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/os.py", line 220, in makedirs mkdir(name, mode) PermissionError: [Errno 13] Permission denied: '/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/certifi-2019.3.9.dist-info' You are using pip version 9.0.1, however version 19.1.1 is available. You should consider upgrading via the 'pip install --upgrade pip' command. Traceback (most recent call last): File "<stdin>", line 44, in <module> File "<stdin>", line 25, in main File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/subprocess.py", line 291, in check_call raise CalledProcessError(retcode, cmd) subprocess.CalledProcessError: Command '['/Library/Frameworks/Python.framework/Versions/3.6/bin/python3.6', '-E', '-s', '-m', 'pip', 'install', '--upgrade', 'certifi']' returned non-zero exit status 2. logout Saving session... ...copying shared history... ...saving history...truncating history files... ...completed. Deleting expired sessions...30 completed. [Process completed] EDIT 5/19
Additional Terminal Sessions (from suggested solutions):
Ryan-Cocuzzos-Laptop:~ Ryan$ sudo easy_install pip Password: Searching for pip Best match: pip 19.1.1 Adding pip 19.1.1 to easy-install.pth file Installing pip script to /usr/local/bin Installing pip3.7 script to /usr/local/bin Installing pip3 script to /usr/local/bin Using /usr/local/lib/python2.7/site-packages Processing dependencies for pip Finished processing dependencies for pip Ryan-Cocuzzos-Laptop:~ Ryan$ brew install python Updating Homebrew... Warning: python 3.7.3 is already installed, it's just not linked You can use `brew link python` to link this version. Ryan-Cocuzzos-Laptop:~ Ryan$ brew link python Linking /usr/local/Cellar/python/3.7.3... Error: Could not symlink bin/2to3 Target /usr/local/bin/2to3 already exists. You may want to remove it: rm '/usr/local/bin/2to3' To force the link and overwrite all conflicting files: brew link --overwrite python To list all files that would be deleted: brew link --overwrite --dry-run python Ryan-Cocuzzos-Laptop:~ Ryan$ rm '/usr/local/bin/2to3' Ryan-Cocuzzos-Laptop:~ Ryan$ brew link --overwrite python Linking /usr/local/Cellar/python/3.7.3... 24 symlinks created Ryan-Cocuzzos-Laptop:~ Ryan$ python -c "import ssl;print(ssl.OPENSSL_VERSION)" OpenSSL 1.0.2r 26 Feb 2019 Ryan-Cocuzzos-Laptop:~ Ryan$ EDIT 5/21
Attempted to include SSL in the project and it threw an error.. this appears to be relevant, but, when checking (and as is shown above), you can notice that python is definitely using OpenSSL v. 1.0.2r.
python -c "import ssl;print(ssl.OPENSSL_VERSION)"output when run from your shell? You will get around the 2nd error by installing (upgrading) pip manually: bootstrap.pypa.io/get-pip.py.pythonin the command line. Using the full path can maybe help to solve your problem or help others help you solve it. See also stackoverflow.com/a/51970236/6018688