0

I have two python environments and somehow there is some kind of link between them.

/home/testapi/API25/env is the original venv /home/preprodapi/API25/env was created by means of a cpio copy from the first. This worked find for many months. But now a problem has cropped up.

The symptom is that in preprodapi the pytz package isnt able to find the timezone Africa/Johannesburg (and presumably others), as evidenced by the stack trace:

 Traceback (most recent call last): File "/home/preprodapi/API25.8512/validator/echo.py", line 244, in jsonified_wrapper response_obj = request_handler(*args, **kwargs) File "/home/preprodapi/API25.8512/validator/echo.py", line 478, in bearer_token_wrapper return request_handler(*args, **kwargs) File "/home/preprodapi/API25.8512/validator/echo.py", line 1068, in globaldb_connection_wrapper return request_handler(*args, **kwargs) File "/home/preprodapi/API25.8512/validator/echo.py", line 569, in get_school_wrapper return request_handler(*args, **kwargs) File "/home/preprodapi/API25.8512/validator/echo.py", line 697, in school_admin_wrapper return request_handler(*args, **kwargs) File "/home/preprodapi/API25.8512/routehandlers.py", line 4035, in email_report do_email_report(kwargs.get('reportid'), json_attrs, getctx_school().get('schoolname')) File "/home/preprodapi/API25.8512/validator/echo.py", line 1155, in do_email_report tz = pytz.timezone(sender.school.get("local_timezone")) File "/home/testapi/API25/env/lib64/python3.5/site-packages/pytz/__init__.py", line 181, in timezone pytz.exceptions.UnknownTimeZoneError: 'Africa/Johannesburg'

Notice how it switches from /home/preprodapi/.... to /home/testapi/... in the final item.

But WHY is this happening?

(env) [root@ip-172-31-8-200 API25]# deactivate [root@ip-172-31-8-200 API25]# pwd /home/preprodapi/API25 [root@ip-172-31-8-200 API25]# . env/bin/activate (env) [root@ip-172-31-8-200 API25]# pip uninstall pytz Uninstalling pytz-2017.2: Would remove: /home/testapi/API25/env/lib/python3.5/site-packages/pytz-2017.2.dist-info/* /home/testapi/API25/env/lib/python3.5/site-packages/pytz/* Proceed (y/n)? n (env) [root@ip-172-31-8-200 API25]# python Python 3.5.5 (default, Feb 6 2018, 10:57:32) [GCC 4.8.5 20150623 (Red Hat 4.8.5-16)] on linux Type "help", "copyright", "credits" or "license" for more information. >>> import pytz >>> print(pytz.timezone('Africa/Johannesburg')) Africa/Johannesburg

For the record I can't find any soft links between the venv's

(env) [root@ip-172-31-8-200 API25]# pwd /home/preprodapi/API25 (env) [root@ip-172-31-8-200 API25]# find env -type l -ls 40549590 0 lrwxrwxrwx 1 root root 3 Feb 4 12:54 env/lib64 -> lib 40549592 0 lrwxrwxrwx 1 root root 15 Feb 4 12:54 env/bin/python3.5m -> /bin/python3.5m 40549593 0 lrwxrwxrwx 1 root root 10 Feb 4 12:54 env/bin/python -> python3.5m 40549594 0 lrwxrwxrwx 1 root root 10 Feb 4 12:54 env/bin/python3 -> python3.5m

Help please!

P.S note /home/preprodapi/API25.8512 is a cpio copy of /home/preprodapi/API25. I get the exact same results when I test in the API25.8512 sub-directory

Note #2: The same does not happen with another venv on this host

[root@ip-172-31-8-200 API25.8512]# cd /home/apiuser [root@ip-172-31-8-200 apiuser]# cd API25 [root@ip-172-31-8-200 API25]# . env/bin/activate (env) [root@ip-172-31-8-200 API25]# pip uninstall pytz Uninstalling pytz-2018.9: Would remove: /home/apiuser/API25/env/lib/python3.5/site-packages/pytz-2018.9.dist-info/* /home/apiuser/API25/env/lib/python3.5/site-packages/pytz/* Proceed (y/n)? n
Improve this question
5
  • 1
    What's a "cpio copy"? cpio is an archiver, not a copy utility. Commented Feb 4, 2019 at 20:45
  • Could you upgrade your examples to a minimal reproducible example? Without the means to reproduce the problem, it's impossible to say with confidence what the issue is. E.g. will the same happen if you create a new venv, install some packages into it, then copy it? Commented Feb 4, 2019 at 20:48
  • What's the value of pytz.__file__ after you "uninstalled" pytz from the current virtual env? Commented Feb 4, 2019 at 20:53
  • What's sys.path and PYTHONPATH? Are you sure you're inside the right venv when the first error happens? Commented Feb 5, 2019 at 0:17
  • cpio is indeed a very very capable copy program. Commented Feb 5, 2019 at 4:38

1 Answer 1

Reset to default
1

You need to check your sys.path and find the source of anomalies, if any. See Debugging modifications of sys.path for a way to track changes to sys.path and Can I zip all the python standard libs and the python still able to import it? for how it's constructed.

venv is implemented via Py3's stock site.py:

If a file named "pyvenv.cfg" exists one directory above sys.executable, sys.prefix and sys.exec_prefix are set to that directory and it is also checked for site-packages (sys.base_prefix and sys.base_exec_prefix will always be the "real" prefixes of the Python installation). If "pyvenv.cfg" (a bootstrap configuration file) contains the key "include-system-site-packages" set to anything other than "false" (case-insensitive), the system-level prefixes will still also be searched for site-packages; otherwise they won't.

When creating a venv, python and a number of other files are copied into <venv>/bin (<venv\Scripts in Windows), and a pyvenv.cfg is placed into <venv> for site.py to find upon Python startup. activate prepends <venv>/bin to PATH so that the local executable is started instead of the system one when you type "python".

Ultimately, this results in a sys.path that combines system-wide standard library with venv-specific 3rd-party modules. It would look something like this:

>>> sys.path ['', '<venv>/bin/python36.zip', <system Python platlib>, <system Python purelib>, '<venv>', '<venv>/lib/site-packages']

So, normally, there shouldn't be folders from another venv in sys.path resulting directly from venv logic. They may result from PYTHONPATH, or from some .pth files, or even from your own code. The above diagnostics should show where they come from.

Improve this answer
Sign up to request clarification or add additional context in comments.

1 Comment

It appears that the solution is when one clones a Python application, exclude the venv, re-create it, and re-install those packages from a pip freeze > requirements file from the source. I don't have time to test this properly right now but it makes complete sense based on my findings so I'm awarding the solution.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.