Error - no module named virtualenv #360
Description
- Bitbucket: https://bitbucket.org/hpk42/tox/issue/360
- Originally reported by: jaraco
- Originally created at: 2016-08-24T02:40:36.844
When using the advertised setuptools integration on a clean operating system that doesn't already have virtualenv installed, the tox run will fail when it attempts to invoke virtualenv:
py35 create: /vagrant/Dropbox/code/public/cherrypy/.tox/py35 ERROR: invocation failed (exit code 1), logfile: /vagrant/Dropbox/code/public/cherrypy/.tox/py35/log/py35-0.log ERROR: actionid: py35 msg: getenv cmdargs: ['/usr/bin/python3', '-m', 'virtualenv', '--python', '/usr/bin/python3.5', 'py35'] env: {'TERM': 'xterm-256color', 'HOME': '/home/vagrant', 'SHLVL': '1', 'LS_COLORS': 'rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.jpg=01;35:*.jpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36:', 'MAIL': '/var/mail/vagrant', 'LOGNAME': 'vagrant', 'LESSOPEN': '| /usr/bin/lesspipe %s', 'USER': 'vagrant', 'PWD': '/vagrant/p/cherrypy', 'SSH_CLIENT': '10.0.2.2 65337 22', 'LANG': 'en_US', 'SSH_CONNECTION': '10.0.2.2 65337 10.0.2.15 22', 'PYTHONHASHSEED': '2586869941', 'LESSCLOSE': '/usr/bin/lesspipe %s %s', 'XDG_SESSION_ID': '9', '_': '/usr/bin/python3', 'SSH_TTY': '/dev/pts/0', 'PLAT': 'linux-x86_64', 'VIRTUAL_ENV': '/vagrant/Dropbox/code/public/cherrypy/.tox/py35', 'LANGUAGE': 'en_US:', 'PATH': '/vagrant/Dropbox/code/public/cherrypy/.tox/py35/bin:/home/vagrant/bin:/home/vagrant/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games', 'OLDPWD': '/home/vagrant', 'XDG_RUNTIME_DIR': '/run/user/900', 'SHELL': '/bin/bash'} /usr/bin/python3: No module named virtualenv ERROR: InvocationError: /usr/bin/python3 -m virtualenv --python /usr/bin/python3.5 py35 (see /vagrant/Dropbox/code/public/cherrypy/.tox/py35/log/py35-0.log) I think the issue is obvious - tox is launching a subprocess to invoke virtualenv, but since virtualenv was loaded dynamically by the setuptools distutils command test, the subprocess doesn't have that library.
I've worked around this issue in the past (when using pytest-runner to test libraries whose test suite would launch subprocesses) by adding all of the items in sys.path to the environment variable PYTHONPATH when launching the subprocess.
I think there are a few options to address this issue.
- Tox could rely on pyvenv on supported Python 3 versions, eliminating the dependency on virtualenv in those environments.
- Tox could launch virtualenv in-process rather than as a subprocess.
- Setuptools could implement the PYTHONPATH technique for all
testcommands, avoiding these issues in the general case. - Tox could implement the PYTHONPATH technique when launching virtualenv.
- Tox could explicitly ensure that virtualenv will be on the path in the subprocess (knowing that it could have been resolved by the
testcommand. - Tox could drop support for the distutils-bootstrapped technique, possibly preferring rwt as an alternate one-line bootstrapper for tox (though rwt itself currently requires bootstrapping).
There may be other options as well. I've listed these approaches in order by my preference.
What are the thoughts and preferences of the tox maintainers?