82

I've gotten myself into some kind of horrible virtualenv mess. Help?!

I manage environments with conda. Until recently, I only had a python2 jupyter notebook kernel, but I decided to drag myself kicking and screaming into the 21st century and installed a python3 kernel; I forget how I did it.

My main (anaconda) python defaults to 2.7.

So here I am, merrily trying to use beautiful soup from inside my shiny new python3 kernel, and I don't seem to be able to do anything to get at whatever environment it's finding packages in. Viz (all from notebook):

from bs4 import BeautifulSoup -> ImportError: No module named 'bs4' 

Ok, fine, I'll install it using shell magic. Right? Right?

! pip install bs4 --> Collecting bs4 Downloading bs4-0.0.1.tar.gz Requirement already satisfied (use --upgrade to upgrade): beautifulsoup4 in /Users/[MY-USER]/anaconda/lib/python2.7/site-packages (from bs4) [...] Successfully built bs4 Installing collected packages: bs4 Successfully installed bs4-0.0.1 from bs4 import BeautifulSoup -> ImportError: No module named 'bs4' 

Oh no. Does it think I'm in a 2.7 env even though I'm running a python3 kernel? That won't do.

! conda info --envs --> # conda environments: # flaskenv /Users/[MY-USER]/anaconda/envs/flaskenv mesa /Users/[MY-USER]/anaconda/envs/mesa py35 /Users/[MY-USER]/anaconda/envs/py35 root * /Users/[MY-USER]/anaconda 

Ok, I can fix that. One of those is a 3.5 env.

! source activate py35 --> prepending /Users/[MY-USER]/anaconda/envs/py35/bin to PATH ! conda install beautifulsoup4 --> Fetching package metadata ....... Solving package specifications: .......... # All requested packages already installed. # packages in environment at /Users/[MY-USER]/anaconda: # beautifulsoup4 4.4.1 py27_0 

concerning...

! pip install bs4 --> Requirement already satisfied (use --upgrade to upgrade): bs4 in /Users/[MY-USER]/anaconda/lib/python2.7/site-packages 

more concerning...

from bs4 import BeautifulSoup -> ImportError: No module named 'bs4' 

ARRGH!!! headdesk Am I going to have to kill the kernel in order to fix this (and re-run a bit of work)? Is killing the kernel even going to work? How do I get my jupyter kernel to know what environment it's supposed to be running under?

thanks!

3
  • Possible duplicate of Execute Python script within Jupyter notebook using a specific virtualenv Commented Jul 20, 2018 at 19:09
  • +1 for headdesk - exactly how I have been for the last 2 days with Tensorboard and Jupyter and ... oh, the humanity! Commented Jan 3, 2020 at 10:25
  • Funny into to question made me laugh, "I've gotten myself into some kind of horrible virtualenv mess" Commented Jun 20, 2022 at 6:19

7 Answers 7

193

This is a tricky part of ipython / Jupyter. The set of kernels available are independent of what your virtualenv is when you start jupyter Notebook. The trick is setting up the the ipykernel package in the environment you want to identify itself uniquely to jupyter. From docs on multiple ipykernels,

source activate ENVNAME pip install ipykernel python -m ipykernel install --user --name ENVNAME --display-name "Python (whatever you want to call it)" 

If you only want to have a single Python 3 kernel, from the conda environment, just use python -m ipykernel install --user and it will reset the default python to the one in the virtualenv.

And yes, you will need to restart the kernel and re-run the prior steps.

See Also Using both Python 2.x and Python 3.x in IPython Notebook

Sign up to request clarification or add additional context in comments.

2 Comments

I also found that I hadn't installed jupyter into the new environment, but it still launched with the 'juptyer notebook' command. So, I didn't realize it at first.
It works but I don't get how Jupyter then finds the path to the environment. It is not mentioned in the kernel.json file.
13

@tschundler's solution works perfectly if your environment has already been created.

If you want to change the default kernel at the creation of your virtual environment and avoid any manual configuration, you just need to add jupyter at the end of the conda command:

conda create --name ENVNAME python=PYTHONVERSION jupyter

The correct kernel will then be used when you use ipython or jupyter notebook.

Comments

11

In my case somehow jupyter wasn't able to 'pick' the virtual environment's python. So I had to edit ~/.local/share/jupyter/kernels/{my_env_name}/kernel.json and add path to the interpreter

in the argv key.

3 Comments

If I could upvote this 5 times I would. I have Visual Studio Code, with Windows Subsystem as its console, I installed a virtualenv, then a jupyter, then I could not use libraries from the virtualenv. This solved it!!
do you have to replace the first argument of argv? tt stilll didnt seem to work....how does jupyter get notified about it?
I can see the jupyter folder but there is no kernels under it.
4

There is also an easy way here

workon my-virtualenv-name # activate your virtualenv, if you haven't already pip install tornado==4.5.3 pip install ipykernel==4.8.2 

You should now be able to see your kernel in the IPython notebook menu: Kernel -> Change kernel and be able to switch to it (you may need to refresh the page before it appears in the list). IPython will remember which kernel to use for that notebook from then on.

This worked for me. source

1 Comment

workon does- not working do you know any solution i have tried searching but it doesn't work i am using vs code
1
pip install --user ipykernel python -m ipykernel install --user --name=myenv Output Installed kernelspec myenv in /home/user/.local/share/jupyter/kernels/myenv 

and go to above directory open kernel.json

{ "argv": [ "/home/user/anaconda3/envs/myenv/bin/python", # path to your virtual environment python "-m", "ipykernel_launcher", "-f", "{connection_file}" ], "display_name": "myenv", "language": "python" } 

Comments

1

I know this is an old question, but nobody has mentioned nb_conda_kernels, which was made to solve this problem. You must have it installed in the environment from which you launched the running notebook, and any other environment which has kernel/s installed (ipykernel, r-irkernel, etc ) will be available as an option (Start Preferred Kernel):

enter image description here

Comments

0

Sharing my experience here, in case someone made the same mistake:

TL;DR

In VSCode I created the virtual environment from an Ubuntu terminal but forgot to activate the WSL extension too. Once I activated the WSL extension, I was able to change the Jupyterkernel to my virtual environment.

Full explanation

Using VSCode I work with the WSL extension (see here). I created my virtual environment from an Ubuntu terminal [Created a new terminal and selected 'Ubuntu (WSL)'] and then went to my Jupyter notebook to change my kernel, but it did not appear in the list.

I tried the solution proposed above of adding the path to kernel.json in my virtual environment, adapting to my case (I don't use conda) but I was surprised to see that the folder bin in my environment did not have a file named python.

The solution was to activate the WSL extension: lower right corner > click on the arrows (Open a remote connection) > reopen folder in WSL.

After this, I could see in the explorer two new entries in the bin folder for Python and Python3 (the arrow on the right side seem to indicate it's a symbolic link). Virtual environment now shows the python symbolic links

Then I went to my Jupyter notebook and changed the kernel to my virtual environment.

In the end, I did not have to change the path in kernel.json.

P.S: The above is my understanding of my problem and the solution given my non-existing knowledge about Linux, so any comments to complement/improve this answer are very welcome!

Comments

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.