Virtual Environments

The default Python environment contains the Anaconda Python distribution which comes with many of the Python packages in the scientific stack. It is however likely that at some point you will need to install additional packages.

Python environments#

Packages should be installed in their own Python virtual environments. Use the mkenv command to create a Python virtual environment:

mkenv
Usage: mkenv <name> [--clear] [--python python3.9] [--jupyter "Jupyter Launcher Name"]

For example, to create an environment with the default Python version:

mkenv myenv

or to use the Python 3.9 version:

mkenv myenv --python /opt/conda/envs/py39/bin/python

Environments are created inside the $HOME/Python directory. In order to activate one environment use workon:

workon myenv

Conda environments#

Alternatively one may use conda in order to create an environment. Conda is more appropiate when it is also required to install binary libraries and it is not limited to Python. In order to use conda, the useconda function loads the necessary variables

useconda

For example, in order to install TensorFlow 1 in a conda virtual environment with Python 3.6 one can do then

conda create -y -p $HOME/.conda/envs/tf python=3.6
conda install -y -p $HOME/.conda/envs/tf tensorflow=1.15

and activate the environment with

conda activate tf

Conda environments can be listed with the command

conda env list
# conda environments:
#
base                     /soft/conda
hub                      /soft/conda/envs/hub
napari                   /soft/conda/envs/napari
py311                    /soft/conda/envs/py311
py36                     /soft/conda/envs/py36
py37                     /soft/conda/envs/py37
py38                     /soft/conda/envs/py38
py39                     /soft/conda/envs/py39
renv                     /soft/conda/envs/renv

Adding the environment to Jupyter#

In order to write notebooks in Jupyter using a newly created environment it is necessary to install the Jupyter kernel and register it with the interface. The easiest is to add the --jupyter option to the mkenv command. For example:

mkenv py39 --python /opt/conda/envs/py39/bin/python --jupyter "Python 3.9"

To add an already created environment, or a conda environment, follow these steps:

workon myenv  # if using Pip
conda activate myenv  # if using Conda

pip install ipykernel
python -m ipykernel install --user --display-name 'My Env' --name myenv

If you are using conda and pip fails with the message ERROR: Could not find an activated virtualenv (required). you can use the following command instead:

env PIP_REQUIRE_VIRTUALENV=false pip install ipykernel

Installing Python packages#

Python packages should only be installed into their respective environment. After creating a virtual environemnt myenv you can, for example, install the numpy package with:

workon myenv
pip install numpy

or

conda activate myenv
conda install numpy

Installing packages in the default Python environment is disabled. If you need to install a package in the default Python environment, please contact the support team.

Removing environments#

In order to remove an environment created with mkenv you just need to remove the directory with the environment name inside $HOME/Python. For example:

rm -rf $HOME/Python/myenv

will remove the environment myenv.

In order to remove a Conda environment, you can use the conda env remove command followed by the environment name. For example:

conda env remove myenv

Troubleshooting#

I cannot install a package in my environment#

If you receive an error when trying to install a package in your environment, check that you have activated the environment with workon or conda activate before trying to install the package.

I cannot see my environment in Jupyter#

If you cannot see your environment in Jupyter, check that you have installed the Jupyter kernel for your environment. See the section Adding the environment to Jupyter for more information.