Skip to content

User Conda Environments

Anaconda Licensing

Conda is a popular method for installing research software and is often recommended by developers, particularly for Python-based tools but please be aware of the following:

The conda command that is provided via the Anaconda and Miniconda packages uses the Anaconda Public Repository Conda channel by default. This channel may not be free for use in academic research and may require a paid Anaconda licence for some types of research. We recommend the use of Miniforge, which defaults to using the Conda Forge channel.

Further information on Anaconda licensing can be found here:
https://www.anaconda.com/pricing

Support for User Conda Environments

Please be aware that whilst users can create and utilise Conda environments on Baskerville Portal it is an unsupported feature.

Baskerville Portal’s JupyterLab app can find and utilise user-defined Conda environments. This is done by parsing the file ~/.conda/environments.txt and for each line, generating a kernel.

Creation process

You must first create a Conda environment. Due to the potential size of Conda environments, please ensure that you create the environment in an appropriate project directory and not in your home directory, which has a maximum quota of 20 GB.

Environment naming

Please ensure that your Conda environments have unique names.

Our recommendation is that you use Jupyter’s Launcher to access a Terminal on a compute node and then use the following boilerplate:

N.B. the following examples use mamba, which is a newer and faster implementation of the conda command.

module purge; module load baskerville

module load bask-apps/live
module load Miniforge3/24.1.2-0

eval "$(${EBROOTMINIFORGE3}/bin/conda shell.bash hook)" # (1)!
source "${EBROOTMINIFORGE3}/etc/profile.d/mamba.sh"

# Define the path to your Conda environment (modify as appropriate)
# N.B. this path will be created by the subsequent commands if it doesn't already exist
CONDA_ENV_PATH="/bask/projects/_initial_/_projectname_/${USER}_conda_env" # (2)!

CONDA_PKGS_DIRS=/tmp # (3)!

# Create the environment. Only required once.
mamba create --yes --prefix "${CONDA_ENV_PATH}" # (4)!
# Activate the environment
mamba activate "${CONDA_ENV_PATH}"
# Choose your version of Python
mamba install --yes python=3.10

# Continue to install any further items as required.
# For example:
mamba install --yes numpy

  1. Running these eval and source statements removes the need to execute conda init or mamba init, both of which insert content into the user ~/.bashrc file that can in turn cause issues with other software on Baskerville.

    If your ~/.bashrc file contains the Conda initialisation content (denoted by # >>> conda initialize >>>) then we would advise removing it.

  2. Conda environments can be very large so we recommend creating them within project directories (i.e. /bask/projects) and not in home directories, where quota is limited.

    (_initial_ is the first letter of the project name.)

  3. By default, packages are cached in the user’s home directory. Baskerville home directories have limited quota and as such we advise caching packages in /tmp.

    See Conda Cache for further details.

  4. Passing the --yes option sets any confirmation values to ‘yes’ automatically, meaning that the user will not be asked to confirm anything manually.

Usage and ongoing modifications

Info

When the JupyterLab app locates a compatible Conda environment it will check whether it has the necessary kernel module (e.g. ipykernel) and install it if required.

After initially creating your new Conda environment you will need to start a fresh JupyterLab app in Baskerville Portal so that it is made available as a kernel. However, once this is completed you can run a Notebook using the Conda kernel whilst simultaneously modifying the Conda environment using a complementary Terminal tab (see below).

You will need to start a new Terminal and run the following code each time you want to modify the environment:

module purge; module load baskerville

module load bask-apps/live
module load Miniforge3/24.1.2-0

eval "$(${EBROOTMINIFORGE3}/bin/conda shell.bash hook)"
source "${EBROOTMINIFORGE3}/etc/profile.d/mamba.sh"

# Define the path to your existing Conda environment (modify as appropriate)
CONDA_ENV_PATH="/bask/projects/_initial_/_projectname_/${USER}_conda_env"

CONDA_PKGS_DIRS=/tmp

# Activate the environment
mamba activate "${CONDA_ENV_PATH}"

# Make any modifications as required.
# For example:
mamba install --yes requests

Note that there is no need to restart a running Conda kernel after making modifications – any changes will be available immediately.

modules and Conda environments

Do not load modules in combination with a Conda environment as you may experience issues. If you would like to switch from using modules to running User Conda Environments, and vice versa, then please shut down your JupyterLab session and relaunch a new interactive batch job.

Conda cache

Despite creating the Conda environment in a project directory (as covered above), by default Conda will still cache data in your home directory, which can rapidly increase in size and use-up the limited 20 GB quota. We therefore recommend that you perform one of the following options:

  • Periodically clean the cache by executing the following command in a Terminal window:

    mamba clean --all

  • Prior to executing any Conda installation commands, export the following environment variable to a path with more storage space:

    export CONDA_PKGS_DIRS=/tmp

    (See the storage documentation for further information on temporary storage on Baskerville.)