Skip to content

User Conda Environments

Support for User Conda Environments

Please be aware that whilst users can create and utilise Conda environments on Baskerville Portal it is an unsupported feature.

Baskerville Portal’s JupyterLab app can find and utilise user-defined Conda environments. This is done by parsing the file ~/.conda/environments.txt and for each line, generating a kernel.

Creation process

You must first create a Conda environment. Due to the potential size of Conda environments, please ensure that you create the environment in an appropriate project directory and not in your home directory, which has a maximum quota of 20 GB.

Environment naming

Note that the names of Conda environments cannot contain any uppercase characters.

Our recommendation is that you use Jupyter’s Launcher to access a Terminal on a compute node and then use the following boilerplate:

module restore system
module load bask-apps/test
module load Miniconda3/4.10.3
eval "$(${EBROOTMINICONDA3}/bin/conda shell.bash hook)"

# Define the path to your Conda environment (modify as appropriate)
# N.B. the environment directory's name must be lowercase only
# N.B. this path will be created by the subsequent commands if it doesn't already exist
CONDA_ENV_PATH="/bask/projects/b/bloggsj-project-name/${USER}_conda_env"

# Create the environment. Only required once.
conda create --yes --prefix "${CONDA_ENV_PATH}"
# Activate the environment
conda activate "${CONDA_ENV_PATH}"
# Choose your version of Python
conda install --yes python=3.10

# Continue to install any further items as required.
# For example:
conda install --yes numpy

Note

When the app locates a compatible Conda environment it will check whether it has the necessary kernel module (e.g. ipykernel) and install it if required.

Usage and ongoing modifications

After initially creating your new Conda environment you will need to start a fresh JupyterLab app in Baskerville Portal so that it is made available as a kernel. However, once this is completed you can run a Notebook using the Conda kernel whilst simultaneously modifying the Conda environment using a complementary Terminal tab (see below).

You will need to start a new Terminal and run the following code each time you want to modify the environment:

module restore system
module load bask-apps/test
module load Miniconda3/4.10.3
eval "$(${EBROOTMINICONDA3}/bin/conda shell.bash hook)"

# Define the path to your existing Conda environment (modify as appropriate)
CONDA_ENV_PATH="/bask/projects/b/bloggsj-project-name/${USER}_conda_env"

# Activate the environment
conda activate "${CONDA_ENV_PATH}"

# Make any modifications as required.
# For example:
conda install --yes requests

Note that there is no need to restart a running Conda kernel after making modifications – any changes will be available immediately.

Conda cache

Despite creating the Conda environment in a project directory (as covered above), by default Conda will still cache data in your home directory, which can rapidly increase in size and use-up the limited 20 GB quota. We therefore recommend that you perform one of the following options:

  • Periodically clean the cache by executing the following command in a Terminal window:

    conda clean --all
    
  • Prior to executing any Conda installation commands, export the following environment variable to a path with more storage space:

    export CONDA_PKGS_DIRS=/tmp
    

    (See the storage documentation for further information on temporary storage on Baskerville.)


Last update: June 13, 2022
Back to top