Skip to content

Containerisation with Apptainer

Apptainer docs

This documentation page provides a brief overview of using Apptainer on Baskerville, along with some specifics concerning its implementation. However, for further information please refer to the relevant version of the Apptainer official documentation.

Baskerville supports containerisation using Apptainer. The apptainer command is available on all nodes (although note the warning below regarding /tmp) and does not require the explicit loading of a corresponding module.

Apptainer replaces Singularity

The Apptainer application replaces the previous containerisation solution, Singularity. It is broadly a drop-in replacement and for now the command singularity will continue to function, although it will actually run Apptainer. For further information on the move from Singularity to Apptainer please see this article: https://apptainer.org/news/community-announcement-20211130

Pulling images and running containers

/tmp directory size

Due to the limited size of the /tmp directory on the Baskerville login nodes, please run all apptainer pull commands inside Slurm jobs and therefore on the compute nodes. (We recommend using an Interactive Job to do this.)

To pull an image from Docker Hub:

apptainer pull docker://python
apptainer pull docker://python:3.8.11  # pulls a specific tag

Two common methods for using an Apptainer container are exec and shell:

  • apptainer exec is for where you want to run a command in a container and then exit the container as soon as the command completes
  • apptainer shell is for where you want to launch a container and then attach to its shell

An example of apptainer exec:

apptainer exec python_3.8.11.sif python --version

The above command will spawn a container from the specified image file, execute the command python --version, print the output and then exit the container.

Using GPUs with containers

Accessing GPU resources inside your container is as simple as adding the --nv flag when starting your container. For example, if you run 

apptainer exec python_3.8.11.sif nvidia-smi

you will get a FATAL error that the nvidia-smi executable is not found. However, if you instead run

apptainer exec --nv python_3.8.11.sif nvidia-smi

you will see information about the GPUs you have been allocated for your job.

Building Containers

--fakeroot not required!

If you have previous experience of building Singularity containers please note that it’s no longer necessary to build using --fakeroot as the required privilege escalation is handled automatically.

To build an image from an Apptainer Definition File, please execute the following commands:

unset APPTAINER_BIND
apptainer build my_image.sif my_image_definition.def

Resetting APPTAINER_BIND

After you’ve built your image you may want to reset APPTAINER_BIND with export APPTAINER_BIND=/bask so that you can access your project directory from your container. You can bind other directories, such as /scratch-global by passing the --bind option to your exec or shell command.

Interactive Development

To test development of an Apptainer image interactively use the --sandbox facility, which builds the image as a directory that can then be run with the --writable option.

Suggested workflow:

  1. Run: unset APPTAINER_BIND

  2. Create a sandbox directory either…

    1. from a base OS image, e.g. Rocky Linux:

      apptainer build --fix-perms --sandbox "/tmp/${USER}/my-sandbox-dir" docker://rockylinux:8.6

      or…

    2. From an Apptainer definition file:

      apptainer build --fix-perms --sandbox "/tmp/${USER}/my-sandbox-dir" ./my-definition-file.def

  3. Run the sandbox as a container in writeable mode with “root” privileges:

    apptainer shell --fakeroot --writable "/tmp/${USER}/my-sandbox-dir"

  4. Perform the necessary package installs and test your image’s functionality iteratively.

  5. Write the required commands back into an Apptainer Definition File.
  6. Exit the sandbox container.
  7. Build the image from the resultant definition file as per the instructions above.

Apptainer’s cache and temporary directories

By default, Apptainer uses the following directory to store its cache: ~/.apptainer/cache. However, due to the limited size of users’ home directories on Baskerville we move the Apptainer cache to the following location: /tmp/apptainer/cache. This means that the Apptainer cache does not persist between jobs. There could be situations where it is desirable to retain the cache between jobs, in which case please define the following environment variable as appropriate, whilst being mindful of available storage space:

export APPTAINER_CACHEDIR=/path/to/preferred/cache/directory