Storage and Data Transfer
All storage on Baskerville is provided for active research data. The storage is not provided for long-term data retention or archival.
Home Directory¶
User home directories are restricted to 20GB. Please avoid storing any research data in your home directory, which
has only a small quota and so should only be used for settings and config files like .bashrc, or .gitconfig etc.
You can see the location of your home directory, and how much space you are using, by running:
my_quota
Home directory Quota
The quota on your home directory is 20GB. We will not increase the quota of your home directory.
Project Space¶
Project space, available at /bask/projects/, is the main storage space and should be used to store your
research-related data, code, etc. Each Baskerville project has a dedicated storage space available to all members of
the project. Project space by default is allocated at 1TB, however additional quota for projects can be requested and
is allocated based on justified need.
You can see the project space available to you:
Temporary (scratch) storage¶
There are two directories of scratch space provided for temporary storage of the user data that is generated by running jobs:
/tmp: local scratch storage (scoped to the running job – see info below)./scratch-global: shared (network) scratch storage, available across all the compute nodes.
Info
When inside a Slurm job the /tmp directory is constructed in a per-job privately mounted
namespace and therefore not visible to the users of other jobs running on the node.
If a job uses significant I/O (Input/Output) then local scratch space (i.e. the /tmp directory) should be
utilised and the files only written back to the final project directory (on the network storage) once the job
completes. An example of where this would apply is if an application or script makes heavy use of the disk (as
opposed to RAM) for its working data.
Data Transfer¶
Useful sites:
rsync– see the rsync man page for usage instructions; for general guidance please see the following webpage: https://linuxize.com/post/how-to-use-rsync-for-local-and-remote-data-transfer-and-synchronizationscp– see the scp man page for usage instructions; for general guidance please see the following webpage: https://linuxize.com/post/how-to-use-scp-command-to-securely-transfer-filesGlobus- see below for a guide how to setup Globus on Baskerville
Globus¶
To use Globus on Baskerville:
- Go to the Globus website: https://app.globus.org
-
If this is your first time accessing Globus you will be asked to choose how to login. In the ‘Look-up your organisation’ box search for your organisation. You should be sent to your organisation’s Single Sign-On (SSO) web page. Sign in and you should be redirected back to the Globus website.
Globus website image
-
To access the Baskerville endpoint select the ‘File Manager’ icon and in the Collection box search Baskerville and select
Baskerville Tier2 SystemFile Manager image
-
When you click the Baskerville collection the first time you will see a prompt saying “Authentication/Consent Required” - click
Continue -
On the Identity Required page you have two options, depending on whether you have previously accessed the system:
- If this is your first time, click on the single link that is presented.
- If you have accessed the system before then click on the upper link which contains your username.
Link identity request
-
Both of the previous links will take you to a Baskerville authentication page where you will be required to enter your Baskerville username, Baskerville password (N.B. not your organisation’s password) and OTP (One Time Password)
Baskerville authentication page
-
If you are a new user you will be redirected back to the Identity Required page once more in order to re-authenticate. However, this time please click the first link which contains your username.
- You will automatically start in your home directory; to find the path to your project directory you can check Baskerville Admin.