Upcoming Wiki Changes
Important! There are changes coming to the Wiki. For information about these changes, as well as what options users have, please visit the Wiki Service Change Options and FAQs page.
A scratch filesystem is a place to store intermediate job data which can be destroyed when a job is finished. Performance is better than using your home directory or an LSS share which are meant for long term data storage.
Each compute node has its own local scratch filesystem. Users may read from and write to this using their own exclusive directory at /localscratch/Users/<HawkID>.
In addition to local storage, the HPC cluster system has two large, shared filesystems mounted across all its nodes via NFS and BeeGFS. Analogously, users can read from and write to their own exclusive directory at /nfsscratch/Users/<HawkID> for the NFS system and /scratch/Users/<HawkID> for the BeeGFS filesystem.
Scratch filesystems are a shared resource available for the convenience of all users. Therefore, files on these filesystems are subject to deletion after a certain lifespan as determined by the HPC policy committee. Home account storage and purchased storage are not subject to this policy.
On /localscratch, the allowed file lifespan is 30 days after the file was last accessed, where each file's age is the time elapsed since its access timestamp ("atime"). An automated cleanup process runs periodically on each node to delete files whose atime has reached the maximum lifespan.
Cleaning of /localscratch on compute nodes is done on an opportunistic basis, cleaning when no jobs are on the node. However, if the space becomes limited, the node will go into an alarm state and will then be cleaned.
If your job writes data to /localscratch, please retrieve everything you need and remove unneeded files as the last part of the job, because it's difficult to access that same compute node after a job exits! A compute node can become unavailable if its /localscratch filesystem becomes too full. If that happens, all files will be removed from /localscratch without considering lifespan in order to restore the compute node to service. For more information on using localscratch see Advanced Job Submission#localscratch.
On /nfsscratch, the allowed file lifespan is 40 days after first being written, where each file's age is the time elapsed since its creation timestamp ("crtime") which is tracked on the fileserver. An automated cleanup process will run periodically on the server to delete files whose crtime has reached the maximum lifespan. This space is provided by a single ZFS storage server connected via NFS. It is best suited to large streaming I/O, reading or writing large data sequentially to or from a small number of files.
Please contact firstname.lastname@example.org with any questions or for assistance with this.
On /scratch, the allowed file lifespan is 60 days after first being written. This space is provided by a BeeGFS parallel filesystem. It is best suited for small random I/O, reading or writing to many small files or specific sections of files.
ls -l file
ls -lc file
ls -lu file
All of these timestamps can be different for a single file. Most file and archive utilities will maintain the first 3 timestamps, either by default or optionally. This includes using archive mode ('-a') with either 'cp' or 'rsync'. However, note that no utility can affect a file's crtime at all over NFS.
If you have data to stage for a job or to retrieve after a job is finished in a scratch filesystem, there are a few options.