Getting Started on Cayuga

Table of Contents Overview New Cayuga accounts Getting started guides CAC guides Slurm Using Miniconda/Anaconda File transfers Job examples

Overview

This guide is intended to point you to resources to get started using the Cayuga cluster, and to address issues new users often encounter.

• Don't run anything resource-intensive (significant RAM or CPU usage) directly on the head node. Things like compiling code, data transfers, single-threaded data pre-processing, and installing software in your own virtual environments are ok.
• Please be sure to carefully review the main documentation before signficant usage.
• Also be sure to review additional documentation sources on Slurm (and other related systems) if you don't already have a lot of experience with it.
• Read or ask about the implications of filesystem performance before running I/O intensive jobs.
• If you plan to use a large fraction of available resources, use long walltime limits, or queue many jobs at once (including with job arrays), please communicate with the other active/heavy users to coordinate your usage.

New Cayuga accounts

• Set up your private/public key with CAC Credentials for CAC Services
• Step by Step Instructions

Getting started guides

CAC guides

These CAC guides may be useful if this is your first time using an HPC cluster similar to those hosted by CAC, as well as for many other tools that might be new to you. Note that many of the details are cluster-dependent and you will need to make substitutions in many commands:

• CAC Getting Started on Private Clusters
• CAC Virtual Workshops

Slurm

• Slurm quick start
• Slurm video tutorials
• CAC's Slurm overview

Using Miniconda/Anaconda

• Download and install Miniconda (or Anaconda) in $HOME.
• Be sure to choose the Linux 64-bit installer with the Python version you need.
• It is generally recommended to create a separate environment for each project if they might have different software requirements; separate installations are also possible for more isolation. You can keep an environment.yml file for each environment's packages so that it's easy to move or redeploy them as needed.
• The installer will add init code to ~/.bashrc, so be sure this will actually get loaded (check that your default shell is bash, or that ~/.profile also does what is normally does in ~/.bash_profile by default.
• venv is also an alternative; python 3.9 is available. You can also consider using Singularity containers instead.

File transfers

• Globus docs

Job examples