Skip to content

Commit

Permalink
Networking, Jupyter Docs
Browse files Browse the repository at this point in the history
  • Loading branch information
Indigo5684 committed Sep 24, 2024
1 parent 775f392 commit d7f5f69
Showing 1 changed file with 46 additions and 3 deletions.
49 changes: 46 additions & 3 deletions docs/01-users/01-hpc.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,14 +13,57 @@ ssh [email protected]
# ssh [email protected]
```

## Resources


## Data Storage
### Data Storage

We provide three networked volumes for your convenience:

- `/home/your-name` is the generic home directory. Assume that space is limited.
- `/bighome/your-name` is based on HDDs, and intended for large objects. While we don't set per-user quotas, we do have an overall cluster limit.
- `/scratch/your-name` is based on SSDs. Note that this directory should **not be used for long-term storage**, and will be **wiped regularly**.

Note that your `bighome` and `scratch` directories are availible under `$BIGHOME` and `$SCRATCH`.
Note that your `bighome` and `scratch` directories are availible under `$BIGHOME` and `$SCRATCH`.

### Network Ports

Ports 30000-30500 are exposed on all nodes.

## Using Jupyter

We will create a Jupyterlab setup in a Conda environment. For the purposes of instruction, we will install **Miniforge** and create a sample Conda environment.

```bash
# Install Miniforge
curl https://github.com/conda-forge/miniforge/releases/latest/download/Miniforge3-Linux-x86_64.sh
chmod +x Miniforge3-Linux-x86_64.sh
./Miniforge3-Linux-x86_64.sh

# Create Sample Environment
conda create --name JupyterTutorial

# Activate the sample environment
conda activate JupyterTutorial

# Install Jupyter
conda install jupyter

# Install Slurm Extension
jupyter labextension install jupyterlab-slurm
```

Next, we will create a script to launch a Jupyter environment, to be executed via `srun`. Note that the port below, `30000`, may not be availible on your particular node. See the [Network Ports](#network-ports) section above for potentially availible ports.

```bash
echo 'conda activate JupyterTutorial
jupyter-notebook --no-browser --ip=0.0.0.0 --port 30000' > ./jupyter.sh
chmod +x ~/jupyter.sh
```


Next, use `srun` to launch the notebook.

```bash
# 2
srun -t 6-09:59:59 --cpus-per-task=2 --ntasks=1 --mem-per-cpu=8G --pty bash -i
```

0 comments on commit d7f5f69

Please sign in to comment.