before diving into the guide let's see quickly how an HPC cluster works, in general, the structure of an HCP cluster would look like
the cluster is composed of headnodes, computenodes, and the storage that is mounted to the nodes through the network. Those three components are connected with an interconnection network (ethernet, infinitband etc ..).
the headnodes are responsible for managing everything related to the cluster like the deployment, networking, and resources management as well as handling connections from users via ssh or any remote access tool, this task can be handled by another type of nodes we can name it loginnodes.
usually, HPC clusters are running a Linux system, and because they can scale from tens to thousands of nodes, a lot of tools are created to automate the management of the clusters (system deployment, networking, workload management) some of them are free (opensource) and others are paid, the difference is, the paid tools are easy to install, use and more automating plus they have support. one of the paid tools is Bright Cluster Manager which has a free version called easy8 that is limited to up to 8 nodes. we will use it to get familiar with how an HPC cluster works, and then we can use other opensource tools.
Now let's get started building a virtual cluster based on bright cluster manager
First, we need to have an account in brightcomputing.com, so we can request a license key as well as the IOS image. This will take several minutes from creating an account and getting approved to downloading the IOS image.
- Here is the link to create the account https://customer.brightcomputing.com/Customer-Create-Account
- After entering the customer portal, go to Easy8 — Request Free License option, fill the required chunks and click request.
then you will receive an email contain the license key, and a link to set some settings and download the IOS image,
go to the link and choose what Linux distro suits you and click download. wait several minutes until your download will get ready
wait several minutes until your download will get ready
reload the web page from time to time until you get the following page, click on the ISO file to download the ISO image
- we will use the bare metal installation method, which means installing the IOS image on the machine with no operating system, this can be done either on a physical machine or a virtual one. For the physical machines, you can use a bootable USB stick or a DVD to store the ISO image and boot it from them. but in this tutorial, we will use virtual machines with VirtualBox, here are the steps to create the VMs.
NOTE: we will create two VMs, the "headnode" and "node01" with the same steps.
then we need to decide what network topology our cluster will use, usually in HPC clusters the following network topology is used:
So based on that our cluster will have two networks, one is the external network which will include only the head/logins nodes, and a private network that includes all the compute nodes plus head/login nodes, in real life the nodes will be connected to a switch, inside the switch will be two VLANs which will represent the two networks, but in VirtualBox, we will simply add network adapters and set there types to the following:
network settings for the head node:
select the headnode and click settings
choose network
At "attached to" option, change NAT to bridged adapter, so we can configure a static ip address of the headnode, and be able to remote access it via ssh
note that the first adapter will be used by the external network, but for the internal one, we will add another one with a simple modification. click on "adapter 2", check the box "Enable Network Adapter", at the "Attached to" option select internal network, and keep the default name. this should look like
click "ok" to save the settings.
For node01 we need just to change its first adapter from NAT to internel network,
click "ok" to save the settings.
we need an aditional modification within the node01 so it can boot from the network via IPXE.
in settings->system, check the network's box and drag it to the top
now we can bring up the cluster, by powering on the head node first and boot it with the ISO image.
In the VirtualBox window, double on the headnode or click the arrow start
click the browse icon
click "Add disk image" icon
find the ISO image (usually it will be in the downloads folder), select it, click open
choose the loaded ISO image and click "choose"
finally, click start
if everything went well this ISO boot menu will show up
select "start bright cluster manager graphical installer" and procced to the installation section
for the installation steps, please refer to https://support.brightcomputing.com/manuals/9.2/installation-manual.pdf section 3.3 Head Node Installation: Bare Metal Method
Be careful with the network configuration, after the installation you should be able to connect to the internet, so you can update system packages and upgrade them, by sudo yum update and then sudo yum upgrade, after that bring your product key and install your license, please refer to https://support.brightcomputing.com/manuals/9.2/installation-manual.pdf section 4.3.3 Requesting A License With The request-license Script
after installing the headnode properlly and while it is running, now, we can power on node01 by double-clicking it in Virutaulbox, it will ask you to load an ISO but we setup node01 to be booted from the network, so we can just click cancel, after few moments the VM should look like:
if everything went well, the VM should look like:
choose "AUTO" and procced by hitting enter. you will be asked to accept the given configuratoin or choose manually, choose accept the configuratoin looks right
then choose "AUTO" for the install mode.
after node installation the screen should look like
At this point, you have a very fresh mini HPC cluster based on birght cluster manager, please refer to https://support.brightcomputing.com/manuals/9.2/admin-manual.pdf for advanced usage
if you need help, contact [email protected]
Thank you