Chapter/parallel computing completed

[k493r0]

Details

Parallel Computing section is complete. I've removed the "shared memory" section as it is merged in "What is parallel computing" section.

Linked Issues

• Parallel Computing Content #6

[oraqlle]

Overall pretty good. I would like you to try and focus more on the general concept of parallel computing in general (threads, multicore CPU) and ignore OpenMP in the beginning because it is a specific parallel architecture ie. save it for a specific section later.

Similarly you don't need to mention distributed computing, distributed memory architectures or Slurm. These topics are covered in the next and previous chapters respectively.

With the challenges, I want to have some be more conceptual/theory like:

• How you parallelize a reduction?
• What invariants must you uphold for this to work for any binary operator (associativity and commutative) etc.

and then some that are coding tasks like:

• Create a C program that sums the numbers in a array from a file
• Run on you local machine
• Log into M3
• Copy you file (maybe some SSH/SCP can be done for this as a bonus skill)
• Create a Slurm job file for this program that runs on a single machine and single core/thread and measure the execution time and record it in a file
• Now create another copy and adapt the program to use OpenMP parallelism tools
• Create a new Slurm job file that runs on a single machine multiple cores/threads and measure the execution time and save to a text file.
• Is there an improvement?

Now that I think about it, we could have a GitHub template for training content that recruits can then make a copy from (not fork or clone there is an actual template option) with different files with a list of numbers with each file having a different size. This way they can clone it locally, write the original program and push the changes (Git practice), clone to M3 and write the new programs, same the results and Slurm batch files etc. in the repo which they can then save and we can "assess".

Regardless the goal is go more in depth with each topic and focus on the concepts and then give them challenges that actually make them write code for themselves with varying degrees of difficulty to get sufficiently comfortable with writing, building and running HPC programs not just building and running.

Have a look at the README for the instructions on how to run mdBook which uses CommonMark spec (a more general specification for Markdown than GitHub's) and so you can see and check how the pages render. Finally have a look at the Getting Started and Intro to C sections for an idea of the overall style and flow I'm aiming to achieve.

[k493r0]

Thanks for the review, I didn't actually attempt compiling the mdBook before I made the PR, and I should've done it earlier because I can see a lot of things breaking/feature not existing in mdBook.

The contents are ripped straight off the presentation slides with no additional details added - I will add the contents you have mentioned moving forwards .

Log into M3
Copy you file (maybe some SSH/SCP can be done for this as a bonus skill)
Create a Slurm job file for this program that runs on a single machine and single core/thread > and measure the execution time and record it in a file

I will presume that these tasks/challenges are going to be in the M3 section already, so in the parallel computing section, I will assume the reader has the knowledge to perform these tasks (maybe link to the M3 section), does that sound alright?

[oraqlle]

... I didn't actually attempt compiling the mdBook before I made the PR, and I should've done it earlier because I can see a lot of things breaking/feature not existing in mdBook.

I figured but all good. You can have it running will editing and it will dynamically update.

The contents are ripped straight off the presentation slides with no additional details added - I will add the contents you have mentioned moving forwards .

Awesome, I look forward to seeing what you come up with.

I will presume that these tasks/challenges are going to be in the M3 section already, so in the parallel computing section, I will assume the reader has the knowledge to perform these tasks (maybe link to the M3 section), does that sound alright?

Yes that is correct. Everything from Slurm to loading modules is planned for the M3 chapter while multi-machine and message passing is planned for the Distributed Computing chapter. Links or even just weak references are a good idea as well.

I'll also create the template GitHub project for the recruits now so you and @MonashDeepNeuron/hpc-training can start creating challenges in there. We can break each chapter into different directories and break those down further as need.

[oraqlle]

@MonashDeepNeuron/hpc-training HPC Training Challenges Template repo is up. 🚀

[oraqlle]

Looks good.