Complete Initial S1-24 Revamp

README.md

@@ -35,6 +35,7 @@ $ mdbook serve --open
 - Yuki Kume
 - Osman Haji
 - Duc Thanh Vinh Nguyen
+- Linton Charles
 
 ## Code of Conduct, License & Contributing
 

src/.chapter7/challenges.md

@@ -0,0 +1 @@
+# Challenges

src/SUMMARY.md

@@ -25,21 +25,21 @@
   - [Challenges](./chapter2/challenges.md)
 
 - [Operating Systems](./chapter3/chapter3.md)
-  - [Components of Linux](./chapter3/linux-components.md)
-  - [Memory & IO](./chapter3/memory-io.md)
-  - [Processes & Scheduling](./chappter3/processes-sched.md)
+  - [Computer Architecture](./chapter3/computer-architecture.md)
+  - [Pointers & Memory](./chapter3/memory-pointers.md)
+  - [Intro to Linux](./chapter3/linux-intro.md)
   - [Threading & Concurrency](./chapter3/threads-concurrency.md)
-  - [Inter-Process Communication](./chapter3/ipi.md)
+  - [Processes](./chapter3/processes.md)
+  - [Scheduling Algorithms](./chapter3/scheduling.md)
   - [Challenges](./chapter3/challenges.md)
 
-- [More C]()
-  - [Pointers](./chapter4/pointers.md)
+- [More C](./chapter4/chapter4.md)
   - [Dynamic Memory](./chapter4/memory.md)
   - [Structures](./chapter4/structs.md)
   - [Macros & The Preprocessor](./chapter4/macros.md)
-  - [System Calls]()
-  - [Spawning Processes & Threads]()
-  - [Challenges]()
+  - [System Calls](./chapter4/syscalls.md)
+  - [Spawning Processes & Threads](./chapter4/spawn-procs.md)
+  - [Challenges](./chapter4/challenges.md)
 
 - [M3 & SLURM](./chapter5/chapter5.md)
 
@@ -51,33 +51,26 @@
   - [Software & Tooling](./chapter5/software-tooling.md)
   - [Challenges](./chapter5/challenges.md)
 
-- [Introduction to Parallel Computing](./chapter8/chapter8.md)
-  - [Multithreading](./chapter8/multithreading.md)
-  - [Synchronisation](./chapter8/synchronisation.md)
-  - [Locks](./chapter8/locks.md)
-  - [Message Passing](./chapter8/message-passing.md)
-  - [Types of Parallelism](./chapter8/parallelism.md)
-  - [Challenges](./chapter8/challenges.md)
+- [Introduction to Parallel Computing](./chapter6/chapter6.md)
+  - [Multithreading](./chapter6/multithreading.md)
+  - [Synchronisation](./chapter6/synchronisation.md)
+  - [Locks](./chapter6/locks.md)
+  - [Message Passing](./chapter6/message-passing.md)
+  - [Challenges](./chapter6/challenges.md)
 
-- [Parallellisation of Algorithms](./chapter7/parallellisation-of-algorithms.md)
-  - [Distributed Databases]()
-  - [Parallel Search]()
-  - [Parallel Sort]()
-  - [Other Parallel Algorithms]()
+- [Parallellisation of Algorithms](./chapter7/chapter7.md)
+  - [Parallel Search](./chapter7/parallel-search.md)
+  - [Parallel Sort](./chapter7/parallel-sort.md)
+  - [Other Parallel Algorithms](./chapter7/other-parallel-algos.md)
   - [Machine Learning & HPC](./chapter7/machine-learning-and-hpc.md)
-  - [Optimisation Algorithms](./chapter7/optimisation-algorithms.md)
-
-- [Apache Spark](./chapter10/chapter10.md)
-  - [Installation & Cluster Set-up](./chapter10/set-up.md)
-  - [Internal Architecture](./chapter10/internals.md)
-  - [Data Processing](./chapter10/data-processing.md)
-  - [Job Batching](./chapter10/job-batching.md)
-  - [Challenges](./chapter10/challenges.md)
+  - [Optimisation Algorithms](./chapter7/optim-algos.md)
+  - [Challenges](./chapter7/challenges.md)
 
-- [Being a HPC Member](./chapter11/chapter11.md)
-
-  - [Expectations & Leadership](./chapter11/expectations-leadership.md)
-  - [Project Workflow](./chapter11/project-workflow.md)
-  - [Academic Supervisors & Papers](./chapter11/supervisors-papers.md)
+- [Apache Spark](./chapter8/chapter8.md)
+  - [Installation & Cluster Set-up](./chapter8/set-up.md)
+  - [Internal Architecture](./chapter8/internals.md)
+  - [Data Processing](./chapter8/data-processing.md)
+  - [Job Batching](./chapter8/job-batching.md)
+  - [Challenges](./chapter8/challenges.md)
 
-[Acknowledgements](./acknowledgements.md)
+[Acknowledgements](./acknowledgements.md)

src/acknowledgements.md

@@ -9,6 +9,7 @@ This book is part of Monash DeepNeurons collection of technical information and
 - [Yuki Kume](https://github.com/UnciaBit)
 - [Jaspar Martin](https://github.com/jasparm)
 - [Duc Thanh Vinh Nguyen](https://github.com/VincentNguyenDuc)
+- [Linton Charles](https://github.com/linton2000)
 
 ## Contributors
 

src/chapter3/challenges.md

@@ -1 +1,3 @@
 # Challenges
+
+![under-const](../imgs/under-const.gif)

src/chapter3/chapter3.md

@@ -4,3 +4,6 @@ A decent chunk of HPC involves using low-level tools and techniques to find opti
 
 ![comp-levels](./imgs/programming-levels.jpg)
 
+> **Note:** Not all low-level, machine (Assembly) code is faster than high-level code. The primary reason that lower level coding tends to be faster is that it avoids a lot of the overhead (eg. garbage collection) involved in executing higher level code.
+
+If you have done FIT2100 Operating Systems, this chapter would mostly be a refresher for you. It's intended to provide you with a crash course intro to operating systems theory so that you are capable of using low-level tools and implementing things like cache optimisations.

src/chapter3/computer-architecture.md

@@ -0,0 +1,66 @@
+# Computer Architecture
+
+Nearly all modern computers are based on the Von-Neumann architecture which was introduced way back in 1945. Historically there have been 2 types of Computers: 
+
+1. **Fixed Program Computers:** Their function is very specific and they couldn’t be reprogrammed, e.g. Calculators. 
+2. **Stored Program Computers:** These can be programmed to carry out many different tasks, applications are stored on them, hence the name.
+
+Modern computers are based on a stored-program concept introduced by John Von Neumann. In this stored-program concept, both software programs (code) and data (text, images, metadata, etc...) are stored in a separate storage unit called memory and are treated the same. This novel idea meant that a computer built with this architecture would be much easier to reprogram. 
+
+![basic-arch](./imgs/basic_structure.png)
+
+### Central Processing Unit (CPU)
+Hopefully everyone here knows what a CPU is and what it generally does. We've got a brief description of it's components and a lower level view of its hardware design to give you guys some relevant insight.
+
+1. **Control Unit:** A control unit directs all input and output flow, fetches code for instructions, and controls how data moves around the system. I think of it as the brains behind the CPU (the CPU of the CPU). 
+2. **Arithmetic Logic Unit (ALU):** The ALU is the CPU component that handles all the calculations the CPU may need, e.g. Addition, Subtraction, Comparisons. It performs Logical Operations, Bit Shifting Operations, and Arithmetic operations. 
+
+### Overview of Hardware Components
+You don't need to remember everything here but as we pursue more hardware-related and low-level projects (Cluster Dev, potentially something in robotics...), these concepts are hopefully a good foundation for that.
+
+![basic-hardware](./imgs/vn_cpu.png)
+
+#### Registers
+Registers refer to high-speed storage areas in the CPU. They are the most expensive but also the fastest unit of memory. The data processed by the CPU are fetched from the registers. There are different types of registers used in architecture:
+- **Accumulator:** Stores the results of calculations made by ALU. It holds the intermediate of arithmetic and logical operatoins.it act as  a temporary storage location or device.
+- **Program Counter (PC):** Keeps track of the memory location of the next instructions to be dealt with. The PC then passes this next address to the Memory Address Register (MAR). 
+- **Memory Address Register (MAR):** It stores the memory locations of instructions that need to be fetched from memory or stored in memory. 
+- **Memory Data Register (MDR):** It stores instructions fetched from memory or any data that is to be transferred to, and stored in, memory. 
+- **Current Instruction Register (CIR):** It stores the most recently fetched instructions while it is waiting to be coded and executed. 
+- **Instruction Buffer Register (IBR):** The instruction that is not to be executed immediately is placed in the instruction buffer register IBR. 
+
+#### Buses
+You can think of a bus as a group of tiny wires that carry data (64 tiny wires = 64 bit bus). Data is transmitted from one part of a computer to another, connecting all major internal components to the CPU and memory, by buses. Types: 
+- **Data Bus:** It carries data among the memory unit, the I/O devices, and the processor. 
+- **Address Bus:** It carries the address of data (not the actual data) between memory and processor. 
+- **Control Bus:** It carries control commands from the CPU (and status signals from other devices) in order to control and coordinate all the activities within the computer.
+
+#### Input/Output Devices
+Program or data is read into main memory from the input device or secondary storage under the control of CPU input instruction. Output devices are used to output information from a computer. If some results are evaluated by the computer and it is stored in the computer, then with the help of output devices, we can present them to the user.
+
+#### Von Neumann bottleneck
+Whatever we do to enhance performance, we cannot get away from the fact that instructions can only be done one at a time and can only be carried out sequentially. Both of these factors hold back the competence of the CPU. We can provide a Von Neumann processor with more cache, more RAM, or faster components but if original gains are to be made in CPU performance then a full hardware upgrade needs to happen. This is what motivates the use of parallel & distributed computing in HPC - it allows us to overcome this limitation without expensive hardware upgrades (details in later chapters).
+
+
+## Memory Heirarchy
+
+It's very important that you understand these concepts as they go to the heart of memory-based optimisations in HPC. The main purpose behind designing a memory heirarchy is to use [locality of reference](https://en.wikipedia.org/wiki/Locality_of_reference) to minimise access times as much as possible.
+
+As you can see in the image below, there are multiple levels of memory with levels closer to the top (closer to the CPU) being smaller, more expensive but much faster than the ones to the bottom.
+
+![memory-heirarchy](./imgs/Memory-Hierarchy-Design.png)
+
+We can broadly divide this heirarchy into two:
+1. **Primary Memory:** This comprises Main Memory, Cache Memory & CPU registers and it's directly accessible by the processor.
+2. **Secondary Memory:** This is handled by external devices like Magnetic Disk, Optical Disk, and Magnetic Tape i.e. peripherals which are accessible by the processor via an I/O module and device driver.
+
+Putting this heirarchy into perspective, we have some indicative stats below for each type of memory.
+
+| Level | 1 |	2 |	3 |	4 |
+| --- | --- | --- | --- | --- |
+| Name | Register |	Cache |	Main Memory | Secondary Memory |
+| Size | <1 KB | <16 MB |<16GB | >100 GB |
+| Hardware |	D-latches |	SRAM |	DRAM | Magnetic |
+| Access Time |	 0.25-0.5ns |	0.5-25ns |	80-250ns |	500000 ns |
+| Bandwidth | 20-100 GB |	5-15 GB |	1-5 GB | 20-150 MB |
+| Managed by |	Compiler |	Hardware |	Operating System | 	Operating System |

src/chapter3/linux-intro.md

@@ -0,0 +1,10 @@
+# Introduction to Linux
+
+Linux is one of the most popular versions of the UNIX operating System. It is open source as its source code is freely available. It is free to use. Linux was designed considering UNIX compatibility. Its functionality list is quite similar to that of UNIX.
+
+Linux Operating System has primarily three components:
+- **Kernel:** The kernel is the core part of Linux. It is responsible for all major activities of this operating system. It consists of various modules and it interacts directly with the underlying hardware. Kernel provides the required abstraction to hide low level hardware details to system or application programs.
+- **System Library:** System libraries are special functions or programs using which application programs or system utilities access Kernel’s features. These libraries implement most of the functionalities of the operating system and do not require kernel module’s code access rights.
+- **System Utility:** System Utility programs are responsible to do specialised, individual level tasks.
+
+![linux-struct](./imgs/Structure-Of-Linux-Operating-System.png)

src/chapter3/processes.md

@@ -0,0 +1,3 @@
+# Processes
+
+![under-const](../imgs/under-const.gif)

src/chapter3/scheduling.md

@@ -0,0 +1,3 @@
+# Scheduling Algorithms
+
+![under-const](../imgs/under-const.gif)

src/chapter3/threads-concurrency.md

@@ -1 +1,3 @@
 # Threading & Concurrency
+
+![under-const](../imgs/under-const.gif)

src/chapter4/chapter4.md

@@ -0,0 +1,3 @@
+# More C
+
+This chapter will walk you through the more intermediate features of the C language. It aims to build on the theoretical knowledge of operating systems you gained in the last chapter with the practical skills to actually use it. You will learn about other C language constructs, memory allocation, system calls (the Kernel's API) and actually spawning processes & threads.

src/chapter4/spawn-procs.md

@@ -0,0 +1,3 @@
+# Spawning Processes & Threads
+
+![under-const](../imgs/under-const.gif)

src/chapter4/syscalls.md

@@ -0,0 +1,3 @@
+# System Calls
+
+![under-const](../imgs/under-const.gif)