Skip to content

Commit

Permalink
Merge pull request #22 from MonashDeepNeuron/dev
Browse files Browse the repository at this point in the history
Version bump and minor fixes
  • Loading branch information
oraqlle committed Apr 16, 2023
2 parents a26801b + ad8fcda commit d55f4ba
Show file tree
Hide file tree
Showing 18 changed files with 13 additions and 13 deletions.
2 changes: 1 addition & 1 deletion src/chapter5/challenges.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@

Output should be similar to this. May be slightly different due to process scheduling

![Ping pong](../imgs/ping_pong.png)
![Ping pong](imgs/ping_pong.png)

## Task 3: Monte Carlo

Expand Down
6 changes: 3 additions & 3 deletions src/chapter5/distributed-computing.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@

This essentially means it is a form of parallel computing, where the processing power is spread across multiple machines in a network rather than being contained within a single system. In this memory architecture, the problems are broken down into smaller parts, and each machine is assigned to work on a specific part.

![distributed memory architecture](../imgs/distributed_memory_architecture.png)
![distributed memory architecture](imgs/distributed_memory_architecture.png)

## Distributed Memory Architecture

Expand All @@ -13,13 +13,13 @@ Lets have a look at the distributed memory architecture in more details.
- Each processor has its own local memory, with its own address space
- Data is shared via a communications network using a network protocol, e.g Transmission Control Protocol (TCP), Infiniband etc..

![Distributed Memory Architecture](../imgs/distributed_memory_architecture_2.png)
![Distributed Memory Architecture](imgs/distributed_memory_architecture_2.png)

## Distributed vs Shared program execution

The following diagram provides another way of looking at the differences between distributed and shared memory architecture and their program execution.

![Distributed vs Shared](../imgs/distributed_vs_shared.png)
![Distributed vs Shared](imgs/distributed_vs_shared.png)

## Advantages of distributed computing

Expand Down
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
6 changes: 3 additions & 3 deletions src/chapter5/openmpi.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ int MPI_Comm_rank(MPI_Comm comm, int \* rank);
// rank contains the value for that process- the function return value is an error code
```

![MPI routines](../imgs/mpi_routines.png)
![MPI routines](imgs/mpi_routines.png)

### Point-to-Point communication

Expand Down Expand Up @@ -242,7 +242,7 @@ mpirun -np 4 ./my-awesome-program
There are some useful commands to check the parallelism of the code.
The command top or htop looks into a process. As you can see from the image below, it shows the CPU usages

![htop](../imgs/htop.png)
![htop](imgs/htop.png)

- The command ```time``` checks the overall performance of the code
- By running this command, you get real time, user time and system time.
Expand All @@ -251,4 +251,4 @@ The command top or htop looks into a process. As you can see from the image belo
- Sys is the amount of CPU time spent in the kernel within the process.
- User time +Sys time will tell you how much actual CPU time your process used.

![time](../imgs/time.png)
![time](imgs/time.png)
10 changes: 5 additions & 5 deletions src/chapter5/parallel-refresher.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,28 +4,28 @@

We saw in the last chapter parallel computing can be used to solve problems by executing code in parallel as opposed to in series.

![Task parallelism](../imgs/task_parallelism.jpg)
![Task parallelism](imgs/task_parallelism.jpg)

## Data Parallelism

Note that not all programs can be broken down into independent tasks and we might instead data parallelism like the following.

![Data parallelism](../imgs/data_parallelism.jpg)
![Data parallelism](imgs/data_parallelism.jpg)

## Parallel computing example

Think back to the example below which was provided in the last chapter. We will look at the cost of memory transactions soon.

![Parallel computing example](../imgs/parallel_computing_arrays_eg.png)
![Parallel computing example](imgs/parallel_computing_arrays_eg.png)

## Parallel Scalability

The speed up achieved from parallelism is dictated by your algorithm. Notably the serial bits of your algorithm can not be sped up by increasing the number of processors. The diagram below looks at the benefits we can achieve from writing parallel code as the number of processes increases.

![Parallel scalability](../imgs/parallel_scalability.jpg)
![Parallel scalability](imgs/parallel_scalability.jpg)

## Memory Architectures

Lastly, the different memory architectures we looked at in the last section included shared memory, distributed memory and hybrid architectures. We have looked at shared memory in detail and now we will dive into distributed memory architecture.

![Memory architectures](../imgs/memory_architectures.jpg)
![Memory architectures](imgs/memory_architectures.jpg)
2 changes: 1 addition & 1 deletion src/version.md
Original file line number Diff line number Diff line change
@@ -1 +1 @@
version: 0.1.0
version: 1.0.0

0 comments on commit d55f4ba

Please sign in to comment.