From 83caf2dd85ce094674ba78de6470ae477f563706 Mon Sep 17 00:00:00 2001 From: Rares Balcan Date: Wed, 8 Oct 2025 00:17:19 +0300 Subject: [PATCH] io/optimizations/drills/tasks/async-server Update README The instructions numbering in README was only 1s, oddly looking Signed-off-by: Rares Balcan io/optimizations/drills/tasks/multiplexed-client-server Update README The instructions numbering in README was only 1s, oddly looking Signed-off-by: Rares Balcan io/optimizations Add checker for async-server Added a checker for async-server. Added `make check` rule to the Makefile. Updated README.md with checker instructions. Modified the `test-file` size that is being sent from 1GB to 100MB, so the checker can run in a shorter time. Signed-off-by: Rares Balcan io/optimizations: Add checker for async-server Added a checker for async-server. Added `make check` rule to the Makefile. Updated README.md with checker instructions. Modified the `test-file` size that is being sent from 1GB to 100MB, so the checker can run in a shorter time. Signed-off-by: Rares Balcan io/ipc: Add checker for client-server Added checker(s) for client-server, one for the client, another one for the server. Added instructions in README upon using the checker and a Makefile rule for it. Added a script for resetting connection, in case the port is blocked. Signed-off-by: Rares Balcan labs/lab-11: Add checkers and fix wording Added checkers for async-server and client-server. Added 'make check' rules to the Makefile. Updated README.md with checker and arhchive instructions. Added lab11.md in overview/reading. Signed-off-by: Rares Balcan Update chapters/io/ipc/drills/tasks/client-server/README.md Co-authored-by: Teodor Dutu Update chapters/io/optimizations/drills/tasks/async-server/tests/check_server.sh Co-authored-by: Teodor Dutu Update chapters/io/ipc/drills/tasks/client-server/tests/check_server.sh Co-authored-by: Teodor Dutu Update chapters/io/ipc/drills/tasks/client-server/tests/check_client.sh Co-authored-by: Teodor Dutu Update chapters/io/optimizations/drills/tasks/async-server/tests/check_server.sh Co-authored-by: Teodor Dutu Update chapters/io/ipc/drills/tasks/client-server/solution/Makefile Co-authored-by: Teodor Dutu Update chapters/io/optimizations/drills/tasks/async-server/solution/Makefile Co-authored-by: Teodor Dutu Update chapters/io/optimizations/drills/tasks/async-server/solution/Makefile Co-authored-by: Teodor Dutu software-stack/system-calls: Fix minor bugs in lab01 - Fix SPDX license comment to use asm comment style - Fix `syscall-wrapper/support/` directory structure - Fix no. of skipped lines when generating `main.c` Signed-off-by: Vlad Grigore assignments/elf-loader: Add ELF Loader assignment Add a new PIE statically linked ELF Loader assignment. Signed-off-by: Stefan Jumarea data/memory-security: Fix typo in `buffer-overflow-leak` guide Replace "don" with "don't". Signed-off-by: Mihnea Firoiu software-stack/high-level-languages: Fix typo in `high-level-lang` task Replace `spport/` with `support/`. Signed-off-by: Sorin Birchi data/working-with-memory: Fix inconsistencies in lab 3 - `make skels` would generate the src file which had random brace symbols scattered throughout and made compiling the code impossible. - Replaced C Syntax code (scanf, printf) with dlang equivalent (readf, write/writeln). Pass `in_bits` array by ref in the `to_bits` function so the code actually does what it is supposed to. Minor additional spacing. - Made the read-only variable `ro` a global variable instead of a local one to be added to the .rodata section. Keeping it as a local variable stopped executing but still permitted writing via `do_write()` function. Signed-off-by: dariusica2 compute/synchronization: Add link to the lab 8 archives and fix typos - Add links to the lab archive and change directory references. - Fix typo in `tls-on-demand` TODO. Signed-off-by: Vlad Hosu io/ipc: Add link to lab 10 archive and review lab 10 - Add links to the lab archive and change directory references. - Add man links and additional explanations in the text. - Correct some comments. Signed-off-by: Laura Ruse data/guides: Created dedicated guide folder for support files Created `static-dynamic` guide folder and moved assosiated files to it. Modified references to it in `.md` files. Signed-off-by: Cristian-Stefan Lazar Modified generate_skels.py from Lab2 (#195) Added go files when generating skels Signed-off-by: Sorin Birchi software-stack/libc: Fix minor bugs in lab02 - Fix SPDX license comment to use asm comment style - Remove redundant calls to 'os_string' functions in task 'libc' (probably a leftover from the other task) - Change 'putchar_buffer_len' from 'char' to 'int' to hold 1024 characters (had a student ask why the length was stored in a char, it shouldn't) Signed-off-by: Vlad Grigore Update lab archive generator workflow Remove `.zip` from gitignore, this is needed because the workflow will overwrite the changes to the archive if the file is ignored by git. Create a commit for removing the outdated archives, since `git stash pop` will generate a merge conflict otherwise. Check if the name of the task contains `drills`. If it does not, do not add it to the archive. Signed-off-by: Stefan Jumarea labs/lab6: Review and fix typos Fix #176 Signed-off-by: Mihai-Carol Bazga /data/working-with-memory: Update `static-dynamic` Makefile for proper static and dynamic linking The previous Makefile did not provide both dynamic and static executables. Signed-off-by: Oprea Stefan Antoniu data/tasks/copy: Add a rule to generate in.dat (#204) Added new `input` rule to create `in.dat` containing "Hello, world!". Updated the `all` target to include `input`, ensuring the file is generated automatically during the build. Fixes #203 Signed-off-by: Andreia Ocanoaia software-stack/drills: Remove the `support/` folder for the `libc` task. This folder is generated by `make skels`. Signed-off-by: Andrei Lungu exec-shellcode/Makefile: Fix Makefile Fix: Utils directory is from now on being copied to output. Signed-off-by: Mihnea Firoiu software-stack/system-calls: Fix typo and rephrase Fix a typo in drills/questions/syscall-numbers.md Signed-off-by: nicolasdumitru io/file-descriptors: Add testing suggestion for each task. Suggested students to test manually each task before using 'tests/checher.sh', for a better understanding of the concept. Added `lab9.md` that instructs students to download the arhive or use GitHub. Changed config.yaml to add `lab9.md` to the archive. Added guidance to tasks directories. Signed-off-by: Matei Stanuca --- .checkpatch.ignore | 1 + .github/workflows/lab-archive.yml | 24 +- .gitignore | 1 - chapters/compute/overview/reading/lab6.md | 1 + chapters/compute/overview/reading/lab8.md | 1 + .../drills/tasks/create-process/README.md | 5 +- .../processes/drills/tasks/sleepy/README.md | 5 +- .../tasks/wait-for-me-processes/README.md | 5 +- .../compute/processes/reading/processes.md | 2 +- .../apache2-simulator-condition/README.md | 6 +- .../drills/tasks/atomic-assembly/README.md | 6 +- .../tasks/race-condition-atomic/README.md | 8 +- .../drills/tasks/race-condition/README.md | 2 +- .../tasks/threadsafe-data-struct/README.md | 2 +- .../drills/tasks/tls-on-demand/README.md | 2 +- .../solution/src/race_condition_tls.c | 2 +- .../drills/tasks/wrap-the-for/README.md | 2 +- .../drills/tasks/multithreaded/README.md | 7 +- .../drills/tasks/sum-array-bugs/README.md | 4 +- .../threads/drills/tasks/sum-array/README.md | 20 +- .../guides/sum-array-threads/README.md | 18 +- .../guides/wait-for-me-threads/README.md | 8 +- .../drills/tasks/exec-shellcode/Makefile | 5 +- .../guides/buffer-overflow-leak/README.md | 2 +- .../drills/tasks/copy/solution/src/Makefile | 8 +- .../tasks/static-dynamic/support/Makefile | 43 -- .../process-memory/reading/process-memory.md | 2 +- .../drills/tasks/memory-access/README.md | 7 +- .../drills/tasks/memory-corruption/README.md | 2 +- .../solution/src/c_segfault.c | 24 +- .../solution/src/d_segfault.d | 22 +- .../memory-protection/solution/src/mem_prot.c | 6 +- .../guides}/static-dynamic/.gitignore | 0 .../guides/static-dynamic/support/Makefile | 25 ++ .../guides}/static-dynamic/support/hello.c | 0 .../support/utils/log/CPPLINT.cfg | 0 .../static-dynamic/support/utils/log/log.c | 0 .../static-dynamic/support/utils/log/log.h | 0 .../static-dynamic/support/utils/utils.h | 0 .../drills/tasks/buffering/README.md | 3 + .../drills/tasks/mmap_cp/README.md | 9 +- .../drills/tasks/my-cat/README.md | 9 +- .../io/ipc/drills/tasks/anon-pipes/README.md | 9 +- .../ipc/drills/tasks/client-server/README.md | 5 +- .../tasks/client-server/solution/Makefile | 4 + .../tasks/client-server/tests/check_client.sh | 83 ++++ .../tasks/client-server/tests/check_server.sh | 78 ++++ .../client-server/tests/input_client.txt | 4 + .../client-server/tests/input_server.txt | 3 + .../client-server/tests/output_client.txt | 3 + .../client-server/tests/output_server.txt | 3 + .../client-server/tests/reset_connection.sh | 15 + .../io/ipc/drills/tasks/named-pipes/README.md | 9 +- .../named-pipes/solution/src/named_pipe.c | 8 +- .../ipc/drills/tasks/network-socket/README.md | 13 +- .../network-socket/solution/src/tcp_socket.c | 6 +- .../network-socket/solution/src/udp_socket.c | 4 +- .../drills/tasks/receive-challenges/README.md | 3 +- .../solution/src/receive_fifo.c | 6 +- .../solution/src/receive_net_dgram_socket.c | 4 +- .../solution/src/receive_pipe.c | 2 +- .../solution/src/receive_unix_socket.c | 2 +- .../solution/src/send_fifo.c | 6 +- .../solution/src/send_net_dgram_socket.c | 4 +- .../solution/src/send_unix_socket.c | 6 +- .../io/ipc/drills/tasks/unix-socket/README.md | 10 +- .../unix-socket/solution/src/unix_socket.c | 8 +- .../drills/tasks/async-server/README.md | 10 +- .../tasks/async-server/solution/Makefile | 7 +- .../tasks/async-server/solution/client.c | 8 +- .../tasks/async-server/solution/server.c | 10 +- .../tasks/async-server/tests/check_server.sh | 90 +++++ .../tasks/multiplexed-client-server/README.md | 2 +- chapters/io/overview/reading/lab10.md | 1 + chapters/io/overview/reading/lab9.md | 1 + chapters/io/overview/reading/overview.md | 2 + .../drills/tasks/high-level-lang/README.md | 2 +- .../tasks/high-level-lang/generate_skels.py | 1 + .../solution/src/main_printf.c | 2 +- .../common-functions/solution/src/syscall.s | 2 +- .../drills/tasks/libc/solution/main_printf.c | 10 +- .../libc/drills/tasks/libc/support/Makefile | 23 -- .../libc/drills/tasks/libc/support/hello.c | 20 - .../drills/tasks/libc/support/main_printf.c | 26 -- .../drills/tasks/libc/support/main_string.c | 17 - .../libc/drills/tasks/libc/support/memory.c | 17 - .../libc/drills/tasks/libc/support/vendetta.c | 27 -- .../drills/questions/syscall-numbers.md | 2 +- .../basic-syscall/solution/src/arm/hello.s | 2 +- .../basic-syscall/solution/src/hello.asm | 2 +- .../tasks/basic-syscall/solution/src/hello.s | 2 +- .../drills/tasks/syscall-wrapper/Makefile | 2 +- .../tasks/syscall-wrapper/solution/src/main.c | 4 +- .../syscall-wrapper/solution/src/syscall.asm | 2 +- config.yaml | 5 + content/assignments/elf-loader/README.md | 368 ++++++++++++++++++ .../elf-loader/img/auxv-example.drawio.svg | 4 + .../elf-loader/img/auxv.drawio.svg | 4 + .../elf-loader/img/stack-layout.drawio.svg | 4 + content/assignments/elf-loader/src/.gitignore | 0 content/assignments/elf-loader/src/Makefile | 9 + .../assignments/elf-loader/src/elf-loader.c | 102 +++++ .../assignments/elf-loader/tests/.gitignore | 0 content/assignments/elf-loader/tests/Makefile | 41 ++ content/assignments/elf-loader/tests/grade.sh | 136 +++++++ .../assignments/elf-loader/tests/ref/envp.ref | 0 .../elf-loader/tests/ref/error-bad-magic.ref | 0 .../elf-loader/tests/ref/error-not-64.ref | 0 .../elf-loader/tests/ref/no_pie.ref | 0 .../elf-loader/tests/ref/no_pie_argc.ref | 1 + .../elf-loader/tests/ref/no_pie_argv.ref | 4 + .../elf-loader/tests/ref/no_pie_auxv.ref | 1 + .../elf-loader/tests/ref/no_pie_envp.ref | 1 + .../elf-loader/tests/ref/no_pie_hello.ref | 1 + .../elf-loader/tests/ref/nolibc.ref | 2 + .../tests/ref/nolibc_no_rwx_rodata.ref | 1 + .../tests/ref/nolibc_no_rwx_text.ref | 1 + .../assignments/elf-loader/tests/ref/pie.ref | 1 + .../assignments/elf-loader/tests/run_tests.sh | 106 +++++ .../elf-loader/tests/snippets/Makefile | 46 +++ .../elf-loader/tests/snippets/argc.c | 16 + .../elf-loader/tests/snippets/argv.c | 18 + .../elf-loader/tests/snippets/auxv.c | 18 + .../elf-loader/tests/snippets/envp.c | 21 + .../elf-loader/tests/snippets/error-bad-magic | Bin 0 -> 9040 bytes .../elf-loader/tests/snippets/error-not-64 | Bin 0 -> 15428 bytes .../elf-loader/tests/snippets/hello.c | 16 + .../elf-loader/tests/snippets/nolibc.asm | 31 ++ .../tests/snippets/nolibc_no_rwx_rodata.asm | 24 ++ .../tests/snippets/nolibc_no_rwx_text.asm | 24 ++ gen-zip.py | 2 +- 131 files changed, 1550 insertions(+), 349 deletions(-) create mode 100644 chapters/compute/overview/reading/lab6.md create mode 100644 chapters/compute/overview/reading/lab8.md delete mode 100644 chapters/data/process-memory/drills/tasks/static-dynamic/support/Makefile rename chapters/data/{process-memory/drills/tasks => working-with-memory/guides}/static-dynamic/.gitignore (100%) create mode 100644 chapters/data/working-with-memory/guides/static-dynamic/support/Makefile rename chapters/data/{process-memory/drills/tasks => working-with-memory/guides}/static-dynamic/support/hello.c (100%) rename chapters/data/{process-memory/drills/tasks => working-with-memory/guides}/static-dynamic/support/utils/log/CPPLINT.cfg (100%) rename chapters/data/{process-memory/drills/tasks => working-with-memory/guides}/static-dynamic/support/utils/log/log.c (100%) rename chapters/data/{process-memory/drills/tasks => working-with-memory/guides}/static-dynamic/support/utils/log/log.h (100%) rename chapters/data/{process-memory/drills/tasks => working-with-memory/guides}/static-dynamic/support/utils/utils.h (100%) create mode 100755 chapters/io/ipc/drills/tasks/client-server/tests/check_client.sh create mode 100755 chapters/io/ipc/drills/tasks/client-server/tests/check_server.sh create mode 100644 chapters/io/ipc/drills/tasks/client-server/tests/input_client.txt create mode 100644 chapters/io/ipc/drills/tasks/client-server/tests/input_server.txt create mode 100644 chapters/io/ipc/drills/tasks/client-server/tests/output_client.txt create mode 100644 chapters/io/ipc/drills/tasks/client-server/tests/output_server.txt create mode 100755 chapters/io/ipc/drills/tasks/client-server/tests/reset_connection.sh create mode 100755 chapters/io/optimizations/drills/tasks/async-server/tests/check_server.sh create mode 100644 chapters/io/overview/reading/lab10.md create mode 100644 chapters/io/overview/reading/lab9.md delete mode 100644 chapters/software-stack/libc/drills/tasks/libc/support/Makefile delete mode 100644 chapters/software-stack/libc/drills/tasks/libc/support/hello.c delete mode 100644 chapters/software-stack/libc/drills/tasks/libc/support/main_printf.c delete mode 100644 chapters/software-stack/libc/drills/tasks/libc/support/main_string.c delete mode 100644 chapters/software-stack/libc/drills/tasks/libc/support/memory.c delete mode 100644 chapters/software-stack/libc/drills/tasks/libc/support/vendetta.c create mode 100644 content/assignments/elf-loader/README.md create mode 100644 content/assignments/elf-loader/img/auxv-example.drawio.svg create mode 100644 content/assignments/elf-loader/img/auxv.drawio.svg create mode 100644 content/assignments/elf-loader/img/stack-layout.drawio.svg create mode 100644 content/assignments/elf-loader/src/.gitignore create mode 100644 content/assignments/elf-loader/src/Makefile create mode 100644 content/assignments/elf-loader/src/elf-loader.c create mode 100644 content/assignments/elf-loader/tests/.gitignore create mode 100644 content/assignments/elf-loader/tests/Makefile create mode 100755 content/assignments/elf-loader/tests/grade.sh create mode 100644 content/assignments/elf-loader/tests/ref/envp.ref create mode 100644 content/assignments/elf-loader/tests/ref/error-bad-magic.ref create mode 100644 content/assignments/elf-loader/tests/ref/error-not-64.ref create mode 100644 content/assignments/elf-loader/tests/ref/no_pie.ref create mode 100644 content/assignments/elf-loader/tests/ref/no_pie_argc.ref create mode 100644 content/assignments/elf-loader/tests/ref/no_pie_argv.ref create mode 100644 content/assignments/elf-loader/tests/ref/no_pie_auxv.ref create mode 100644 content/assignments/elf-loader/tests/ref/no_pie_envp.ref create mode 100644 content/assignments/elf-loader/tests/ref/no_pie_hello.ref create mode 100644 content/assignments/elf-loader/tests/ref/nolibc.ref create mode 100644 content/assignments/elf-loader/tests/ref/nolibc_no_rwx_rodata.ref create mode 100644 content/assignments/elf-loader/tests/ref/nolibc_no_rwx_text.ref create mode 100644 content/assignments/elf-loader/tests/ref/pie.ref create mode 100755 content/assignments/elf-loader/tests/run_tests.sh create mode 100644 content/assignments/elf-loader/tests/snippets/Makefile create mode 100644 content/assignments/elf-loader/tests/snippets/argc.c create mode 100644 content/assignments/elf-loader/tests/snippets/argv.c create mode 100644 content/assignments/elf-loader/tests/snippets/auxv.c create mode 100644 content/assignments/elf-loader/tests/snippets/envp.c create mode 100755 content/assignments/elf-loader/tests/snippets/error-bad-magic create mode 100755 content/assignments/elf-loader/tests/snippets/error-not-64 create mode 100644 content/assignments/elf-loader/tests/snippets/hello.c create mode 100644 content/assignments/elf-loader/tests/snippets/nolibc.asm create mode 100644 content/assignments/elf-loader/tests/snippets/nolibc_no_rwx_rodata.asm create mode 100644 content/assignments/elf-loader/tests/snippets/nolibc_no_rwx_text.asm diff --git a/.checkpatch.ignore b/.checkpatch.ignore index 541c6ba0cb..97660c8b8a 100644 --- a/.checkpatch.ignore +++ b/.checkpatch.ignore @@ -1,3 +1,4 @@ # Ignore directories containing third-party files chapters/compute/synchronization/drills/tasks/threadsafe-data-struct/support content/assignments/async-web-server/src/http-parser +content/assignments/elf-loader/tests diff --git a/.github/workflows/lab-archive.yml b/.github/workflows/lab-archive.yml index d29aeb2bf0..77e5a151ae 100644 --- a/.github/workflows/lab-archive.yml +++ b/.github/workflows/lab-archive.yml @@ -37,6 +37,13 @@ jobs: exit 0 fi + git config user.name "github-actions[bot]" + git config user.email "github-actions[bot]@users.noreply.github.com" + + ls -A *.zip > zip-list + git add *.zip + git stash + # Create or switch to lab-archives branch if git ls-remote --exit-code origin lab-archives; then git fetch origin lab-archives @@ -46,15 +53,12 @@ jobs: git rm -rf . fi + # Remove old archives + for f in $(cat zip-list); do rm "$f"; git rm "$f"; done + git commit -m "Remove outdated lab archives for commit $GITHUB_SHA" + # Copy new zips into branch root + git stash pop git add *.zip - - # Only commit if there are changes - if ! git diff --cached --quiet; then - git config user.name "github-actions[bot]" - git config user.email "github-actions[bot]@users.noreply.github.com" - git commit -m "Update lab archives for commit $GITHUB_SHA" - git push origin lab-archives - else - echo "No changes to commit." - fi + git commit -m "Update lab archives for commit $GITHUB_SHA" + git push origin lab-archives diff --git a/.gitignore b/.gitignore index bbb7ac1930..0ad012c3d1 100644 --- a/.gitignore +++ b/.gitignore @@ -15,7 +15,6 @@ *.war *.nar *.ear -*.zip *.tar.gz *.rar diff --git a/chapters/compute/overview/reading/lab6.md b/chapters/compute/overview/reading/lab6.md new file mode 100644 index 0000000000..d80655c363 --- /dev/null +++ b/chapters/compute/overview/reading/lab6.md @@ -0,0 +1 @@ +The contents of the lab are located in the [lab archive](https://github.com/cs-pub-ro/operating-systems/raw/refs/heads/lab-archives/Lab_6_Multiprocess_and_Multithread.zip) and in the [GitHub repository](https://github.com/cs-pub-ro/operating-systems). diff --git a/chapters/compute/overview/reading/lab8.md b/chapters/compute/overview/reading/lab8.md new file mode 100644 index 0000000000..c062f70a76 --- /dev/null +++ b/chapters/compute/overview/reading/lab8.md @@ -0,0 +1 @@ +The contents of the lab are located in the [lab archive](https://github.com/cs-pub-ro/operating-systems/raw/refs/heads/lab-archives/Lab_8_Synchronization.zip) and in the [GitHub repository](https://github.com/cs-pub-ro/operating-systems). diff --git a/chapters/compute/processes/drills/tasks/create-process/README.md b/chapters/compute/processes/drills/tasks/create-process/README.md index 4493a8cf37..5ad16facae 100644 --- a/chapters/compute/processes/drills/tasks/create-process/README.md +++ b/chapters/compute/processes/drills/tasks/create-process/README.md @@ -1,11 +1,12 @@ # Create Process -Enter the `chapters/compute/processes/drills/tasks/create-process/` directory, run `make skels`, open the `support/src` folder and go through the practice items below. +Enter the `create-process/` directory (or `chapters/compute/processes/drills/tasks/create-process/` if you are working directly in the repository). +Run `make` and then enter `support/` folder and go through the practice items below. Use the `tests/checker.sh` script to check your solutions. ```bash -./checker.sh +./tests/checker.sh exit_code22 ...................... passed ... 50 second_fork ...................... passed ... 50 100 / 100 diff --git a/chapters/compute/processes/drills/tasks/sleepy/README.md b/chapters/compute/processes/drills/tasks/sleepy/README.md index 50336f306b..041550f0b5 100644 --- a/chapters/compute/processes/drills/tasks/sleepy/README.md +++ b/chapters/compute/processes/drills/tasks/sleepy/README.md @@ -2,12 +2,13 @@ ## Higher level - Python -Enter the `chapters/compute/processes/drills/tasks/sleepy` directory, run `make skels`, open the `support/src` folder and go through the practice items below. +Enter the `sleepy/` directory (or `chapters/compute/processes/drills/tasks/sleepy` if you are working directly in the repository). +Run `make` and then enter `support/` folder and go through the practice items below. Use the `tests/checker.sh` script to check your solutions. ```bash -./checker.sh +./tests/checker.sh sleepy_creator ...................... passed ... 30 sleepy_creator_wait ................. passed ... 30 sleepy_creator_c .................... passed ... 40 diff --git a/chapters/compute/processes/drills/tasks/wait-for-me-processes/README.md b/chapters/compute/processes/drills/tasks/wait-for-me-processes/README.md index b57238fd78..1a1e90a643 100644 --- a/chapters/compute/processes/drills/tasks/wait-for-me-processes/README.md +++ b/chapters/compute/processes/drills/tasks/wait-for-me-processes/README.md @@ -1,6 +1,7 @@ # Wait for Me -Enter the `chapters/compute/processes/drills/tasks/wait-for-me-processes/` directory, run `make skels`, open the `support/src` folder and go through the practice items below. +Enter the `wait-for-me/` directory (or `chapters/compute/processes/drills/tasks/wait-for-me/` if you are working directly in the repository). +Run `make` and then enter `support/` folder and go through the practice items below. Use the `tests/checker.sh` script to check your solutions. @@ -10,7 +11,7 @@ wait_for_me_processes ...................... passed ... 100 ``` 1. Run the code in `wait_for_me_processes.py` (e.g: `python3 wait_for_me_processes.py`). - The parent process creates one child that writes and message to the given file. + The parent process creates one child that writes a message to the given file. Then the parent reads that message. Simple enough, right? But running the code raises a `FileNotFoundError`. diff --git a/chapters/compute/processes/reading/processes.md b/chapters/compute/processes/reading/processes.md index 4a42cbc4c4..703cba3051 100644 --- a/chapters/compute/processes/reading/processes.md +++ b/chapters/compute/processes/reading/processes.md @@ -14,7 +14,7 @@ student@os:~$ file /usr/bin/ls ``` When you run it, the `ls` binary stored **on the disk** at `/usr/bin/ls` is read by another application called the **loader**. -The loader spawns a **process** by copying some of the contents `/usr/bin/ls` in memory (such as the `.text`, `.rodata` and `.data` sections). +The loader spawns a **process** by copying some contents of `/usr/bin/ls` into memory (for example the `.text`, `.rodata` and `.data` sections). Using `strace`, we can see the [`execve`](https://man7.org/linux/man-pages/man2/execve.2.html) system call: ```console diff --git a/chapters/compute/synchronization/drills/tasks/apache2-simulator-condition/README.md b/chapters/compute/synchronization/drills/tasks/apache2-simulator-condition/README.md index 9502cad7f0..d15a8babae 100644 --- a/chapters/compute/synchronization/drills/tasks/apache2-simulator-condition/README.md +++ b/chapters/compute/synchronization/drills/tasks/apache2-simulator-condition/README.md @@ -22,8 +22,8 @@ There are a few rules though, such as: - The consumer must not retrieve data if the buffer is empty. - The producer and the consumer can't access the shared buffer at the same time. -Now enter `chapters/compute/synchronization/drills/tasks/apache2-simulator-condition/` and run `make skels`. -Look at the code in `chapters/compute/synchronization/drills/tasks/apache2-simulator/support/src/producer_consumer.py`. +Now enter the `apache2-simulator-condition/` directory of the extracted archive (or `chapters/compute/synchronization/drills/tasks/apache2-simulator-condition/` if you are working directly in the repository) and run `make skels`. +Look at the code in `support/src/producer_consumer.py`. We have one producer and one consumer for simplicity. Observe that the producer calls `notify()` once there is data available, and the consumer calls `notify()`, when data is read. Notice that this call is preceded by an `acquire()` call, and succeeded by a `release()` call. @@ -58,7 +58,7 @@ Neat! So now we have both synchronization **and** signalling. This is what conditions are for, ultimately. -Open `chapters/compute/synchronization/drills/tasks/apache2-simulator/support/src/apache2_simulator_condition.py` and follow the TODOs. +Open the `apache2-simulator-condition/` directory of the extracted archive (or `chapters/compute/synchronization/drills/tasks/apache2-simulator-condition/` if you are working directly in the repository), then go to `support/src/apache2_simulator_condition.py` and follow the TODOs. The code is similar to `apache2_simulator_semaphore.py`, but this time we use condition variables as shown in `producer_consumer.py`. [Quiz](../../../drills/questions/notify-only-with-mutex.md) diff --git a/chapters/compute/synchronization/drills/tasks/atomic-assembly/README.md b/chapters/compute/synchronization/drills/tasks/atomic-assembly/README.md index ba2b9718c4..d1dbd61cf4 100644 --- a/chapters/compute/synchronization/drills/tasks/atomic-assembly/README.md +++ b/chapters/compute/synchronization/drills/tasks/atomic-assembly/README.md @@ -11,14 +11,14 @@ It is not an instruction with its own separate opcode, but a prefix that slightl For example, we cannot place it before a `mov` instruction, as the action of a `mov` is simply `read` or `write`. Instead, we can place it in front of an `inc` instruction if its operand is memory. -Go in `chapters/compute/synchronization/drills/tasks/atomic-assembly/` and run: +Go in the `atomic-assembly/` directory of the extracted archive (or `chapters/compute/synchronization/drills/tasks/atomic-assembly/` if you are working directly in the repository) and run: ```bash make skels ``` -Look at the code in `chapters/compute/synchronization/drills/tasks/atomic-assembly/support/src/race_condition_lock.asm`. -It's an Assembly equivalent of the code you've already seen many times so far (such as `chapters/compute/synchronization/drills/tasks/race-condition/support/c/race_condition.c`). +Look at the code in `support/src/race_condition_lock.asm`. +It's an Assembly equivalent of the code you've already seen many times so far (such as `race-condition/support/c/race_condition.c`). The 2 assembly functions (**increment_var** and **decrement_var**) are called by `race_condition_lock_checker.c` Now add the `lock` prefix before `dec`. diff --git a/chapters/compute/synchronization/drills/tasks/race-condition-atomic/README.md b/chapters/compute/synchronization/drills/tasks/race-condition-atomic/README.md index cf47bbe758..5c29fbd0ba 100644 --- a/chapters/compute/synchronization/drills/tasks/race-condition-atomic/README.md +++ b/chapters/compute/synchronization/drills/tasks/race-condition-atomic/README.md @@ -13,7 +13,7 @@ Modern processors are capable of _atomically_ accessing data, either for reads o An atomic action is an indivisible sequence of operations that a thread runs without interference from others. Concretely, before initiating an atomic transfer on one of its data buses, the CPU first makes sure all other transfers have ended, then **locks** the data bus by stalling all cores attempting to transfer data on it. This way, one thread obtains **exclusive** access to the data bus while accessing data. -As a side note, the critical sections in `chapters/compute/synchronization/drills/tasks/race-condition/support/c/race_condition_mutex.c` are also atomic once they are wrapped between calls to `pthread_mutex_lock()` and `pthread_mutex_unlock()`. +As a side note, the critical sections in `race-condition/support/c/race_condition_mutex.c` are also atomic once they are wrapped between calls to `pthread_mutex_lock()` and `pthread_mutex_unlock()`. As with every hardware feature, the `x86` ISA exposes an instruction for atomic operations. In particular, this instruction is a **prefix**, called `lock`. @@ -27,19 +27,19 @@ Compilers provide support for such hardware-level atomic operations. GCC exposes [built-ins](https://gcc.gnu.org/onlinedocs/gcc/_005f_005fatomic-Builtins.html) such as `__atomic_load()`, `__atomic_store()`, `__atomic_compare_exchange()` and many others. All of them rely on the mechanism described above. -Go to `chapters/compute/synchronization/drills/tasks/race-condition-atomic/` and run: +Go to the `race-condition-atomic/` directory of the extracted archive (or `chapters/compute/synchronization/drills/tasks/race-condition-atomic/` if you are working directly in the repository) and run: ```bash make skels ``` -Now enter `chapters/compute/synchronization/drills/tasks/race-condition-atomic/support/src/race_condition_atomic.c` and complete the function `decrement_var()`. +Now enter `support/src/race_condition_atomic.c` and complete the function `decrement_var()`. Compile and run the code. Its running time should be somewhere between `race_condition` and `race_condition_mutex`. The C standard library also provides atomic data types. Access to these variables can be done only by one thread at a time. -Go to `chapters/compute/synchronization/drills/tasks/race-condition-atomic/support/race_condition_atomic2.c`, compile and run the code. +Go to `support/src/race_condition_atomic2.c`, compile and run the code. After both tasks are done, go in the checker folder and run it using the following commands: diff --git a/chapters/compute/synchronization/drills/tasks/race-condition/README.md b/chapters/compute/synchronization/drills/tasks/race-condition/README.md index 638225fc08..3bc947577a 100644 --- a/chapters/compute/synchronization/drills/tasks/race-condition/README.md +++ b/chapters/compute/synchronization/drills/tasks/race-condition/README.md @@ -1,6 +1,6 @@ # C: Race Conditions -Go to `chapters/compute/synchronization/drills/tasks/race-condition/support/c/race_condition_mutex.c` and notice the differences between this code and the buggy one. +Go to the `race-condition/` directory of the extracted archive (or `chapters/compute/synchronization/drills/tasks/race-condition/` if you are working directly in the repository), then open `support/c/race_condition_mutex.c` and notice the differences between this code and the buggy one. We now use a `pthread_mutex_t` variable, which we `lock` at the beginning of a critical section, and we `unlock` at the end. Generally speaking, `lock`-ing a mutex makes a thread enter a critical section, while calling `pthread_mutex_unlock()` makes the thread leave said critical section. Therefore, as we said previously, the critical sections in our code are `var--` and `var++`. diff --git a/chapters/compute/synchronization/drills/tasks/threadsafe-data-struct/README.md b/chapters/compute/synchronization/drills/tasks/threadsafe-data-struct/README.md index abfc6058c9..e2bbbe2d50 100644 --- a/chapters/compute/synchronization/drills/tasks/threadsafe-data-struct/README.md +++ b/chapters/compute/synchronization/drills/tasks/threadsafe-data-struct/README.md @@ -1,7 +1,7 @@ # Synchronization - Thread-Safe Data Structure Now it's time for a fully practical exercise. -Go to `chapters/compute/synchronization/drills/tasks/threadsafe-data-struct/support/`. +Go to the `threadsafe-data-struct/` directory of the extracted archive (or `chapters/compute/synchronization/drills/tasks/threadsafe-data-struct/` if you are working directly in the repository), then open the `support/` folder. In the file `clist.c` you'll find a simple implementation of an array list. Although correct, it is not (yet) thread-safe. diff --git a/chapters/compute/synchronization/drills/tasks/tls-on-demand/README.md b/chapters/compute/synchronization/drills/tasks/tls-on-demand/README.md index f86a9bd41e..94e5c483bc 100644 --- a/chapters/compute/synchronization/drills/tasks/tls-on-demand/README.md +++ b/chapters/compute/synchronization/drills/tasks/tls-on-demand/README.md @@ -4,7 +4,7 @@ The perspective of C towards TLS is the following: everything is shared by defau This makes multithreading easier and more lightweight to implement than in other languages, like D, because synchronization is left entirely up to the developer, at the cost of potential unsafety. Of course, we can specify that some data belongs to the TLS, by preceding the declaration of a variable with `__thread` keyword. -Enter `chapters/compute/synchronization/drills/tasks/tls-on-demand/` and run `make skels`. +Enter the `tls-on-demand/` directory of the extracted archive (or `chapters/compute/synchronization/drills/tasks/tls-on-demand/` if you are working directly in the repository) and run `make skels`. Now enter `support/src` and follow the TODOs. 1. Create the declaration of `var` and add the `__thread` keyword to place the variable in the TLS of each thread. diff --git a/chapters/compute/synchronization/drills/tasks/tls-on-demand/solution/src/race_condition_tls.c b/chapters/compute/synchronization/drills/tasks/tls-on-demand/solution/src/race_condition_tls.c index ff9de25367..e761152dd0 100644 --- a/chapters/compute/synchronization/drills/tasks/tls-on-demand/solution/src/race_condition_tls.c +++ b/chapters/compute/synchronization/drills/tasks/tls-on-demand/solution/src/race_condition_tls.c @@ -36,7 +36,7 @@ void *decrement_var(void *arg) var--; /** - * Print the value of `var` after it's incremented. Also print + * Print the value of `var` after it's decremented. Also print * the ID of the thread. Use `pthread_self()` to get it. */ /* TODO 1: */ diff --git a/chapters/compute/synchronization/drills/tasks/wrap-the-for/README.md b/chapters/compute/synchronization/drills/tasks/wrap-the-for/README.md index 4bd0445928..831ad44d8c 100644 --- a/chapters/compute/synchronization/drills/tasks/wrap-the-for/README.md +++ b/chapters/compute/synchronization/drills/tasks/wrap-the-for/README.md @@ -1,6 +1,6 @@ # Wrap the Whole `for` Statements in Critical Sections -Navigate to the `chapters/compute/synchronization/drills/tasks/wrap-the-for/` directory, run `make skels` and open the `support/src` directory. +Navigate to the `wrap-the-for/` directory of the extracted archive (or `chapters/compute/synchronization/drills/tasks/wrap-the-for/` if you are working directly in the repository), run `make skels`, and open the `support/src` directory. Here you will find two source files: diff --git a/chapters/compute/threads/drills/tasks/multithreaded/README.md b/chapters/compute/threads/drills/tasks/multithreaded/README.md index 576da7cb57..244adee50c 100644 --- a/chapters/compute/threads/drills/tasks/multithreaded/README.md +++ b/chapters/compute/threads/drills/tasks/multithreaded/README.md @@ -1,8 +1,9 @@ # Multithreaded -Enter the `chapters/compute/threads/drills/tasks/multithreaded/` folder, run `make skels`, and go through the practice items below in the `support/` directory. +Enter the `multithreaded/` directory (or `chapters/compute/threads/drills/tasks/multithreaded/` if you are working directly in the repository). +Run `make` and then enter `support/` folder and go through the practice items below. -1. Use the Makefile to compile `multithread.c`, run it and follow the instructions. +1. Use the Makefile to compile `multithreaded.c`, run it and follow the instructions. The aim of this task is to familiarize you with the `pthreads` library. In order to use it, you have to add `#include ` in `multithreaded.c` and `-lpthread` in the compiler options. @@ -15,7 +16,7 @@ Enter the `chapters/compute/threads/drills/tasks/multithreaded/` folder, run `ma Create a new function `sleep_wrapper2()` identical to `sleep_wrapper()` to organize your work. So far, the `data` argument is unused (mind the `__unused` attribute), so that is your starting point. - You cannot change `sleep_wrapper2()` definition, since `pthreads_create()` expects a pointer to a function that receives a `void *` argument. + You must keep `sleep_wrapper2()`'s signature unchanged because `pthread_create()` requires a function of type `void *(*)(void *)`. What you can and should do is to pass a pointer to a `int` as argument, and then cast `data` to `int *` inside `sleep_wrapper2()`. **Note:** Do not simply pass `&i` as argument to the function. diff --git a/chapters/compute/threads/drills/tasks/sum-array-bugs/README.md b/chapters/compute/threads/drills/tasks/sum-array-bugs/README.md index 13d4ac3c2f..23c4cf6316 100644 --- a/chapters/compute/threads/drills/tasks/sum-array-bugs/README.md +++ b/chapters/compute/threads/drills/tasks/sum-array-bugs/README.md @@ -1,7 +1,7 @@ # Wait for It The process that spawns all the others and subsequently calls `waitpid` to wait for them to finish can also get their return codes. -Update the code in `chapters/compute/threads/drills/tasks/sum-array-bugs/support/seg-fault/sum_array_processes.c` and modify the call to `waitpid` to obtain and investigate this return code. +Update the code in `sum-array-bugs/support/seg-fault/sum_array_processes.c` (or `chapters/compute/threads/drills/tasks/sum-array-bugs/support/seg-fault/sum_array_processes.c` if you are working directly in the repository) and modify the call to `waitpid` to obtain and investigate this return code. Display an appropriate message if one of the child processes returns an error. Remember to use the appropriate [macros](https://linux.die.net/man/2/waitpid) for handling the `status` variable that is modified by `waitpid()`, as it is a bit-field. @@ -19,7 +19,7 @@ Thus, an application that uses processes can be more robust to errors than if it ## Memory Corruption Because they share the same address space, threads run the risk of corrupting each other's data. -Take a look at the code in `sum-array-bugs/support/memory-corruption/python/`. +Take a look at the code in `sum-array-bugs/support/memory-corruption/python/` (or `chapters/compute/threads/drills/tasks/sum-array-bugs/support/memory-corruption/python/` if you are working directly in the repository). The two programs only differ in how they spread their workload. One uses threads while the other uses processes. diff --git a/chapters/compute/threads/drills/tasks/sum-array/README.md b/chapters/compute/threads/drills/tasks/sum-array/README.md index 4d29bc664e..1626ee5004 100644 --- a/chapters/compute/threads/drills/tasks/sum-array/README.md +++ b/chapters/compute/threads/drills/tasks/sum-array/README.md @@ -1,6 +1,7 @@ # Libraries for Parallel Processing -In `chapters/compute/threads/drills/tasks/sum-array/support/c/sum_array_threads.c` we spawned threads "manually" by using the `pthread_create()` function. +Enter the `sum-array/` directory (or `chapters/compute/threads/drills/tasks/sum-array/` if you are working directly in the repository). +In `./support/c/sum_array_threads.c` we spawned threads "manually" by using the `pthread_create()` function. This is **not** a syscall, but a wrapper over the common syscall used by both `fork()` (which is also not a syscall) and `pthread_create()`. Still, `pthread_create()` is not yet a syscall. @@ -10,15 +11,12 @@ Most programming languages provide a more advanced API for handling parallel com ## Array Sum in Python -Let's first probe this by implementing two parallel versions of the code in `sum-array/support/python/sum_array_sequential.py`. -One version should use threads and the other should use processes. -Run each of them using 1, 2, 4, and 8 threads / processes respectively and compare the running times. -Notice that the running times of the multithreaded implementation do not decrease. -This is because the GIL makes it so that those threads that you create essentially run sequentially. +First, let's navigate to the `sum-array/` directory (or `chapters/compute/threads/drills/tasks/sum-array/` if you are working directly in the repository). +Let's explore this by implementing two parallel versions of the sequential script located at `./support/python/sum_array_sequential.py`. +Create one version that uses threads and another that uses processes. -The GIL also makes it so that individual Python instructions are atomic. -Run the code in `chapters/compute/synchronization/drills/tasks/race-condition/support/python/race_condition.py`. -Every time, `var` will be 0 because the GIL doesn't allow the two threads to run in parallel and reach the critical section at the same time. -This means that the instructions `var += 1` and `var -= 1` become atomic. +After implementing them, run each version using 1, 2, 4, and 8 workers for both threads and processes and compare their execution times. -If you're having difficulties solving this exercise, go through [this](../../../guides/sum-array-threads.md) reading material. +You will likely notice that the running time of the multi-threaded implementation does not decrease as you add more threads. +This is due to CPython's Global Interpreter Lock (GIL), which prevents multiple native threads from executing Python bytecode at the same time. +For this reason, CPU-bound tasks in Python do not typically see a performance increase from multi-threading. diff --git a/chapters/compute/threads/guides/sum-array-threads/README.md b/chapters/compute/threads/guides/sum-array-threads/README.md index 4665f4d5c7..d7095ea2b2 100644 --- a/chapters/compute/threads/guides/sum-array-threads/README.md +++ b/chapters/compute/threads/guides/sum-array-threads/README.md @@ -1,4 +1,4 @@ -# Sum array Threads +# Sum Array Threads ## Spreading the Work Among Other Threads @@ -12,14 +12,14 @@ Therefore, they are more lightweight than processes. ## `std.parallelism` in D -D language's standard library exposes the [`std.parallelism`](https://dlang.org/phobos/std_parallelism.html), which provides a series of parallel processing functions. +The D language's standard library exposes the [`std.parallelism`](https://dlang.org/phobos/std_parallelism.html), which provides a series of parallel processing functions. One such function is `reduce()`, which splits an array between a given number of threads and applies a given operation to these chunks. In our case, the operation simply adds the elements to an accumulator: `a + b`. Follow and run the code in `chapters/compute/threads/guides/sum-array-threads/support/d/sum_array_threads_reduce.d`. The number of threads is used within a [`TaskPool`](https://dlang.org/phobos/std_parallelism.html#.TaskPool). -This structure is a thread manager (not scheduler). -It silently creates the number of threads we request and then `reduce()` spreads its workload between these threads. +This structure manages a pool of worker threads (not a scheduler). +It creates the requested number of worker threads, `reduce()` then spreads the workload between them. Now that you've seen how parallelism works in D, go in `chapters/compute/threads/guides/sum-array-threads/support/java/SumArrayThreads.java` and follow the TODOs. The code is similar to the one written in D, and it uses `ThreadPoolExecutor`. @@ -31,20 +31,20 @@ javac SumArrayThreads.java java SumArrayThreads 4 ``` -4 is the number of threads used, but you can replace the value with a number less or equal than your available cores. +4 is the number of threads used, but you can replace the value with a number less than or equal to your available cores. ## OpenMP for C -Unlike D, C does not support parallel computation by design. -It needs a library to do advanced things, like `reduce()` from D. +Unlike D, C does not provide built-in high-level parallel constructs. +Libraries such as OpenMP or pthreads provide parallelism. We have chosen to use the OpenMP library for this. Follow the code in `chapters/compute/threads/guides/sum-array-threads/support/c/sum_array_threads_openmp.c`. The `#pragma` used in the code instructs the compiler to enable the `omp` module, and to parallelise the code. In this case, we instruct the compiler to perform a reduce of the array, using the `+` operator, and to store the results in the `result` variable. -This reduction uses threads to calculate the sum, similar to `summ_array_threads.c`, but in a much more optimised form. +This reduction uses threads to calculate the sum, similar to `sum_array_threads.c`, but in a much more optimised form. -One of the advantages of OpenMP is that is relatively easy to use. +One of the advantages of OpenMP is that it is relatively easy to use. The syntax requires only a few additional lines of code and compiler options, thus converting sequential code into parallel code quickly. For example, using `#pragma omp parallel for`, a developer can parallelize a `for loop`, enabling iterations to run across multiple threads. diff --git a/chapters/compute/threads/guides/wait-for-me-threads/README.md b/chapters/compute/threads/guides/wait-for-me-threads/README.md index d60c76c354..bebc5d2c56 100644 --- a/chapters/compute/threads/guides/wait-for-me-threads/README.md +++ b/chapters/compute/threads/guides/wait-for-me-threads/README.md @@ -6,13 +6,13 @@ For now, do not wait for it to finish; simply start it. Compile the code and run the resulting executable several times. -See that the negative numbers appear from different indices. -This is precisely the nondeterminism that we talked about [in the previous section](tasks/wait-for-me-processes.md). +Note how the negative numbers appear at different indices on each run — this demonstrates the nondeterministic scheduling we discussed [in the previous section](tasks/wait-for-me-processes.md). Now wait for that thread to finish and see that all the printed numbers are consistently negative. -As you can see, waiting is a very coarse form of synchronization. -If we only use waiting, we can expect no speedup as a result of parallelism, because one thread must finish completely before another can continue. +Waiting is a coarse form of synchronization. +If you start a thread and then immediately wait for it to finish before starting the next, you serialize the work and will see no speedup. +Finer-grained synchronization or letting threads run concurrently without sequential waits is needed to gain parallel speedup. We will discuss more fine-grained synchronization mechanisms [later in this lab](reading/synchronization.md). Also, at this point, you might be wondering why this exercise is written in D, while [the same exercise, but with processes](reading/processes.md) was written in Python. diff --git a/chapters/data/memory-security/drills/tasks/exec-shellcode/Makefile b/chapters/data/memory-security/drills/tasks/exec-shellcode/Makefile index 35ad62412c..e087c739fe 100644 --- a/chapters/data/memory-security/drills/tasks/exec-shellcode/Makefile +++ b/chapters/data/memory-security/drills/tasks/exec-shellcode/Makefile @@ -2,9 +2,8 @@ PYTHON = python3 SCRIPT = generate_skels.py skels: - mkdir -p support/src - $(PYTHON) $(SCRIPT) --input ./solution/src --output ./support/src - $(PYTHON) $(SCRIPT) --input ./solution/tests --output ./support/tests + mkdir -p support + $(PYTHON) $(SCRIPT) --input ./solution --output ./support clean: rm -rf support/ diff --git a/chapters/data/memory-security/guides/buffer-overflow-leak/README.md b/chapters/data/memory-security/guides/buffer-overflow-leak/README.md index 67f9c556f5..580cefc032 100644 --- a/chapters/data/memory-security/guides/buffer-overflow-leak/README.md +++ b/chapters/data/memory-security/guides/buffer-overflow-leak/README.md @@ -26,7 +26,7 @@ student@os:~/.../guides/buffer-overflow-leak/support/$ ./string_leak | xxd In file `string_leak.c` replace the usage of `memcpy` with `strcpy`. Do not modify anything else (including the size of the buffer). -As the name suggests, `strcpy()` is specialized for string copies, therefore we don need to specify how much we want to copy. +As the name suggests, `strcpy()` is specialized for string copies, therefore we don't need to specify how much we want to copy. What is the result? Is the result correct? Explain the result. diff --git a/chapters/data/process-memory/drills/tasks/copy/solution/src/Makefile b/chapters/data/process-memory/drills/tasks/copy/solution/src/Makefile index 69b1725049..b516185d3b 100644 --- a/chapters/data/process-memory/drills/tasks/copy/solution/src/Makefile +++ b/chapters/data/process-memory/drills/tasks/copy/solution/src/Makefile @@ -19,7 +19,7 @@ OBJS = $(SRCS:.c=.o) BINARIES = mmap_copy read_write_copy # Default rule: Build everything -all: $(BINARIES) +all: $(BINARIES) in.dat # Rule to compile the logger $(LOGGER_OBJ): $(LOGGER_DIR)/log.c @@ -37,9 +37,13 @@ mmap_copy: mmap_copy.o $(LOGGER) read_write_copy: read_write_copy.o $(LOGGER) $(CC) $(CFLAGS) read_write_copy.o $(LOGGER) -o read_write_copy $(LDFLAGS) +# Rule to create the in.dat file with some contents +in.dat: + echo "Hello, world!" > in.dat + # Clean rule: Remove object files and binaries clean: - -rm -f $(OBJS) $(BINARIES) + -rm -f $(OBJS) $(BINARIES) in.dat @make -C $(LOGGER_DIR) clean # Clean the logger directory as well .PHONY: all clean diff --git a/chapters/data/process-memory/drills/tasks/static-dynamic/support/Makefile b/chapters/data/process-memory/drills/tasks/static-dynamic/support/Makefile deleted file mode 100644 index bb2f66fc4c..0000000000 --- a/chapters/data/process-memory/drills/tasks/static-dynamic/support/Makefile +++ /dev/null @@ -1,43 +0,0 @@ -# Get the relative path to the directory of the current makefile. -MAKEFILE_DIR := $(dir $(lastword $(MAKEFILE_LIST))) -INCLUDES_DIR := $(MAKEFILE_DIR) -UTILS_DIR := $(MAKEFILE_DIR)/utils -LOGGER_DIR := $(UTILS_DIR)/log - -# Compiler and flags -CPPFLAGS += -I$(INCLUDES_DIR) -CFLAGS += -g -Wall -Wextra -LDFLAGS += -z lazy - -# Logger object -LOGGER_OBJ = log.o -LOGGER = $(LOGGER_DIR)/$(LOGGER_OBJ) - -# Source and object files for alloc_size -SRC = hello.c -OBJ = $(SRC:.c=.o) - -# Binary name for alloc_size -BINARY = hello - -# Default rule: Build the binary -all: $(BINARY) - -# Rule to compile the logger -$(LOGGER_OBJ): $(LOGGER_DIR)/log.c - $(MAKE) -C $(LOGGER_DIR) $(LOGGER_OBJ) - -# Rule to compile alloc_size object file -$(OBJ): %.o: %.c - $(CC) $(CFLAGS) $(CPPFLAGS) -c $< -o $@ - -# Rule to create the alloc_size binary -$(BINARY): $(OBJ) $(LOGGER) - $(CC) $(CFLAGS) $(OBJ) $(LOGGER) -o $(BINARY) $(LDFLAGS) - -# Clean rule: Remove object files and binaries -clean: - -rm -f $(OBJ) $(BINARY) - @make -C $(LOGGER_DIR) clean # Clean the logger directory as well - -.PHONY: all clean diff --git a/chapters/data/process-memory/reading/process-memory.md b/chapters/data/process-memory/reading/process-memory.md index bdc3ca7fdf..dfb0ee244d 100644 --- a/chapters/data/process-memory/reading/process-memory.md +++ b/chapters/data/process-memory/reading/process-memory.md @@ -94,7 +94,7 @@ The operating system allocates memory in chunks of a predefined size (in our cas We want to see the difference in memory layout between the statically-linked and dynamically-linked executables. -Enter the `chapters/data/process-memory/drills/tasks/static-dynamic/support` directory and build the statically-linked and dynamically-linked executables `hello-static` and `hello-dynamic`: +Enter the `chapters/data/working-with-memory/guides/static-dynamic/support` directory and build the statically-linked and dynamically-linked executables `hello-static` and `hello-dynamic`: ```console student@os:~/.../drills/tasks/static-dynamic/support$ make diff --git a/chapters/data/working-with-memory/drills/tasks/memory-access/README.md b/chapters/data/working-with-memory/drills/tasks/memory-access/README.md index 39eae5e1e3..0f361ecdc0 100644 --- a/chapters/data/working-with-memory/drills/tasks/memory-access/README.md +++ b/chapters/data/working-with-memory/drills/tasks/memory-access/README.md @@ -6,7 +6,10 @@ Inspect the `mem_access.c` source file. 1. Describe each variable by completing its **(address, size, access rights)** tuple. 1. Try to modify the `ca`, `cp` and `cp2` variables by assigning some other value to them. -Check your changes by running the `checker.sh` script in `support/tests/`. -Explain the behavior. + + Check your changes by running the `checker.sh` script in `support/tests/`. + Explain the behavior. + + [Quiz](../../questions/memory-access.md) If you're having difficulties solving this exercise, go through [this](../../../reading/working-with-memory.md) reading material. diff --git a/chapters/data/working-with-memory/drills/tasks/memory-corruption/README.md b/chapters/data/working-with-memory/drills/tasks/memory-corruption/README.md index a432cd0c1e..ff17a4afce 100644 --- a/chapters/data/working-with-memory/drills/tasks/memory-corruption/README.md +++ b/chapters/data/working-with-memory/drills/tasks/memory-corruption/README.md @@ -3,7 +3,7 @@ For this practice item, you will need to identify the programming mistake that makes it possible to corrupt memory. Navigate to the `memory-corruption/` directory in the lab archive (or `chapters/data/working-with-memory/drills/tasks/memory-corruption/` if you are working directly in the repository) run `make skels` and enter `support/src/`. -Inspect the source file `segfault.c`. +Inspect the source file `c_segfault.c`. 1. What does the program do? (this could be a quiz in the final form) 1. Compile and run it. diff --git a/chapters/data/working-with-memory/drills/tasks/memory-corruption/solution/src/c_segfault.c b/chapters/data/working-with-memory/drills/tasks/memory-corruption/solution/src/c_segfault.c index e5feff55b3..fbb6a0ca23 100644 --- a/chapters/data/working-with-memory/drills/tasks/memory-corruption/solution/src/c_segfault.c +++ b/chapters/data/working-with-memory/drills/tasks/memory-corruption/solution/src/c_segfault.c @@ -7,7 +7,7 @@ /* TODO 9: Fix the bug causing the Segmentation Fault */ static void print_bit_array(unsigned int the_bits[SIZE_INT]) { - int i = SIZE_INT-1; + int i = SIZE_INT - 1; while (i >= 0) { printf("%u\n", the_bits[i]); @@ -16,15 +16,15 @@ static void print_bit_array(unsigned int the_bits[SIZE_INT]) } /* REPLACE 9 */ -/* static void print_bit_array(unsigned int the_bits[SIZE_INT]) */ -/* { */ -/* unsigned int i = SIZE_INT-1; */ -/* */ -/* while (i >= 0) { */ -/* printf("%u\n", the_bits[i]); */ -/* i--; */ -/* } */ -/* } */ +/* static void print_bit_array(unsigned int the_bits[SIZE_INT]) +{ + unsigned int i = SIZE_INT - 1; + + while (i >= 0) { + printf("%u\n", the_bits[i]); + i--; + } +} */ static void to_bits(unsigned int value, unsigned int in_bits[SIZE_INT]) { @@ -46,8 +46,8 @@ static unsigned int factorial(unsigned int num) if (num == 0) return 1; - fact = factorial(num-1); - return fact*num; + fact = factorial(num - 1); + return fact * num; } int main(void) diff --git a/chapters/data/working-with-memory/drills/tasks/memory-corruption/solution/src/d_segfault.d b/chapters/data/working-with-memory/drills/tasks/memory-corruption/solution/src/d_segfault.d index 14998df42f..c897b9247a 100644 --- a/chapters/data/working-with-memory/drills/tasks/memory-corruption/solution/src/d_segfault.d +++ b/chapters/data/working-with-memory/drills/tasks/memory-corruption/solution/src/d_segfault.d @@ -7,26 +7,26 @@ enum SIZE_INT = 32; // TODO 9: Fix the bug causing the Segmentation Fault void print_bit_array(uint[SIZE_INT] the_bits) { - int i = SIZE_INT-1; + int i = SIZE_INT - 1; while (i >= 0) { - printf("%u\n", the_bits[i]); + writeln(the_bits[i]); i--; } } // REPLACE 9 -// void print_bit_array(uint[SIZE_INT] the_bits) * +// void print_bit_array(uint[SIZE_INT] the_bits) // { -// uint i = SIZE_INT-1; +// uint i = SIZE_INT - 1; // while (i >= 0) // { -// printf("%u\n", the_bits[i]); +// writeln(the_bits[i]); // i--; // } // } -void to_bits(uint value, uint[SIZE_INT] in_bits) +void to_bits(uint value, ref uint[SIZE_INT] in_bits) { uint shift_bit = 0x0001; @@ -47,8 +47,8 @@ uint factorial(uint num) if (num == 0) return 1; - fact = factorial(num-1); - return fact*num; + fact = factorial(num - 1); + return fact * num; } int main() @@ -57,11 +57,11 @@ int main() uint[SIZE_INT] the_bits; uint fact; - printf("Input a positive integer: "); - scanf("%u", &number); + write("Input a positive integer: "); + readf(" %u", &number); fact = factorial(number); - printf("%u Factorial = %u \n", number, fact); + writeln(number, " Factorial = ", fact); to_bits(fact, the_bits); print_bit_array(the_bits); diff --git a/chapters/data/working-with-memory/drills/tasks/memory-protection/solution/src/mem_prot.c b/chapters/data/working-with-memory/drills/tasks/memory-protection/solution/src/mem_prot.c index 56ef2ffb62..cc84ecb690 100644 --- a/chapters/data/working-with-memory/drills/tasks/memory-protection/solution/src/mem_prot.c +++ b/chapters/data/working-with-memory/drills/tasks/memory-protection/solution/src/mem_prot.c @@ -8,6 +8,9 @@ static void exec_do_nothing(void) /* stored in .text section (r-x) */ { } +/* TODO 1: Add a const variable called ro */ +const int ro = 42; + static void do_write(const char *msg, void *address, int value) { puts(msg); @@ -33,9 +36,6 @@ static void do_exec(const char *msg, void *address) int main(void) { - /* TODO 1: Add a const variable called ro*/ - const int ro = 42; - do_read("reading from .data section", &data[0]); do_write("writing to .data section", &data[0], 77); diff --git a/chapters/data/process-memory/drills/tasks/static-dynamic/.gitignore b/chapters/data/working-with-memory/guides/static-dynamic/.gitignore similarity index 100% rename from chapters/data/process-memory/drills/tasks/static-dynamic/.gitignore rename to chapters/data/working-with-memory/guides/static-dynamic/.gitignore diff --git a/chapters/data/working-with-memory/guides/static-dynamic/support/Makefile b/chapters/data/working-with-memory/guides/static-dynamic/support/Makefile new file mode 100644 index 0000000000..6cbf02596b --- /dev/null +++ b/chapters/data/working-with-memory/guides/static-dynamic/support/Makefile @@ -0,0 +1,25 @@ +SRC := hello.c +OBJ := $(SRC:.c=.o) + +STATIC_BINARY := hello-static +DYNAMIC_BINARY := hello-dynamic + +CC := gcc +CFLAGS := -Wall -Wextra -g +LDFLAGS_STATIC := -static + +all: $(OBJ) $(STATIC_BINARY) $(DYNAMIC_BINARY) + +%.o: %.c + $(CC) $(CFLAGS) -c $< -o $@ + +$(STATIC_BINARY): $(OBJ) + $(CC) $(CFLAGS) $(OBJ) -o $@ $(LDFLAGS_STATIC) + +$(DYNAMIC_BINARY): $(OBJ) + $(CC) $(CFLAGS) $(OBJ) -o $@ + +clean: + -rm -f $(OBJ) $(STATIC_BINARY) $(DYNAMIC_BINARY) + +.PHONY: all clean diff --git a/chapters/data/process-memory/drills/tasks/static-dynamic/support/hello.c b/chapters/data/working-with-memory/guides/static-dynamic/support/hello.c similarity index 100% rename from chapters/data/process-memory/drills/tasks/static-dynamic/support/hello.c rename to chapters/data/working-with-memory/guides/static-dynamic/support/hello.c diff --git a/chapters/data/process-memory/drills/tasks/static-dynamic/support/utils/log/CPPLINT.cfg b/chapters/data/working-with-memory/guides/static-dynamic/support/utils/log/CPPLINT.cfg similarity index 100% rename from chapters/data/process-memory/drills/tasks/static-dynamic/support/utils/log/CPPLINT.cfg rename to chapters/data/working-with-memory/guides/static-dynamic/support/utils/log/CPPLINT.cfg diff --git a/chapters/data/process-memory/drills/tasks/static-dynamic/support/utils/log/log.c b/chapters/data/working-with-memory/guides/static-dynamic/support/utils/log/log.c similarity index 100% rename from chapters/data/process-memory/drills/tasks/static-dynamic/support/utils/log/log.c rename to chapters/data/working-with-memory/guides/static-dynamic/support/utils/log/log.c diff --git a/chapters/data/process-memory/drills/tasks/static-dynamic/support/utils/log/log.h b/chapters/data/working-with-memory/guides/static-dynamic/support/utils/log/log.h similarity index 100% rename from chapters/data/process-memory/drills/tasks/static-dynamic/support/utils/log/log.h rename to chapters/data/working-with-memory/guides/static-dynamic/support/utils/log/log.h diff --git a/chapters/data/process-memory/drills/tasks/static-dynamic/support/utils/utils.h b/chapters/data/working-with-memory/guides/static-dynamic/support/utils/utils.h similarity index 100% rename from chapters/data/process-memory/drills/tasks/static-dynamic/support/utils/utils.h rename to chapters/data/working-with-memory/guides/static-dynamic/support/utils/utils.h diff --git a/chapters/io/file-descriptors/drills/tasks/buffering/README.md b/chapters/io/file-descriptors/drills/tasks/buffering/README.md index bd3bd62247..026e8c3153 100644 --- a/chapters/io/file-descriptors/drills/tasks/buffering/README.md +++ b/chapters/io/file-descriptors/drills/tasks/buffering/README.md @@ -21,6 +21,9 @@ We'll checkout how effective buffering is in `libc` and then we'll do it ourselv Wrote 1048576 bytes to test-file.txt in 38 ms ``` + If possible, try to manually test your implementation. + Use `./checker.sh` only when you are sure of your result. + Buffering achieves dramatic performance gains, reducing read times by **98%** and write times by **99.8%** in this example! This demonstrates the power of buffering, even though it’s an extreme case. diff --git a/chapters/io/file-descriptors/drills/tasks/mmap_cp/README.md b/chapters/io/file-descriptors/drills/tasks/mmap_cp/README.md index 64a22fd61d..1610e87c9a 100644 --- a/chapters/io/file-descriptors/drills/tasks/mmap_cp/README.md +++ b/chapters/io/file-descriptors/drills/tasks/mmap_cp/README.md @@ -1,6 +1,8 @@ # Copy a File with `mmap()` -Navigate to `file-descriptors/drills/tasks/mmap_cp` and run `make` to generate `support`. +Enter the `mmap_cp/` directory in the lab archive (or `chapters/io/file-descriptors/drills/tasks/mmap_cp` if you are working directly in +the repository), run `make skels`, then enter `support/`. +Run through the practice items below. As you know `mmap()` can map files in memory, perform operations on them, and then write them back to the disk. Let's check how well it performs by comparing it to the `cp` command. The benchmarking is automated by `benchmark_cp.sh` so focus on completing `mmap_cp.c` for now. @@ -16,6 +18,11 @@ make: Nothing to be done for 'all'. Test PASSED (File copies are identical) ``` +To test your implementation, navigate to the `mmap_cp/support/src` folder. +Run `make` to compile the code and test it manually with files generated by you. + +When you think your code is correct, go to `mmap_cp/support/test` and run `checker.sh` to validate your solution. + 1. Open `mmap_cp.c` and complete the TODOs to map the files in memory and copy the contents. Do not forget to clean up by unmapping and closing the files. diff --git a/chapters/io/file-descriptors/drills/tasks/my-cat/README.md b/chapters/io/file-descriptors/drills/tasks/my-cat/README.md index a7927c7210..359426b2b4 100644 --- a/chapters/io/file-descriptors/drills/tasks/my-cat/README.md +++ b/chapters/io/file-descriptors/drills/tasks/my-cat/README.md @@ -1,6 +1,8 @@ # My `cat` -Navigate to `chapters/io/file-descriptors/drills/tasks/my-cat/support/src` and checkout `my_cat.c`. +Enter the `my_cat/` directory in the lab archive (or `chapters/io/file-descriptors/drills/tasks/my_cat` if you are working directly in +the repository), run `make skels`, then enter `support/`. +Run through the practice items below. We propose to implement the Linux command `cat` that reads one or more files, **concatenates** them (hence the name `cat`), and prints them to standard output. 1. Inside the `tests/` directory, you will need to run `checker.sh`. The output for a successful implementation should look like this: @@ -18,6 +20,11 @@ Good job! ---------------------------------------- ``` +To test your implementation, navigate to the `mmap_cp/support/src` folder. +Run `make` to compile the code and test it manually with files generated by you. + +When you think your code is correct, go to `mmap_cp/support/test` and run `checker.sh` to validate your solution. + 1. Implement `rread()` wrapper over `read()`. `read()` system call does not guarantee that it will read the requested number of bytes in a single call. diff --git a/chapters/io/ipc/drills/tasks/anon-pipes/README.md b/chapters/io/ipc/drills/tasks/anon-pipes/README.md index 3bbb762b00..167c6f08b9 100644 --- a/chapters/io/ipc/drills/tasks/anon-pipes/README.md +++ b/chapters/io/ipc/drills/tasks/anon-pipes/README.md @@ -1,6 +1,8 @@ # Anonymous Pipes Communication -Navigate to `chapters/io/ipc/drills/tasks/anon-pipes` and run `make` to generate the `support/` folder. +Enter the `anon_pipes/` directory in the lab archive (or `chapters/io/ipc/drills/tasks/anon-pipes` if you are working directly in +the repository), run `make skels`, then enter `support/`. +Run through the practice items below. In this exercise, you'll implement client-server communication between a parent and a child process using an anonymous pipe. The parent will act as the sender, while the child acts as the receiver, with both processes sharing messages through the pipe. Since pipes are unidirectional, each process should close the end of the pipe it does not use. @@ -15,6 +17,11 @@ Test for short string ........... PASSED Test for long string ........... PASSED ``` +To test your implementation, navigate to the `mmap_cp/support/src` folder. +Run `make` to compile the code and test it manually with files generated by you. + +When you think your code is correct, go to `mmap_cp/support/test` and run `checker.sh` to validate your solution. + 1. Use the [`pipe()` syscall](https://man7.org/linux/man-pages/man7/pipe.7.html) to create the pipe. Remember, the first file descriptor (`fds[0]`) is the read end, and the second (`fds[1]`) is the write end, similar to how `stdin` and `stdout` are represented by file descriptors `0` and `1`. diff --git a/chapters/io/ipc/drills/tasks/client-server/README.md b/chapters/io/ipc/drills/tasks/client-server/README.md index 48351f5694..301cde5b3b 100644 --- a/chapters/io/ipc/drills/tasks/client-server/README.md +++ b/chapters/io/ipc/drills/tasks/client-server/README.md @@ -1,6 +1,6 @@ # Ordered Client-Server Communication -Navigate to `chapters/io/ipc/drills/tasks/client-server/` and run `make` to generate the `support` directory. +Navigate to `client-server` directory of the extracted archive (or `chapters/io/ipc/drills/tasks/client-server/` if you are working directly in the repository) and run `make` to generate the `support` directory. This exercise will guide you in creating a basic messaging protocol between a server and a client. Although in real-world applications a server typically handles multiple connections at once, here we focus on a single connection. Handling multiple connections is further explored in [I/O multiplexing](../../../../io-multiplexing/reading/io-multiplexing.md). @@ -15,10 +15,13 @@ Our application protocol is defined as follows: Since we are blocking on `recv()`, the message order is fixed - the client **must** initiate communication. In real-world applications, this constraint can be avoided with [I/O multiplexing](../../../../io-multiplexing/reading/io-multiplexing.md). +To use the checker, run `make check` from `support`, it will test the client and afterwards the server. +You **must** implement the client first to later test the server using the checker. 1. Open `support/client.c` and complete the TODOs to enable message exchange with the server. Test your client by running `python server.py` in one terminal and then `./client` in another. If correctly implemented, you should be able to exchange messages as outlined above. + If the port is blocked, run `reset_connection.sh` from `tests`. **Bonus Question:** Why is it OK for the client to be implemented in C while the server is implemented in Python? diff --git a/chapters/io/ipc/drills/tasks/client-server/solution/Makefile b/chapters/io/ipc/drills/tasks/client-server/solution/Makefile index 8aa3737d14..b98e0bfbc9 100644 --- a/chapters/io/ipc/drills/tasks/client-server/solution/Makefile +++ b/chapters/io/ipc/drills/tasks/client-server/solution/Makefile @@ -34,4 +34,8 @@ clean: -rm -f $(OBJS) $(BINARIES) -rm -f $(LOGGER) +check: + ../tests/check_client.sh + ../tests/check_server.sh + .PHONY: all clean diff --git a/chapters/io/ipc/drills/tasks/client-server/tests/check_client.sh b/chapters/io/ipc/drills/tasks/client-server/tests/check_client.sh new file mode 100755 index 0000000000..8c4235d7b4 --- /dev/null +++ b/chapters/io/ipc/drills/tasks/client-server/tests/check_client.sh @@ -0,0 +1,83 @@ +#!/bin/bash +# SPDX-License-Identifier: BSD-3-Clause + +PORT=5000 +SUPPORT_DIR="../support" +TESTS_DIR="../tests" + +SERVER_PY="$SUPPORT_DIR/server.py" +CLIENT_BIN="$SUPPORT_DIR/client" + +SERVER_LOG="$SUPPORT_DIR/server_output.log" +CLIENT_LOG="$SUPPORT_DIR/client_output.log" + +INPUT_CLIENT="$TESTS_DIR/input_client.txt" +INPUT_SERVER="$TESTS_DIR/input_server.txt" + +OUTPUT_CLIENT="$TESTS_DIR/output_client.txt" +OUTPUT_SERVER="$TESTS_DIR/output_server.txt" + +make -C "$SUPPORT_DIR" +echo + +# Kill any leftover server process +PID_LIST=$(lsof -ti tcp:$PORT) +if [ -n "$PID_LIST" ]; then + echo "Killing leftover server processes on port $PORT: $PID_LIST" + kill -9 "$PID_LIST" + sleep 1 +fi + +# Start server with input_server.txt feeding it +stdbuf -oL python3 "$SERVER_PY" < "$INPUT_SERVER" > "$SERVER_LOG" 2>&1 & +SERVER_PID=$! + +# Wait for server port ready +for _i in {1..10}; do + if lsof -ti tcp:$PORT >/dev/null 2>&1; then break; fi + sleep 1 +done + +# Start client with input_client.txt feeding it +stdbuf -oL "$CLIENT_BIN" < "$INPUT_CLIENT" > "$CLIENT_LOG" 2>&1 & +CLIENT_PID=$! + +echo "Establishing connection between client and server..." + +# Wait up to 8 seconds for client to exit +SECONDS=0 +while kill -0 $CLIENT_PID 2>/dev/null; do + if [ $SECONDS -ge 8 ]; then + echo "Client test .......................... failed ... 0" + kill -9 $CLIENT_PID $SERVER_PID 2>/dev/null + break + fi + sleep 0.1 +done + +echo "Extracting outputs..." + +# Extract what client output received +cat "$CLIENT_LOG" | cut -c26- > "$OUTPUT_CLIENT" + +# Extract what server received from client +tail -n +3 "$SERVER_LOG" | cut -c32- > "$OUTPUT_SERVER" + +if [ "$(tail -n 1 "$INPUT_SERVER")" = "exit" ]; then + _server_exit=true +fi + +echo "Comparing outputs with expected inputs..." + +diff_client=$(diff -q "$INPUT_SERVER" "$OUTPUT_CLIENT") +diff_server=$(diff -q "$INPUT_CLIENT" "$OUTPUT_SERVER") + +if [ -z "$diff_client" ] || [ -z "$diff_server" ]; then + echo "Client test .......................... passed ... 100" + RESULT=0 +else + echo "Client test .......................... failed ... 0" + RESULT=1 +fi + +exit $RESULT diff --git a/chapters/io/ipc/drills/tasks/client-server/tests/check_server.sh b/chapters/io/ipc/drills/tasks/client-server/tests/check_server.sh new file mode 100755 index 0000000000..4282672336 --- /dev/null +++ b/chapters/io/ipc/drills/tasks/client-server/tests/check_server.sh @@ -0,0 +1,78 @@ +#!/bin/bash +# SPDX-License-Identifier: BSD-3-Clause + +PORT=5000 +SUPPORT_DIR="../support" +TESTS_DIR="../tests" + +SERVER_BIN="$SUPPORT_DIR/server" +CLIENT_PY="$SUPPORT_DIR/client.py" + +SERVER_LOG="$SUPPORT_DIR/server_output.log" +CLIENT_LOG="$SUPPORT_DIR/client_output.log" + +INPUT_SERVER="$TESTS_DIR/input_server.txt" +INPUT_CLIENT="$TESTS_DIR/input_client.txt" + +OUTPUT_SERVER="$TESTS_DIR/output_server.txt" +OUTPUT_CLIENT="$TESTS_DIR/output_client.txt" + +make -C "$SUPPORT_DIR" +echo + +# Kill leftover server processes +PID_LIST=$(lsof -ti tcp:$PORT) +if [ -n "$PID_LIST" ]; then + echo "Killing leftover server processes on port $PORT: $PID_LIST" + kill -9 "$PID_LIST" + sleep 1 +fi + +# Start server with input_server.txt feeding it +stdbuf -oL "$SERVER_BIN" < "$INPUT_SERVER" > "$SERVER_LOG" 2>&1 & +SERVER_PID=$! + +# Wait for server port ready +for _i in {1..10}; do + if lsof -ti tcp:$PORT >/dev/null 2>&1; then break; fi + sleep 1 +done + +# Start client with input_client.txt feeding it +stdbuf -oL python3 "$CLIENT_PY" < "$INPUT_CLIENT" > "$CLIENT_LOG" 2>&1 & + +echo "Establishing connection between client and server..." + +# Wait up to 8 seconds for server to exit +SECONDS=0 +while kill -0 $SERVER_PID 2>/dev/null; do + if [ $SECONDS -ge 8 ]; then + echo "Client test .......................... failed ... 0" + kill -9 $SERVER_PID 2>/dev/null + exit 0 + fi + sleep 0.1 +done + +echo "Extracting outputs..." + +# Extract what client output received +tail -n +2 "$CLIENT_LOG" | cut -c26- > "$OUTPUT_CLIENT" + +# Extract what server received from client +tail -n +2 "$SERVER_LOG" | cut -c27- > "$OUTPUT_SERVER" + +echo "Comparing outputs with expected inputs..." + +diff_client=$(diff -q "$INPUT_SERVER" "$OUTPUT_CLIENT") +diff_server=$(diff -q "$INPUT_CLIENT" "$OUTPUT_SERVER") + +if [ -z "$diff_client" ] || [ -z "$diff_server" ]; then + echo "Server test .......................... passed ... 100" + RESULT=0 +else + echo "Server test .......................... failed ... 0" + RESULT=1 +fi + +exit $RESULT diff --git a/chapters/io/ipc/drills/tasks/client-server/tests/input_client.txt b/chapters/io/ipc/drills/tasks/client-server/tests/input_client.txt new file mode 100644 index 0000000000..66fe34297c --- /dev/null +++ b/chapters/io/ipc/drills/tasks/client-server/tests/input_client.txt @@ -0,0 +1,4 @@ +If +Got +Well +exit diff --git a/chapters/io/ipc/drills/tasks/client-server/tests/input_server.txt b/chapters/io/ipc/drills/tasks/client-server/tests/input_server.txt new file mode 100644 index 0000000000..65bccf9495 --- /dev/null +++ b/chapters/io/ipc/drills/tasks/client-server/tests/input_server.txt @@ -0,0 +1,3 @@ +You +Here +Done diff --git a/chapters/io/ipc/drills/tasks/client-server/tests/output_client.txt b/chapters/io/ipc/drills/tasks/client-server/tests/output_client.txt new file mode 100644 index 0000000000..65bccf9495 --- /dev/null +++ b/chapters/io/ipc/drills/tasks/client-server/tests/output_client.txt @@ -0,0 +1,3 @@ +You +Here +Done diff --git a/chapters/io/ipc/drills/tasks/client-server/tests/output_server.txt b/chapters/io/ipc/drills/tasks/client-server/tests/output_server.txt new file mode 100644 index 0000000000..95e1efa90e --- /dev/null +++ b/chapters/io/ipc/drills/tasks/client-server/tests/output_server.txt @@ -0,0 +1,3 @@ +If +Got +Well diff --git a/chapters/io/ipc/drills/tasks/client-server/tests/reset_connection.sh b/chapters/io/ipc/drills/tasks/client-server/tests/reset_connection.sh new file mode 100755 index 0000000000..97884d7df9 --- /dev/null +++ b/chapters/io/ipc/drills/tasks/client-server/tests/reset_connection.sh @@ -0,0 +1,15 @@ +#!/bin/bash +# SPDX-License-Identifier: BSD-3-Clause + +PORT=5000 + +# Kill any leftover server process +PID_LIST=$(lsof -ti tcp:$PORT) +if [ -n "$PID_LIST" ]; then + echo "Killing leftover server processes on port $PORT: $PID_LIST" + kill -9 "$PID_LIST" + sleep 1 + echo "Connection reset complete." +else + echo "No processes found on port $PORT." +fi diff --git a/chapters/io/ipc/drills/tasks/named-pipes/README.md b/chapters/io/ipc/drills/tasks/named-pipes/README.md index ff3f0a1f5b..b3fade71a0 100644 --- a/chapters/io/ipc/drills/tasks/named-pipes/README.md +++ b/chapters/io/ipc/drills/tasks/named-pipes/README.md @@ -1,12 +1,13 @@ # Named Pipes Communication -Navigate to `chapters/io/ipc/drills/tasks/named-pipes` and run `make` to generate the `support` directory. +Navigate to `named-pipes/` directory of the extracted archive (or `chapters/io/ipc/drills/tasks/named-pipes` if you are working directly in the repository). +Run `make` and then enter `support/` folder and go through the practice items below. In this exercise, you'll implement client-server communication between two processes using a named pipe, also called **FIFO**. Both the sender and receiver are created from the same binary: run without arguments for a receiver, or with `-s` for a sender. -1. Use the [`mkfifo()` syscall](https://man7.org/linux/man-pages/man3/mkfifo.3.html) to create a named pipe. - If the FIFO already exists, use [`access()`](https://man7.org/linux/man-pages/man2/access.2.html) to check its permissions. - If permissions are incorrect, or if it does not exist, recreate the FIFO. +1. Use [`access()`](https://man7.org/linux/man-pages/man2/access.2.html) to check if the FIFO already exists and has the right permissions. + If it exists but has the wrong permissions, delete it using [`unlink()`](https://man7.org/linux/man-pages/man2/unlink.2.html). + If it doesn't exist create it using [`mkfifo()`](https://man7.org/linux/man-pages/man3/mkfifo.3.html). 1. Complete the TODOs in `receiver_loop()` and `sender_loop()` to enable communication. Ensure the FIFO is open before reading from or writing to it. diff --git a/chapters/io/ipc/drills/tasks/named-pipes/solution/src/named_pipe.c b/chapters/io/ipc/drills/tasks/named-pipes/solution/src/named_pipe.c index b6ac4363e8..1d1762f3be 100644 --- a/chapters/io/ipc/drills/tasks/named-pipes/solution/src/named_pipe.c +++ b/chapters/io/ipc/drills/tasks/named-pipes/solution/src/named_pipe.c @@ -33,7 +33,7 @@ static void receiver_loop(void) char output[BUFSIZ]; int fd, rc; - /* TODO 4: Create FIFO if it does not exist, then open it for reading. */ + /* TODO 4: Create the FIFO if it does not exist, then open it for reading. */ create_fifo_if_needed(); fd = open(fifo_path, O_RDONLY); @@ -51,7 +51,7 @@ static void receiver_loop(void) fflush(stdout); } - /* TODO 2: Close FIFO. */ + /* TODO 2: Close the FIFO. */ rc = close(fd); DIE(rc < 0, "close"); } @@ -61,7 +61,7 @@ static void sender_loop(void) char input[BUFSIZ]; int fd, rc; - /* TODO 4: Create FIFO if it does not exist, then open it for writing. */ + /* TODO 4: Create the FIFO if it does not exist, then open it for writing. */ create_fifo_if_needed(); fd = open(fifo_path, O_WRONLY); @@ -82,7 +82,7 @@ static void sender_loop(void) DIE(rc < 0, "write"); } - /* TODO 2: Close FIFO. */ + /* TODO 2: Close the FIFO. */ rc = close(fd); DIE(rc < 0, "close"); } diff --git a/chapters/io/ipc/drills/tasks/network-socket/README.md b/chapters/io/ipc/drills/tasks/network-socket/README.md index 5eed4c374a..234d2dcbe6 100644 --- a/chapters/io/ipc/drills/tasks/network-socket/README.md +++ b/chapters/io/ipc/drills/tasks/network-socket/README.md @@ -1,22 +1,25 @@ # Network Socket Communication -Navigate to `chapters/io/ipc/drills/tasks/network-socket` and run `make` to generate the `support` directory. -In this exercise, you'll implement client-server communication between two processes using a **network socket**. +Navigate to `network-socket/` directory of the extracted archive (or `chapters/io/ipc/drills/tasks/network-socket` if you are working directly in the repository). +Run `make` and then enter `support/` folder and go through the practice items below. +In this exercise, you'll implement client-server communication between two processes using **network sockets**. Both the sender and receiver are created from the same binary: run without arguments for a receiver, or with `-s` for a sender. 1. Complete the TODOs in the `sender_loop()` from `tcp_socket.c`. You need to verify whether the socket exists i.e. check if the receiver has created it. Next, **create** your own socket and **connect** to the receiver's socket using its address (**Hint:** use `get_sockaddr(, )` to obtain it). - Once the connection is established, you can send messages using `send()`. + Once the connection is established, you can send messages using [`send()`](https://man7.org/linux/man-pages/man2/send.2.html). 1. Complete the TODOs in the `receiver_loop()` from `tcp_socket.c`. Similarly, you will need to **create** a socket and **bind** it to the receiver's address (**Hint:** use `get_sockaddr(, )` for this). Instead of connecting, you will **listen** for and **accept** incoming connections. - When `accept()` receives a connection request, it will return a new socket file descriptor that you can use to receive messages via `recv()`. + When [`accept()`](https://man7.org/linux/man-pages/man2/accept.2.html) receives a connection request, it will return a new socket file descriptor that you can use to receive messages via [`recv()`](https://man7.org/linux/man-pages/man2/recv.2.html). + All sockets should be closed after their designated task has been completed. -1. Now we’ll implement the same functionality using datagrams (`SOCK_DGRAM`). +1. Now we’ll implement the same functionality using UDP datagrams (`SOCK_DGRAM`). Open `udp_socket.c` and complete the TODOs for `sender_loop()` and `receiver_loop()` functions. The workflow is similar, but `listen()`, `accept()`, and `connect()` are not required for datagram sockets. + Use `sendto()` and `receivefrom()` instead of `send()` and `receive()`. 1. Inside the `tests/` directory, you will need to run `checker.sh`. The output for a successful implementation should look like this: diff --git a/chapters/io/ipc/drills/tasks/network-socket/solution/src/tcp_socket.c b/chapters/io/ipc/drills/tasks/network-socket/solution/src/tcp_socket.c index 28cc59d784..91138f9c56 100644 --- a/chapters/io/ipc/drills/tasks/network-socket/solution/src/tcp_socket.c +++ b/chapters/io/ipc/drills/tasks/network-socket/solution/src/tcp_socket.c @@ -47,7 +47,7 @@ static void receiver_loop(void) rc = bind(listenfd, (struct sockaddr *) &addr, sizeof(addr)); DIE(rc < 0, "bind"); - /* TODO 2: Mark socket as passive socket using listen(). */ + /* TODO 2: Mark the socket as passive using listen(). */ rc = listen(listenfd, 1); DIE(rc < 0, "listen"); @@ -106,13 +106,13 @@ static void sender_loop(void) DIE(rc < 0, "send"); } - /* TODO 2: Close socket. */ + /* TODO 2: Close the socket. */ rc = close(sockfd); DIE(rc < 0, "close"); } /** - * Simulate a sender-receiver communication using a named pipe. + * Simulate a sender-receiver communication using network sockets. * Run the program as a receiver by default, or as a sender if the -s or --sender. */ int main(int argc, char *argv[]) diff --git a/chapters/io/ipc/drills/tasks/network-socket/solution/src/udp_socket.c b/chapters/io/ipc/drills/tasks/network-socket/solution/src/udp_socket.c index 14587c247a..bc27a49a6f 100644 --- a/chapters/io/ipc/drills/tasks/network-socket/solution/src/udp_socket.c +++ b/chapters/io/ipc/drills/tasks/network-socket/solution/src/udp_socket.c @@ -94,13 +94,13 @@ static void sender_loop(void) break; } - /* TODO 2: Close socket. */ + /* TODO 2: Close the socket. */ rc = close(sockfd); DIE(rc < 0, "close"); } /** - * Simulate a sender-receiver communication using a named pipe. + * Simulate a sender-receiver communication using network sockets. * Run the program as a receiver by default, or as a sender if the -s or --sender. */ int main(int argc, char *argv[]) diff --git a/chapters/io/ipc/drills/tasks/receive-challenges/README.md b/chapters/io/ipc/drills/tasks/receive-challenges/README.md index 3b1137e7d1..1afe852a61 100644 --- a/chapters/io/ipc/drills/tasks/receive-challenges/README.md +++ b/chapters/io/ipc/drills/tasks/receive-challenges/README.md @@ -1,6 +1,7 @@ # Receive Challenges -Navigate to `chapters/io/ipc/drills/tasks/receive-challenges` and run `make` to generate the `support` directory. +Navigate to `receive-challenges/` directory of the extracted archive (or `chapters/io/ipc/drills/tasks/receive-challenges` if you are working directly in the repository). +Run `make` and then enter `support/` folder and go through the practice items below. In this task, we will review all the IPC methods we have explored, including [anonymous pipes](../../../reading/pipes.md), [named pipes (FIFOs)](../../../reading/pipes.md), [UNIX sockets](../../../reading/unix-sockets.md), and [network sockets](../../../reading/network-sockets.md). Each challenge involves building a communication channel using the specified IPC method. diff --git a/chapters/io/ipc/drills/tasks/receive-challenges/solution/src/receive_fifo.c b/chapters/io/ipc/drills/tasks/receive-challenges/solution/src/receive_fifo.c index 497f83f0c0..a984915986 100644 --- a/chapters/io/ipc/drills/tasks/receive-challenges/solution/src/receive_fifo.c +++ b/chapters/io/ipc/drills/tasks/receive-challenges/solution/src/receive_fifo.c @@ -40,7 +40,7 @@ int main(void) char buffer[BUFSIZ]; /** - * Create FIFO if it doesn't exist or if it exists and has incorrect + * Create the FIFO if it doesn't exist or if it exists and has incorrect * permissions. */ rc = access(fifo_path, R_OK | W_OK); @@ -50,11 +50,11 @@ int main(void) DIE(rc < 0, "mkfifo"); } - /* TODO 2: Open FIFO for reading. */ + /* TODO 2: Open the FIFO for reading. */ fd = open(fifo_path, O_RDONLY); DIE(fd < 0, "open"); - /* TODO 2: Read flag from FIFO. */ + /* TODO 2: Read flag from the FIFO. */ rc = rread(fd, buffer, BUFSIZ); DIE(rc < 0, "rread"); diff --git a/chapters/io/ipc/drills/tasks/receive-challenges/solution/src/receive_net_dgram_socket.c b/chapters/io/ipc/drills/tasks/receive-challenges/solution/src/receive_net_dgram_socket.c index 70e14ee8ec..1f142e2915 100644 --- a/chapters/io/ipc/drills/tasks/receive-challenges/solution/src/receive_net_dgram_socket.c +++ b/chapters/io/ipc/drills/tasks/receive-challenges/solution/src/receive_net_dgram_socket.c @@ -46,13 +46,13 @@ int main(void) rc = bind(fd, (struct sockaddr *)&addr, sizeof(addr)); DIE(rc < 0, "bind"); - /* TODO 2: Read flag from socket. */ + /* TODO 2: Read flag from the socket. */ rc = recvfrom(fd, buffer, BUFSIZ, 0, NULL, NULL); DIE(rc < 0, "recvfrom"); printf("Flag is: %s\n", buffer); - /* TODO 2: Close socket. */ + /* TODO 2: Close the socket. */ rc = close(fd); DIE(rc < 0, "close"); diff --git a/chapters/io/ipc/drills/tasks/receive-challenges/solution/src/receive_pipe.c b/chapters/io/ipc/drills/tasks/receive-challenges/solution/src/receive_pipe.c index b6c3884a1f..95c5eded48 100644 --- a/chapters/io/ipc/drills/tasks/receive-challenges/solution/src/receive_pipe.c +++ b/chapters/io/ipc/drills/tasks/receive-challenges/solution/src/receive_pipe.c @@ -53,7 +53,7 @@ int main(void) printf("Flag is: %s\n", buf); - /* Wait for child process. */ + /* Wait for the child process. */ wait(NULL); return 0; diff --git a/chapters/io/ipc/drills/tasks/receive-challenges/solution/src/receive_unix_socket.c b/chapters/io/ipc/drills/tasks/receive-challenges/solution/src/receive_unix_socket.c index f41a6f06cf..b576e82969 100644 --- a/chapters/io/ipc/drills/tasks/receive-challenges/solution/src/receive_unix_socket.c +++ b/chapters/io/ipc/drills/tasks/receive-challenges/solution/src/receive_unix_socket.c @@ -56,7 +56,7 @@ int main(void) connectfd = accept(listenfd, NULL, NULL); DIE(connectfd < 0, "accept"); - /* TODO 2: Read flag from socket. */ + /* TODO 2: Read flag from the socket. */ rc = read(connectfd, buffer, BUFSIZ); DIE(rc < 0, "read"); diff --git a/chapters/io/ipc/drills/tasks/receive-challenges/solution/src/send_fifo.c b/chapters/io/ipc/drills/tasks/receive-challenges/solution/src/send_fifo.c index d597d6d64c..f641fb4745 100644 --- a/chapters/io/ipc/drills/tasks/receive-challenges/solution/src/send_fifo.c +++ b/chapters/io/ipc/drills/tasks/receive-challenges/solution/src/send_fifo.c @@ -15,15 +15,15 @@ int main(void) int rc; int fd; - /* FIFO must exist. It must be created by receiver. */ + /* The FIFO must exist. It must be created by receiver. */ rc = access(fifo_path, R_OK | W_OK); DIE(rc < 0, "access"); - /* Open FIFO. */ + /* Open the FIFO. */ fd = open(fifo_path, O_RDWR); DIE(fd < 0, "open"); - /* Write flag to FIFO. */ + /* Write flag to the FIFO. */ rc = write(fd, FLAG, sizeof(FLAG)); DIE(rc < 0, "write"); diff --git a/chapters/io/ipc/drills/tasks/receive-challenges/solution/src/send_net_dgram_socket.c b/chapters/io/ipc/drills/tasks/receive-challenges/solution/src/send_net_dgram_socket.c index 817e70f0a3..7add879652 100644 --- a/chapters/io/ipc/drills/tasks/receive-challenges/solution/src/send_net_dgram_socket.c +++ b/chapters/io/ipc/drills/tasks/receive-challenges/solution/src/send_net_dgram_socket.c @@ -19,7 +19,7 @@ int main(void) int fd; struct sockaddr_in raddr; - /* Create socket. */ + /* Create a socket. */ fd = socket(PF_INET, SOCK_DGRAM, 0); DIE(fd < 0, "socket"); @@ -29,7 +29,7 @@ int main(void) raddr.sin_port = htons(remote_port); raddr.sin_addr.s_addr = htonl(INADDR_ANY); - /* Write flag to socket. */ + /* Write flag to the socket. */ rc = sendto(fd, FLAG, sizeof(FLAG), 0, (struct sockaddr *) &raddr, sizeof(raddr)); DIE(rc < 0, "sendto"); diff --git a/chapters/io/ipc/drills/tasks/receive-challenges/solution/src/send_unix_socket.c b/chapters/io/ipc/drills/tasks/receive-challenges/solution/src/send_unix_socket.c index 045db158e8..a4dd9567b1 100644 --- a/chapters/io/ipc/drills/tasks/receive-challenges/solution/src/send_unix_socket.c +++ b/chapters/io/ipc/drills/tasks/receive-challenges/solution/src/send_unix_socket.c @@ -20,11 +20,11 @@ int main(void) int fd; struct sockaddr_un addr; - /* UNIX socket must exist. It must be created by receiver. */ + /* The UNIX socket must exist. It must be created by receiver. */ rc = access(socket_path, R_OK | W_OK); DIE(rc < 0, "access"); - /* Create socket. */ + /* Create a socket. */ fd = socket(PF_UNIX, SOCK_STREAM, 0); DIE(fd < 0, "open"); @@ -35,7 +35,7 @@ int main(void) rc = connect(fd, (struct sockaddr *) &addr, sizeof(addr)); DIE(rc < 0, "connect"); - /* Write flag to socket. */ + /* Write flag to the socket. */ rc = write(fd, FLAG, sizeof(FLAG)); DIE(rc < 0, "write"); diff --git a/chapters/io/ipc/drills/tasks/unix-socket/README.md b/chapters/io/ipc/drills/tasks/unix-socket/README.md index 0b45e16e62..356d5c5b0b 100644 --- a/chapters/io/ipc/drills/tasks/unix-socket/README.md +++ b/chapters/io/ipc/drills/tasks/unix-socket/README.md @@ -1,18 +1,20 @@ # UNIX Socket Communication -Navigate to `chapters/io/ipc/drills/tasks/unix-socket` and run `make` to generate the `support` directory. -In this exercise, you'll implement client-server communication between two processes using a **UNIX socket**. +Navigate to `unix-socket/` directory of the extracted archive (or `chapters/io/ipc/drills/tasks/unix-socket` if you are working directly in the repository). +Run `make` and then enter `support/` folder and go through the practice items below. +In this exercise, you'll implement client-server communication between two processes using **UNIX sockets**. Both the sender and receiver are created from the same binary: run without arguments for a receiver, or with `-s` for a sender. 1. Complete the TODOs in the `sender_loop()`. You need to verify whether the socket exists i.e. check if the receiver has created it. Next, **create** your own socket and **connect** to the receiver's socket using its address (**Hint:** use `get_sockaddr(` to obtain it). - Once the connection is established, you can send messages using `send()`. + Once the connection is established, you can send messages using [`send()`](https://man7.org/linux/man-pages/man2/send.2.html). 1. Complete the TODOs in the `receiver_loop()`. Similarly, you will need to **create** a socket and **bind** it to the receiver's address (**Hint:** use `get_sockaddr(` for this). Instead of connecting, you will **listen** for and **accept** incoming connections. - When `accept()` receives a connection request, it will return a new socket file descriptor that you can use to receive messages via `recv()`. + When [`accept()`](https://man7.org/linux/man-pages/man2/accept.2.html) receives a connection request, it will return a new socket file descriptor that you can use to receive messages via [`recv()`](https://man7.org/linux/man-pages/man2/recv.2.html). + All sockets should be closed after their designated task has been completed. 1. Inside the `tests/` directory, you will need to run `checker.sh`. The output for a successful implementation should look like this: diff --git a/chapters/io/ipc/drills/tasks/unix-socket/solution/src/unix_socket.c b/chapters/io/ipc/drills/tasks/unix-socket/solution/src/unix_socket.c index 1b3f6a26c2..f1b58ce988 100644 --- a/chapters/io/ipc/drills/tasks/unix-socket/solution/src/unix_socket.c +++ b/chapters/io/ipc/drills/tasks/unix-socket/solution/src/unix_socket.c @@ -47,7 +47,7 @@ static void receiver_loop(void) rc = bind(listenfd, (struct sockaddr *) &addr, sizeof(addr)); DIE(rc < 0, "bind"); - /* TODO 2: Mark socket as passive socket using listen(). */ + /* TODO 2: Mark the socket as passive using listen(). */ rc = listen(listenfd, 1); DIE(rc < 0, "listen"); @@ -82,7 +82,7 @@ static void sender_loop(void) int sockfd; int rc; - /* TODO 2: Check if socket exists. Hint: access(). */ + /* TODO 2: Check if the socket exists. Hint: access(). */ rc = access(socket_path, R_OK | W_OK); DIE(rc < 0, "access"); @@ -110,13 +110,13 @@ static void sender_loop(void) DIE(rc < 0, "send"); } - /* TODO 2: Close socket. */ + /* TODO 2: Close the socket. */ rc = close(sockfd); DIE(rc < 0, "close"); } /** - * Simulate a sender-receiver communication using a named pipe. + * Simulate a sender-receiver communication using UNIX sockets. * Run the program as a receiver by default, or as a sender if the -s or --sender. */ int main(int argc, char *argv[]) diff --git a/chapters/io/optimizations/drills/tasks/async-server/README.md b/chapters/io/optimizations/drills/tasks/async-server/README.md index bdc55347f1..ea65d4e0f1 100644 --- a/chapters/io/optimizations/drills/tasks/async-server/README.md +++ b/chapters/io/optimizations/drills/tasks/async-server/README.md @@ -1,19 +1,21 @@ # Async Server -Navigate to `chapters/io/optimizations/drills/tasks/async-server` and run `make` to generate the `support` files. -Enter `support` and run `make test-file.txt` to generate the test file. +Navigate to `async-server` directory of the extracted archive (or `chapters/io/optimizations/drills/tasks/async-server` if you are working directly in the repository) and run `make` to generate the `support` files. Enter `support` and run `make test-file.txt` to generate the test file. This task builds on the previous example of a [multiplexed client-server](../../tasks/multiplexed-client-server/README.md). -The server accepts connections from clients and downloads a file of `1 GB` from each. +The server accepts connections from clients and downloads a file of `100MB` from each. After uploading the file, the clients close the connection. -1. Open `server.c` and complete the TODOs in the main function to setup IO multiplexing using [`epoll`](https://man7.org/linux/man-pages/man7/epoll.7.html). +1. Look at the code in `support` and open `server.c` and complete the TODOs in the main function to setup IO multiplexing using [`epoll`](https://man7.org/linux/man-pages/man7/epoll.7.html). Use `epoll_create()`, `epoll_wait()`, and the wrappers defined in `w_epoll.h` to handle descriptors without blocking. **Remember** to remove the client sockets from the `epoll` instance before closing them. To test, run `./server` in one terminal and `./client` in another terminal.s If successful, the clients should print the upload progress. + To run the checker, run `make check` from the `support` file. You may have to wait a little bit for the checker to be finish its execution. + In case of the checker failing, check out client and server output in the `client_output.log`, `server_output.log` in the `support` file. + 1. There is a problem with our current implementation. Try to start two clients at the same time - the first one will start uploading, and the second one will block at `connect()`. This happens because, even though we are multiplexing file descriptors on the server-side, it is busy with another client. diff --git a/chapters/io/optimizations/drills/tasks/async-server/solution/Makefile b/chapters/io/optimizations/drills/tasks/async-server/solution/Makefile index a95c0a809d..a9dd99e195 100644 --- a/chapters/io/optimizations/drills/tasks/async-server/solution/Makefile +++ b/chapters/io/optimizations/drills/tasks/async-server/solution/Makefile @@ -35,6 +35,11 @@ clean: -rm -f $(LOGGER) test-file.txt: - dd if=/dev/urandom of=test-file.txt bs=1024 count=1M + dd if=/dev/urandom of=test-file.txt bs=1M count=100 + cp test-file.txt ../solution/test-file.txt + +check: + cd ../tests && \ + ./check_server.sh .PHONY: all clean diff --git a/chapters/io/optimizations/drills/tasks/async-server/solution/client.c b/chapters/io/optimizations/drills/tasks/async-server/solution/client.c index 37585ff42c..1a65eb9af1 100644 --- a/chapters/io/optimizations/drills/tasks/async-server/solution/client.c +++ b/chapters/io/optimizations/drills/tasks/async-server/solution/client.c @@ -31,9 +31,9 @@ struct sockaddr_in get_sockaddr(const char *ip, const int port) } /** - * Read 1GB file and send it to the server. + * Read 100MB file and send it to the server. */ -static void send_1GB_file(int sockfd, const char *filename) +static void send_100MB_file(int sockfd, const char *filename) { char buf[BUFSIZ]; FILE *file; @@ -60,7 +60,7 @@ static void send_1GB_file(int sockfd, const char *filename) printf("Sent %lu MB\n", total / (1024 * 1024)); } - printf("Sent 1GB file to server\n"); + printf("Sent 100MB file to server\n"); rc = fclose(file); DIE(rc < 0, "fclose"); @@ -81,7 +81,7 @@ int main(void) rc = connect(sockfd, (struct sockaddr *) &addr, sizeof(addr)); DIE(rc < 0, "connect"); - send_1GB_file(sockfd, "test-file.txt"); + send_100MB_file(sockfd, "test-file.txt"); rc = close(sockfd); DIE(rc < 0, "close"); diff --git a/chapters/io/optimizations/drills/tasks/async-server/solution/server.c b/chapters/io/optimizations/drills/tasks/async-server/solution/server.c index dc7e7f981b..70ef632adf 100644 --- a/chapters/io/optimizations/drills/tasks/async-server/solution/server.c +++ b/chapters/io/optimizations/drills/tasks/async-server/solution/server.c @@ -36,10 +36,10 @@ struct sockaddr_in get_sockaddr(const char *ip, const int port) } /** - * Get a 1GB file over the network. + * Get a 100MB file over the network. * No need to store the data, just receive it to simulate a real-world scenario. */ -static void receive_1GB_file(int sockfd) +static void receive_100MB_file(int sockfd) { char buf[SMALL_BUF]; int rc; @@ -52,7 +52,7 @@ static void receive_1GB_file(int sockfd) break; } - printf("Received 1GB file from [Client %d]\n", sockfd); + printf("Received 100MB file from [Client %d]\n", sockfd); } /** @@ -69,13 +69,13 @@ void handle_client(int epollfd, int sockfd) /* TODO 5: Print the child process id and handle the client */ printf("<< Process %d created to handle client %d >>\n", pid, sockfd); if (pid == 0) { /* Child process */ - receive_1GB_file(sockfd); + receive_100MB_file(sockfd); exit(EXIT_SUCCESS); } /* REPLACE 2*/ /* // Remove this after implementing the child process */ - /* receive_1GB_file(sockfd); */ + /* receive_100MB_file(sockfd); */ /* TODO 2: Remove the client from epoll */ rc = w_epoll_del_fd(epollfd, sockfd); diff --git a/chapters/io/optimizations/drills/tasks/async-server/tests/check_server.sh b/chapters/io/optimizations/drills/tasks/async-server/tests/check_server.sh new file mode 100755 index 0000000000..30253f8fa4 --- /dev/null +++ b/chapters/io/optimizations/drills/tasks/async-server/tests/check_server.sh @@ -0,0 +1,90 @@ +#!/bin/bash +# SPDX-License-Identifier: BSD-3-Clause + +TEST_FILE="../support/test-file.txt" +if [ ! -f "$TEST_FILE" ]; then + echo "run make test-file.txt in the `support/` folder to generate the test file" + exit 1 +fi + +SUPPORT_DIR="../support" +SOLUTION_DIR="../solution" +PORT=5000 + +SERVER_BIN="$SUPPORT_DIR/server" +SERVER_LOG="$SUPPORT_DIR/server_output.log" +CLIENT_LOG="$SUPPORT_DIR/client_output.log" +REF_CONFIRMATION="Received 100MB file from" + +make -C "$SUPPORT_DIR" > /dev/null 2>&1 +make -C "$SOLUTION_DIR" > /dev/null 2>&1 + +rm -f "$SERVER_LOG" "$CLIENT_LOG" + +# Kill existing processes using port +PID_LIST=$(lsof -ti tcp:$PORT) +if [ -n "$PID_LIST" ]; then + echo "Killing previous server processes on port $PORT: $PID_LIST" + lsof -ti tcp:$PORT | xargs kill -9 + sleep 1 +fi + +# Start the server with line-buffered output +setsid timeout --kill-after=5s 300s stdbuf -oL "$SERVER_BIN" > "$SERVER_LOG" 2>&1 & +SERVER_PID=$! + +# Allow server to bind port +sleep 1 + +# Run client with live output in terminal and capture to log +( + timeout 300s bash -c "(cd $SOLUTION_DIR && stdbuf -oL ./client)" 2>&1 | tee "$CLIENT_LOG" +) & +CLIENT_PID=$! + +# Monitor client output log for "Sent 10 MB" within 10 seconds +timeout 10 bash -c "tail -n +0 -F $CLIENT_LOG | grep -q 'Sent 10 MB'" +grep_status=$? + +if [ $grep_status -ne 0 ]; then + echo "Client did not send first 10 MB within 10 seconds. Failing test." + kill -9 $CLIENT_PID 2>/dev/null || true + kill -9 $SERVER_PID 2>/dev/null || true + exit 1 +fi + +wait $CLIENT_PID + +# Wait for server to finish (up to 30 seconds) +MAX_WAIT=30 +elapsed=0 +interval=0.1 +while kill -0 $SERVER_PID 2>/dev/null; do + if (( $(echo "$elapsed >= $MAX_WAIT" | bc -l) )); then + echo "Timeout waiting for server to exit" + break + fi + sleep $interval + elapsed=$(echo "$elapsed + $interval" | bc) +done + +echo "===== server output =====" +cat "$SERVER_LOG" +echo "===============================================" + +if grep -Fq "$REF_CONFIRMATION" "$SERVER_LOG"; then + echo "Server test .......................... passed ... 100" + RESULT=0 +else + echo "Server test .......................... failed ... 0" + echo "Did not find confirmation message:" + echo "\"$REF_CONFIRMATION\"" + tail -20 "$SERVER_LOG" + RESULT=1 +fi + +make -C "$SUPPORT_DIR" clean > /dev/null 2>&1 +make -C "$SOLUTION_DIR" clean > /dev/null 2>&1 +rm -f "$SERVER_LOG" "$CLIENT_LOG" + +exit $RESULT diff --git a/chapters/io/optimizations/drills/tasks/multiplexed-client-server/README.md b/chapters/io/optimizations/drills/tasks/multiplexed-client-server/README.md index e161bf3eb0..b9a130d96a 100644 --- a/chapters/io/optimizations/drills/tasks/multiplexed-client-server/README.md +++ b/chapters/io/optimizations/drills/tasks/multiplexed-client-server/README.md @@ -1,6 +1,6 @@ # Multiplexed Client Server -Navigate to `chapters/io/optimizations/drills/tasks/multiplexed-client-server` and run `make` to generate the `support` files. +Navigate to `multiplexed-client-server` directory of the archive, (or `chapters/io/optimizations/drills/tasks/multiplexed-client-server` if you are working directly in the repository) and run `make` to generate the `support` files. This task builds on the previous implementation of a [client-server ordered communication](../../../../ipc/drills/tasks/client-server/README.md). Previously, the client and server followed a strict, sequential communication pattern: each peer had to send a message and wait for a reply before proceeding. diff --git a/chapters/io/overview/reading/lab10.md b/chapters/io/overview/reading/lab10.md new file mode 100644 index 0000000000..f2ea3f40d3 --- /dev/null +++ b/chapters/io/overview/reading/lab10.md @@ -0,0 +1 @@ +The contents of the lab are located in the [lab archive](https://github.com/cs-pub-ro/operating-systems/raw/refs/heads/lab-archives/Lab_10_Inter_Process_Communication.zip) and in the [GitHub repository](https://github.com/cs-pub-ro/operating-systems). diff --git a/chapters/io/overview/reading/lab9.md b/chapters/io/overview/reading/lab9.md new file mode 100644 index 0000000000..2cb050b78b --- /dev/null +++ b/chapters/io/overview/reading/lab9.md @@ -0,0 +1 @@ +The contents of the lab are located in the [lab archive](https://github.com/cs-pub-ro/operating-systems/raw/refs/heads/lab-archives/Lab_9_File_Descriptors.zip) and in the [GitHub repository](https://github.com/cs-pub-ro/operating-systems). diff --git a/chapters/io/overview/reading/overview.md b/chapters/io/overview/reading/overview.md index c67efa8a97..126260673b 100644 --- a/chapters/io/overview/reading/overview.md +++ b/chapters/io/overview/reading/overview.md @@ -1,5 +1,7 @@ # I/O +The contents of the lab are located in the [lab archive](https://github.com/cs-pub-ro/operating-systems/raw/refs/heads/lab-archives/Lab_11_IO_Optimizations.zip) and in the [GitHub repository](https://github.com/cs-pub-ro/operating-systems). + We know that a compute system is a collection of hardware and software that modifies data. This data has to come from _somewhere_. This _somewhere_ is always outside the compute system: files, network packets, radio signals, sensor data. diff --git a/chapters/software-stack/high-level-languages/drills/tasks/high-level-lang/README.md b/chapters/software-stack/high-level-languages/drills/tasks/high-level-lang/README.md index 50c0fe632a..e89973d255 100644 --- a/chapters/software-stack/high-level-languages/drills/tasks/high-level-lang/README.md +++ b/chapters/software-stack/high-level-languages/drills/tasks/high-level-lang/README.md @@ -1,6 +1,6 @@ # High-Level Languages -Enter the `high-level-lang/` directory from the lab archive (or `chapters/software-stack/high-level-languages/drills/tasks/high-level-lang/` if you are working directly in the repository), run `make skels`, then enter `spport/` +Enter the `high-level-lang/` directory from the lab archive (or `chapters/software-stack/high-level-languages/drills/tasks/high-level-lang/` if you are working directly in the repository), run `make skels`, then enter `support/` Then go through the practice items below. 1. Use `make` to create the `hello` executable from the `hello.go` file (a Go "Hello, World!"-printing program). diff --git a/chapters/software-stack/high-level-languages/drills/tasks/high-level-lang/generate_skels.py b/chapters/software-stack/high-level-languages/drills/tasks/high-level-lang/generate_skels.py index 697c9d5b61..4f2f9286c0 100644 --- a/chapters/software-stack/high-level-languages/drills/tasks/high-level-lang/generate_skels.py +++ b/chapters/software-stack/high-level-languages/drills/tasks/high-level-lang/generate_skels.py @@ -110,6 +110,7 @@ def main(): or re.match(r".*\.sh$", src) or re.match(r".*\.[sS]$", src) or re.match(r".*\.py$", src) + or re.match(r".*\.go$", src) ): pattern = r"(^\s*#\s*TODO)( [0-9]*)(:.*)" replace = r"(^\s*#\s*REPLACE)( [0-9]*)" diff --git a/chapters/software-stack/libc/drills/tasks/common-functions/solution/src/main_printf.c b/chapters/software-stack/libc/drills/tasks/common-functions/solution/src/main_printf.c index 3cd265af3b..eaeff7daa2 100644 --- a/chapters/software-stack/libc/drills/tasks/common-functions/solution/src/main_printf.c +++ b/chapters/software-stack/libc/drills/tasks/common-functions/solution/src/main_printf.c @@ -11,7 +11,7 @@ static char dest[128]; static char out_buffer[256]; static char putchar_buffer[1024]; -static char putchar_buffer_len; +static int putchar_buffer_len; /* REMOVE 5 */ void flush(void) diff --git a/chapters/software-stack/libc/drills/tasks/common-functions/solution/src/syscall.s b/chapters/software-stack/libc/drills/tasks/common-functions/solution/src/syscall.s index 1e1c6ab84a..93647c320e 100644 --- a/chapters/software-stack/libc/drills/tasks/common-functions/solution/src/syscall.s +++ b/chapters/software-stack/libc/drills/tasks/common-functions/solution/src/syscall.s @@ -1,4 +1,4 @@ -/* SPDX-License-Identifier: BSD-3-Clause */ +; SPDX-License-Identifier: BSD-3-Clause section .text diff --git a/chapters/software-stack/libc/drills/tasks/libc/solution/main_printf.c b/chapters/software-stack/libc/drills/tasks/libc/solution/main_printf.c index d88f1e52d0..6128ee0dac 100644 --- a/chapters/software-stack/libc/drills/tasks/libc/solution/main_printf.c +++ b/chapters/software-stack/libc/drills/tasks/libc/solution/main_printf.c @@ -8,14 +8,14 @@ static char dest[128]; int main(void) { - printf("[before] src is at %p, len is %lu, content: \"%s\"\n", src, os_strlen(src), src); - printf("[before] dest is at %p, len is %lu, content: \"%s\"\n", dest, os_strlen(dest), dest); + printf("[before] src is at %p, len is %lu, content: \"%s\"\n", src, strlen(src), src); + printf("[before] dest is at %p, len is %lu, content: \"%s\"\n", dest, strlen(dest), dest); printf("copying src to dest\n"); - os_strcpy(dest, src); + strcpy(dest, src); - printf("[after] src is at %p, len is %lu, content: \"%s\"\n", src, os_strlen(src), src); - printf("[after] dest is at %p, len is %lu, content: \"%s\"\n", dest, os_strlen(dest), dest); + printf("[after] src is at %p, len is %lu, content: \"%s\"\n", src, strlen(src), src); + printf("[after] dest is at %p, len is %lu, content: \"%s\"\n", dest, strlen(dest), dest); printf("a"); printf("b"); diff --git a/chapters/software-stack/libc/drills/tasks/libc/support/Makefile b/chapters/software-stack/libc/drills/tasks/libc/support/Makefile deleted file mode 100644 index 669ca95a04..0000000000 --- a/chapters/software-stack/libc/drills/tasks/libc/support/Makefile +++ /dev/null @@ -1,23 +0,0 @@ -CFLAGS = -Wall -LDFLAGS = -static - -.PHONY: all clean - -all: hello main_printf main_string memory vendetta - -hello: hello.o - -main_printf: main_printf.o - -main_string: main_string.o - -memory: memory.o - -vendetta: vendetta.o - -clean: - -rm -f hello hello.o - -rm -f main_printf main_printf.o - -rm -f main_string main_string.o - -rm -f memory memory.o - -rm -f vendetta vendetta.o diff --git a/chapters/software-stack/libc/drills/tasks/libc/support/hello.c b/chapters/software-stack/libc/drills/tasks/libc/support/hello.c deleted file mode 100644 index 960fc0d963..0000000000 --- a/chapters/software-stack/libc/drills/tasks/libc/support/hello.c +++ /dev/null @@ -1,20 +0,0 @@ -// SPDX-License-Identifier: BSD-3-Clause - -#include - -int main(void) -{ - char buffer[128]; - ssize_t nread; - - write(1, "Hello, world!\n", 14); - write(1, "Bye, world!\n", 12); - - nread = read(0, buffer, 128); - if (nread > 0) - write(1, buffer, nread); - - pause(); - - return 0; -} diff --git a/chapters/software-stack/libc/drills/tasks/libc/support/main_printf.c b/chapters/software-stack/libc/drills/tasks/libc/support/main_printf.c deleted file mode 100644 index d88f1e52d0..0000000000 --- a/chapters/software-stack/libc/drills/tasks/libc/support/main_printf.c +++ /dev/null @@ -1,26 +0,0 @@ -// SPDX-License-Identifier: BSD-3-Clause - -#include -#include "string.h" - -static const char src[] = "warhammer40k"; -static char dest[128]; - -int main(void) -{ - printf("[before] src is at %p, len is %lu, content: \"%s\"\n", src, os_strlen(src), src); - printf("[before] dest is at %p, len is %lu, content: \"%s\"\n", dest, os_strlen(dest), dest); - - printf("copying src to dest\n"); - os_strcpy(dest, src); - - printf("[after] src is at %p, len is %lu, content: \"%s\"\n", src, os_strlen(src), src); - printf("[after] dest is at %p, len is %lu, content: \"%s\"\n", dest, os_strlen(dest), dest); - - printf("a"); - printf("b"); - fflush(stdout); - printf("c\n"); - - return 0; -} diff --git a/chapters/software-stack/libc/drills/tasks/libc/support/main_string.c b/chapters/software-stack/libc/drills/tasks/libc/support/main_string.c deleted file mode 100644 index 9113b0a190..0000000000 --- a/chapters/software-stack/libc/drills/tasks/libc/support/main_string.c +++ /dev/null @@ -1,17 +0,0 @@ -// SPDX-License-Identifier: BSD-3-Clause - -#include -#include - -static const char src[] = "warhammer40k\n"; -static char dest[128]; - -int main(void) -{ - strcpy(dest, src); - - write(1, "Destination string is: ", 23); - write(1, dest, strlen(dest)); - - return 0; -} diff --git a/chapters/software-stack/libc/drills/tasks/libc/support/memory.c b/chapters/software-stack/libc/drills/tasks/libc/support/memory.c deleted file mode 100644 index 4758e3163d..0000000000 --- a/chapters/software-stack/libc/drills/tasks/libc/support/memory.c +++ /dev/null @@ -1,17 +0,0 @@ -// SPDX-License-Identifier: BSD-3-Clause - -#include - -int main(void) -{ - void *p; - - /* TODO: Use malloc() to allocate memory of different sizes. - * Store result in p. - * Use free() to deallocate memory. - * Use ltrace and strace command line tools to monitor library - * and system calls. - */ - - return 0; -} diff --git a/chapters/software-stack/libc/drills/tasks/libc/support/vendetta.c b/chapters/software-stack/libc/drills/tasks/libc/support/vendetta.c deleted file mode 100644 index 65cd97de52..0000000000 --- a/chapters/software-stack/libc/drills/tasks/libc/support/vendetta.c +++ /dev/null @@ -1,27 +0,0 @@ -// SPDX-License-Identifier: BSD-3-Clause - -#include -#include -#include -#include -#include - -int main(void) -{ - /* - * TODO 12: Use standard C library functions. - * Be as creative as you can. - */ - int fd; - FILE *f; - - fd = open("a.txt", O_RDWR | O_CREAT, 0644); - close(fd); - - f = fopen("a.txt", "w"); - fclose(f); - - printf("sin(0): %f, sin(PI/2): %f\n", sin(0), sin(M_PI/2)); - - return 0; -} diff --git a/chapters/software-stack/system-calls/drills/questions/syscall-numbers.md b/chapters/software-stack/system-calls/drills/questions/syscall-numbers.md index 118831ea0a..b12d50b3a9 100644 --- a/chapters/software-stack/system-calls/drills/questions/syscall-numbers.md +++ b/chapters/software-stack/system-calls/drills/questions/syscall-numbers.md @@ -16,4 +16,4 @@ What is the approximate number of system call numbers in Linux? ## Feedback -As show [here](https://x64.syscall.sh/), they're about 300 system calls in Linux. +As shown [here](https://x64.syscall.sh/), there are about 300 system calls in Linux. diff --git a/chapters/software-stack/system-calls/drills/tasks/basic-syscall/solution/src/arm/hello.s b/chapters/software-stack/system-calls/drills/tasks/basic-syscall/solution/src/arm/hello.s index a74ab5aae3..7efa3df8a4 100644 --- a/chapters/software-stack/system-calls/drills/tasks/basic-syscall/solution/src/arm/hello.s +++ b/chapters/software-stack/system-calls/drills/tasks/basic-syscall/solution/src/arm/hello.s @@ -1,4 +1,4 @@ -/* SPDX-License-Identifier: BSD-3-Clause */ +; SPDX-License-Identifier: BSD-3-Clause .section .bss diff --git a/chapters/software-stack/system-calls/drills/tasks/basic-syscall/solution/src/hello.asm b/chapters/software-stack/system-calls/drills/tasks/basic-syscall/solution/src/hello.asm index 0acee2548d..dc45647f1a 100644 --- a/chapters/software-stack/system-calls/drills/tasks/basic-syscall/solution/src/hello.asm +++ b/chapters/software-stack/system-calls/drills/tasks/basic-syscall/solution/src/hello.asm @@ -1,4 +1,4 @@ -/* SPDX-License-Identifier: BSD-3-Clause */ +; SPDX-License-Identifier: BSD-3-Clause section .bss diff --git a/chapters/software-stack/system-calls/drills/tasks/basic-syscall/solution/src/hello.s b/chapters/software-stack/system-calls/drills/tasks/basic-syscall/solution/src/hello.s index fc27e315b9..d8b6375ff7 100644 --- a/chapters/software-stack/system-calls/drills/tasks/basic-syscall/solution/src/hello.s +++ b/chapters/software-stack/system-calls/drills/tasks/basic-syscall/solution/src/hello.s @@ -1,4 +1,4 @@ -/* SPDX-License-Identifier: BSD-3-Clause */ +; SPDX-License-Identifier: BSD-3-Clause .section .bss .lcomm buffer, 128 diff --git a/chapters/software-stack/system-calls/drills/tasks/syscall-wrapper/Makefile b/chapters/software-stack/system-calls/drills/tasks/syscall-wrapper/Makefile index b2411412bd..d688e09d00 100644 --- a/chapters/software-stack/system-calls/drills/tasks/syscall-wrapper/Makefile +++ b/chapters/software-stack/system-calls/drills/tasks/syscall-wrapper/Makefile @@ -4,7 +4,7 @@ SCRIPT = generate_skels.py skels: mkdir -p support/src $(PYTHON) $(SCRIPT) --input ./solution/src --output ./support/src - $(PYTHON) $(SCRIPT) --input ./solution/tests --output ./support/src/tests + cp -r ./solution/tests ./support/tests clean: rm -rf support/ diff --git a/chapters/software-stack/system-calls/drills/tasks/syscall-wrapper/solution/src/main.c b/chapters/software-stack/system-calls/drills/tasks/syscall-wrapper/solution/src/main.c index fb2aaa95c4..3a48a655dc 100644 --- a/chapters/software-stack/system-calls/drills/tasks/syscall-wrapper/solution/src/main.c +++ b/chapters/software-stack/system-calls/drills/tasks/syscall-wrapper/solution/src/main.c @@ -7,7 +7,7 @@ static void reverse_string(char *a, unsigned int len) { - /* TODO 9: reverse_string function */ + /* TODO 8: reverse_string function */ unsigned int i, j; char aux; @@ -20,7 +20,7 @@ static void reverse_string(char *a, unsigned int len) static unsigned int os_itoa(int n, char *a) { - /* TODO 20: itoa function */ + /* TODO 19: itoa function */ unsigned int num_digits = 0; int digit; diff --git a/chapters/software-stack/system-calls/drills/tasks/syscall-wrapper/solution/src/syscall.asm b/chapters/software-stack/system-calls/drills/tasks/syscall-wrapper/solution/src/syscall.asm index d697eb7f77..a8a53cb9d7 100644 --- a/chapters/software-stack/system-calls/drills/tasks/syscall-wrapper/solution/src/syscall.asm +++ b/chapters/software-stack/system-calls/drills/tasks/syscall-wrapper/solution/src/syscall.asm @@ -1,4 +1,4 @@ -/* SPDX-License-Identifier: BSD-3-Clause */ +; SPDX-License-Identifier: BSD-3-Clause section .text diff --git a/config.yaml b/config.yaml index a71d04b1f7..6951665ad0 100644 --- a/config.yaml +++ b/config.yaml @@ -82,6 +82,7 @@ lab_structure: - title: Lab 6 - Multiprocess and Multithread filename: lab6.md content: + - reading/lab6.md - tasks/sleepy.md - tasks/wait-for-me-processes.md - tasks/create-process.md @@ -111,6 +112,7 @@ lab_structure: - title: Lab 8 - Synchronization filename: lab8.md content: + - reading/lab8.md - tasks/race-condition.md - tasks/wrap-the-for.md - tasks/race-condition-atomic.md @@ -126,6 +128,7 @@ lab_structure: - title: Lab 9 - File Descriptors filename: lab9.md content: + - reading/lab9.md - tasks/my-cat.md - tasks/mmap_cp.md - tasks/anon-pipes.md @@ -141,6 +144,7 @@ lab_structure: - title: Lab 10 - Inter-Process Communication filename: lab10.md content: + - reading/lab10.md - tasks/named-pipes.md - tasks/unix-socket.md - tasks/network-socket.md @@ -387,6 +391,7 @@ docusaurus: subsections: - Mini Libc/: chapters/software-stack/libc/projects/mini-libc/ - Memory Allocator/: content/assignments/memory-allocator/ + - ELF Loader/: content/assignments/elf-loader/ - Parallel Firewall/: content/assignments/parallel-firewall/ - Mini Shell/: content/assignments/minishell/ - Asynchronous Web Server/: content/assignments/async-web-server/ diff --git a/content/assignments/elf-loader/README.md b/content/assignments/elf-loader/README.md new file mode 100644 index 0000000000..9b15191c2f --- /dev/null +++ b/content/assignments/elf-loader/README.md @@ -0,0 +1,368 @@ +# ELF Loader Assignment + +## Objecives + +* Practice working with virtual memory, memory protection, and manual relocation. +* Understand the difference between different types of executables, like PIE, non-PIE, statically-linked, etc. +* Understand the stack layout expected by an executable, environment variables, auxiliary vector, command-line arguments, etc. + +## Statement + +Implement a custom minimal ELF loader, capabale of loading and executing statically linked binaries in Linux. + +Your loader must eventually support: + +* Minimal static binaries that make direct Linux syscalls (without libc) +* Statically linked **non-PIE** C programs using `libc` +* Statically linked **PIE** executables + +## Support Code + +The support code consists of three directories: + +* `src/` where you will create your sollution +* `test/` contains the test suite and a bash script to verify your work + +The test suite consists of source code files (`.c` and `.asm`), that will be compiled and then executed using your loader. +You can use the `Makefile` to compile all test files. + +## Implementation + +The assignment is split into **4 graded parts**, totaling **90 points** (10 points are given by the linter): + +### 1. ELF header validation (**10 points**) + +**Goal:** Before loading the ELF file, check if it is a valid, 64-bit ELF. +You must check for 2 cases: + +* Check the [ELF magic](https://unix.stackexchange.com/questions/153352/what-is-elf-magic), defined [here](https://chromium.googlesource.com/external/elfutils/+/dts-0.168/libelf/elf.h#120). +* Check the [ELF class](https://chromium.googlesource.com/external/elfutils/+/dts-0.168/libelf/elf.h#123), it should be `ELFCLASS64`. + +In case any of the items above are wrong, print one of the following messages and exit with the corresponding error code: + +`Not a valid ELF file`, exit with code 3. + +or + +`Not a 64-bit ELF`, exit with code 4. + +### 2. Minimal loader for syscall-only binaries (**10 points**) + +**Goal:** Make the loader work with extremely minimal ELF binaries (usually written in assembly) that make direct syscalls and do not use libc. + +* All memory segments can be loaded with `RWX` permissions. +* No need to set up `argv`, `envp`, or auxiliary vectors. +* These binaries call syscall instructions directly, so `libc` is not used. + +For this task, you will need to: + +* Open the file and map it somewhere in the memory +* Pass through the section headers, and for the `PT_LOAD` sections create new memory regions (they can have RWX permissions for now), then copy the section from the file into the newly created memory region. +* Pass the execution to the new ELF, by jumping to the entry point. + +**Examples/Resources:** + +* [ELF Specification](https://refspecs.linuxbase.org/elf/gabi4+/ch5.pheader.html) +* [OSDev](https://wiki.osdev.org/ELF) + +### 3. Load memory regions with correct permissions (**10 points**) + +**Goal:** Instead of RWX, check the memory protection flags (`PF_R`, `PF_W`, `PF_X`) from the ELF `Program Headers`. + +* Use `mprotect()` or map with the correct permissions directly using `mmap()`. + +**Key Concepts:** + +* `PT_LOAD` program headers contain `p_flags` to specify memory permissions. +* These must be respected to mimic the kernel loader. +* [ELF Specification](https://refspecs.linuxbase.org/elf/gabi4+/ch5.pheader.html) + +### 4. Support static non-PIE binaries with libc (**30 points**) + +**Goal:** Load and run statically linked **non-PIE** C binaries compiled with libc (e.g., via `gcc -static`). + +* Must set up a valid process **stack**, including: + + * `argc`, `argv`, `envp` + * `auxv` vector (with entries like `AT_PHDR`, `AT_PHENT`, `AT_PHNUM`, etc.) + +For this, you need to map a new memory region, that will become the new stack, then copy all the required information there. + +The executable expects the stack layout as seen in the figure below: + +![Stack Layout](./img/stack-layout.drawio.svg) + +You can see more details about the stack [here](https://lwn.net/Articles/631631/). + +You will have to reserve a memory region large enough for the stack (you can use the maximum allowed stack size, using `getrlimit`, or you can use a hardcoded value large enough to fit everything). +After that, you need to copy the argc, argv and envp in the expected layout, then set up the auxv. + +**Note:** `argv` and `envp`, since they consist of strings, will be placed as the **pointer to the string** on the stack, not the string itself. +**Note:** Make sure the mapped regions have the correct length, **be careful of the difference between `p_filesz` and `p_memsz`**. + +#### argc, argv (5 points out of 30) + +The command-line arguments must be placed first at the top of the stack, as seen in the picture above. +The loader can be used as `./elf_loader ./no-pie-exec arg1 arg2 arg3`. +`arg1`, `arg2` and `arg3` must be placed on the stack for the loaded executable. +`argc` will be also placed on the at the top of the stack. + +#### envp (5 points out of 30) + +The environment variables should be placed after the command-line arguments. +For this, you just have to copy everything from the `char **envp` array and place it on the stack. + +#### auxv (10 points out of 30) + +The auxiliary vector, auxv, is a mechanism for communicating information from the kernel to user space. +It's basically a list of key-value pairs that contains different information about the state of the executable. +You can see the keys and required values of the auxv [in the man pages](https://man7.org/linux/man-pages/man3/getauxval.3.html). +For example, for the key `AT_PAGESZ` (defined as 6 in [elf.h](https://elixir.bootlin.com/glibc/glibc-2.42.9000/source/elf/elf.h#L1205)), that needs to contain the value of the page size, the memory will look as follows: + +```text +0xfff...... --> High Addresses +----------- + 4096 # Page Size + 6 # AT_PAGESZ key +----------- +----------- +0x000...... --> Low Addresses +``` + +The auvx must end with an `AT_NULL` key with a 0 value, so an auxv that sets `AT_PAGESZ`, `AT_UID` and `AT_NULL` will look like this on the stack: + +![Auxv Example](./img/auxv-example.drawio.svg) + +**Note:** Beware of the `AT_RANDOM` entry, the application will crash if you do not set it up properly. + +**Docs:** + +* [How programs get run: ELF binaries](https://lwn.net/Articles/631631/) (See section: `Populating the stack`) +* [auxv man page](https://man7.org/linux/man-pages/man3/getauxval.3.html) + +### 5. Support static PIE executables (**30 points**) + +**Goal:** Make your loader support static **PIE (Position Independent Executable)** binaries. + +* ELF type will be `ET_DYN`, and segments must be mapped at a **random base address**. +* Entry point and memory segment virtual addresses must be adjusted by the `load_base`. + +**Additional Requirements:** + +* Must still build a valid stack (`argc`, `argv`, `auxv`, etc.) +* Handle relocation of entry point and program headers correctly. + +You will need to load all the segments relative to a random base address +Beware of the auxv entries, some of them will need to be adjusted to the offset. + +**Docs:** + +* [What is a PIE binary?](https://eli.thegreenplace.net/2011/08/25/load-time-relocation-of-shared-libraries) +* [Example ELF Loader](https://0xc0ffee.netlify.app/osdev/22-elf-loader-p2) +* [Another ELF Loader Example](https://www.mgaillard.fr/2021/04/15/load-elf-user-mode.html) + +## Debugging + +Here are some useful tips and tools to debug your ELF loader: + +### General Tips + +* **Start simple**: First test with a syscall-only ELF binary (e.g., `write` + `exit`). +* **Use GDB**: Run `gdb ./elf_loader` and set breakpoints in the loader and inside the loaded ELF. + You can use `add-symbol-file path-to-elf start-address` to debug the libc entry and the elf execution with debugging symbols. +* **Check memory layout**: Print segment addresses and protections. You can use `pmap $(pidof elf-loader)` +* **Use PWNGDB**: Use [`PwnGDB`](https://github.com/pwndbg/pwndbg) or other similar plugins. They provide a lot of help during debugging. + +#### Useful Tools + +* `readelf -l -h your_binary` +* `objdump -d your_binary` +* `gdb ./elf_loader` +* `pmap $(pidof elf_loader)` + +#### Debugging Example + +Let's say the `no_pie` test fails with a segmentation fault, with no other messages printed. +In order to debug that, we must run `gdb ./src/elf-loader` and `run ./tests/snippets/no_pie`: + +```gdb +$rax : 0xcc0 +$rbx : 0x1 +$rcx : 0x0000000000427aee → 0xc7a777fffff0003d ("="?) +$rdx : 0x0 +$rsp : 0x00007ffff7df6bc0 → 0x0000000000401835 → 0x20ec8348e5894855 +$rbp : 0x00007ffff7df6c00 → 0x0000000000000000 +$rsi : 0x20 +$rdi : 0x0 +$rip : 0x0000000000403e7b → 0x894864c030028b48 +$r8 : 0x0 +$r9 : 0x00000000004a8480 → "glibc.malloc.mxfast" +$r10 : 0x53053053 +$r11 : 0x246 +$r12 : 0x00007ffff7df6c28 → 0x00007ffff7df7fe8 → "./tests/snippets/no_pie" +$r13 : 0x0 +$r14 : 0x00000000004aa000 → 0x00000000004582f0 → 0xffefc1c5fa1e0ff3 +$r15 : 0x00000000004004e8 → 0x0000000000000000 +$eflags: [ZERO carry PARITY adjust sign trap INTERRUPT direction overflow RESUME virtualx86 identification] +$cs: 0x33 $ss: 0x2b $ds: 0x00 $es: 0x00 $fs: 0x00 $gs: 0x00 +─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────── stack ──── +0x00007ffff7df6bc0│+0x0000: 0x0000000000401835 → 0x20ec8348e5894855 ← $rsp +0x00007ffff7df6bc8│+0x0008: 0x0000000000401710 → 0x8949ed31fa1e0ff3 +0x00007ffff7df6bd0│+0x0010: 0x0000000000000000 +0x00007ffff7df6bd8│+0x0018: 0x0000000000000002 +0x00007ffff7df6be0│+0x0020: 0x00007fffffffda98 → 0x00007fffffffde42 → "/home/stefan/projects/facultate/asist/elf-loader/a[...]" +0x00007ffff7df6be8│+0x0028: 0x00007fffffffdab0 → 0x00007fffffffdeb0 → "SHELL=/bin/bash" +0x00007ffff7df6bf0│+0x0030: 0x00007ffff7ff25e8 → 0x00007ffff7f42b60 → endbr64 +0x00007ffff7df6bf8│+0x0038: 0x0000000000000001 +───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────── code:x86:64 ──── + 0x403e6c mov edi, r13d + 0x403e6f call 0x428e80 + 0x403e74 mov rdx, QWORD PTR [rip+0xa5bcd] # 0x4a9a48 + → 0x403e7b mov rax, QWORD PTR [rdx] + 0x403e7e xor al, al + 0x403e80 mov QWORD PTR fs:0x28, rax + 0x403e89 cmp QWORD PTR [rip+0xa60f7], 0x0 # 0x4a9f88 + 0x403e91 je 0x403e9f + 0x403e93 call 0x0 +───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────── threads ──── +[#0] Id 1, Name: "elf-loader", stopped 0x403e7b in ?? (), reason: SIGSEGV +``` + +Note that during this tutorial we use [`gef`](https://github.com/hugsy/gef), that is almost identical to `pwngdb`. +We advise you to use `pwngdb`, there will be no difference in commands used. + +We can see that the program crashes somewhere with no code or debugging symbols attached, so we can assume it is inside the loaded ELF. +To test this, let's break before we jump to the program entry point. + +```gdb +$ break elf-loader.c:197 # This is the line for `__asm__ __volatile__`, it will be a different line for you +Breakpoint 1 at 0x7ffff7f437c8: file elf-loader.c, line 197. +$ run ./tests/snippets/no_pie +●→ 0x7ffff7f437c8 mov rax, QWORD PTR [rbp-0x1d0] + 0x7ffff7f437cf mov rdx, QWORD PTR [rbp-0x190] + 0x7ffff7f437d6 mov rsp, rax + 0x7ffff7f437d9 xor rbp, rbp + 0x7ffff7f437dc jmp rdx + 0x7ffff7f437de nop +───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────── source:elf-loader.c+197 ──── + 192 *(uint64_t *)sp = argc; + 193 + 194 void (*entry)() = base + (void (*)())ehdr->e_entry; + 195 + 196 // Transfer control +●→ 197 __asm__ __volatile__( + 198 "mov %0, %%rsp\n" + 199 "xor %%rbp, %%rbp\n" + 200 "jmp *%1\n" + 201 : + 202 : "r"(sp), "r"(entry) +───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────── threads ──── +[#0] Id 1, Name: "elf-loader", stopped 0x7ffff7f437c8 in load_and_run (), reason: BREAKPOINT +``` + +Now we do some `ni` to step with every instruction, until we reach some point where no c code is available anymore (after the `jmp rdx`). + +```gdb + 0x401707 pop rbp + 0x401708 ret + 0x401709 nop DWORD PTR [rax+0x0] + → 0x401710 endbr64 + 0x401714 xor ebp, ebp + 0x401716 mov r9, rdx + 0x401719 pop rsi + 0x40171a mov rdx, rsp + 0x40171d and rsp, 0xfffffffffffffff0 +───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────── threads ──── +[#0] Id 1, Name: "elf-loader", stopped 0x401710 in ?? (), reason: SINGLE STEP +``` + +We check the mapping of the memory: + +```gdb +$ vmmap +[ Legend: Code | Stack | Heap ] +Start End Offset Perm Path +0x0000000000400000 0x0000000000401000 0x0000000000000000 r-- +0x0000000000401000 0x000000000047f000 0x0000000000000000 r-x +0x000000000047f000 0x00000000004a5000 0x0000000000000000 r-- +0x00000000004a5000 0x00000000004b2000 0x0000000000000000 rwx +0x00007ffff7600000 0x00007ffff7e00000 0x0000000000000000 rw- +0x00007ffff7e72000 0x00007ffff7f33000 0x0000000000000000 r-- .../elf-loader/assignment-elf-loader/tests/snippets/no_pie +0x00007ffff7f33000 0x00007ffff7f35000 0x0000000000000000 r-- [vvar] +0x00007ffff7f35000 0x00007ffff7f37000 0x0000000000000000 r-- [vvar_vclock] +0x00007ffff7f37000 0x00007ffff7f39000 0x0000000000000000 r-x [vdso] +0x00007ffff7f39000 0x00007ffff7f42000 0x0000000000000000 r-- .../elf-loader/assignment-elf-loader/src/elf-loader +0x00007ffff7f42000 0x00007ffff7fc9000 0x0000000000009000 r-x .../elf-loader/assignment-elf-loader/src/elf-loader +0x00007ffff7fc9000 0x00007ffff7ff2000 0x0000000000090000 r-- .../elf-loader/assignment-elf-loader/src/elf-loader +0x00007ffff7ff2000 0x00007ffff7ff7000 0x00000000000b9000 r-- .../elf-loader/assignment-elf-loader/src/elf-loader +0x00007ffff7ff7000 0x00007ffff7ff9000 0x00000000000be000 rw- .../elf-loader/assignment-elf-loader/src/elf-loader +0x00007ffff7ff9000 0x00007ffff7fff000 0x0000000000000000 rw- +0x00007ffff7fff000 0x00007ffff8021000 0x0000000000000000 rw- [heap] +0x00007ffffffdd000 0x00007ffffffff000 0x0000000000000000 rw- [stack] +0xffffffffff600000 0xffffffffff601000 0x0000000000000000 --x [vsyscall] +``` + +Our current instruction is at address `0x401710`, which is inside a memory region allocated by `mmap` for the `no_pie` file, we can use `add-symbol-file`. +`add-symbol-file` expects the **start address of the `.text` section**, so let's get that. + +```bash +$ readelf -S tests/snippets/no_pie + +[...] + [ 7] .text PROGBITS **0000000000401100** 00001100 + 000000000007d880 0000000000000000 AX 0 0 64 + [ 8] .fini PROGBITS 000000000047e980 0007e980 +``` + +The `.text` address is the one placed inside `**`, `0x0000000000401100`. + +So, bach to `gdb`: + +```gdb +$ add-symbol-file tests/snippets/no_pie 0x0000000000401100 +add symbol table from file "tests/snippets/no_pie" at + .text_addr = 0x401100 +Reading symbols from tests/snippets/no_pie... +$ context +... + → 0x401710 <_start+0000> endbr64 + 0x401714 <_start+0004> xor ebp, ebp + 0x401716 <_start+0006> mov r9, rdx + 0x401719 <_start+0009> pop rsi +... +``` + +Now we can see where we are in the code of `no_pie`, the `_start` function. +Let's see where it crashes: + +```gdb + 0x403e74 <__libc_start_main_impl+0144> mov rdx, QWORD PTR [rip+0xa5bcd] # 0x4a9a48 <_dl_random+139145856> + → 0x403e7b <__libc_start_main_impl+014b> mov rax, QWORD PTR [rdx] + 0x403e7e <__libc_start_main_impl+014e> xor al, al + 0x403e80 <__libc_start_main_impl+0150> mov QWORD PTR fs:0x28, rax +``` + +**Note:** If the application crashes in the `__libc_start_main_impl` function, it's most likely because of the stack (`AUXV` values), or the memory layout (make sure the mapped regions have the correct length, **be careful of the difference between `p_filesz` and `p_memsz`**). + +In our case, we can see the `rdx` register, that is dereferenced, is 0. +We can also see on the instruction above, that `rdx` is set to `_dl_random+139145856`. + +The name of `_dl_random` suggests something to do with `auxv[AT_RANDOM]`. +Also, if we look into the libc source code (you are not required to do this, you should be able to solve all the issues using gdb, but sometimes looking at the source code helps), `_dl_random` [is actually set to `auxv[AT_RANDOM]`](https://elixir.bootlin.com/glibc/glibc-2.42.9000/source/sysdeps/unix/sysv/linux/dl-parse_auxv.h#L55). +We did not set the `AT_RANDOM` value in our loader, so it's `NULL`, which is why it will crash with a `SEGV`. + +We set the `AT_RANDOM` value to a memory region pointing to random data, as the [man page](https://man7.org/linux/man-pages/man3/getauxval.3.html) says, the crash disappears, and the `no_pie` elf is loaded properly. + +## Running the Checker + +In order to check the assignment in an environment as similar to the one on Gitlab CI, you can run the checker, including linters with: + +```console +student@so:~/.../assignments/elf-loader$ ./local.sh checker +``` + +## Compilation Tips + +To start the testing, run `make check` in the `tests/` directory. +You can modify the source files in `tests/snippets` and try different things. +To run the loader manually, use `./elf-loader ../tests/snippets/ arg1 arg2 ...`. diff --git a/content/assignments/elf-loader/img/auxv-example.drawio.svg b/content/assignments/elf-loader/img/auxv-example.drawio.svg new file mode 100644 index 0000000000..5eabdac464 --- /dev/null +++ b/content/assignments/elf-loader/img/auxv-example.drawio.svg @@ -0,0 +1,4 @@ + + + +
0x000...
0xfff...
6
AT_PAGESZ
4096
0
AT_NULL
0
11
AT_UID
1000
 \ No newline at end of file diff --git a/content/assignments/elf-loader/img/auxv.drawio.svg b/content/assignments/elf-loader/img/auxv.drawio.svg new file mode 100644 index 0000000000..f7d99764fb --- /dev/null +++ b/content/assignments/elf-loader/img/auxv.drawio.svg @@ -0,0 +1,4 @@ + + + +
0x000...
0xfff...
9
AT_ENTRY
0x401234
0
AT_NULL
0
 \ No newline at end of file diff --git a/content/assignments/elf-loader/img/stack-layout.drawio.svg b/content/assignments/elf-loader/img/stack-layout.drawio.svg new file mode 100644 index 0000000000..37c7aeed1c --- /dev/null +++ b/content/assignments/elf-loader/img/stack-layout.drawio.svg @@ -0,0 +1,4 @@ + + + +
0x000..
0xFFF...
argc
ESP
argv[0]
ESP + 4
argv[1]
...........
0
envp[0]
...........
0
auxv entries
auvx NULL entry
 \ No newline at end of file diff --git a/content/assignments/elf-loader/src/.gitignore b/content/assignments/elf-loader/src/.gitignore new file mode 100644 index 0000000000..e69de29bb2 diff --git a/content/assignments/elf-loader/src/Makefile b/content/assignments/elf-loader/src/Makefile new file mode 100644 index 0000000000..63d0b052bb --- /dev/null +++ b/content/assignments/elf-loader/src/Makefile @@ -0,0 +1,9 @@ +CFLAGS= -g -static-pie -o + +all: elf-loader + +elf-loader: elf-loader.c + gcc $(CFLAGS) $@ $< + +clean: + -rm -f *.o elf-loader diff --git a/content/assignments/elf-loader/src/elf-loader.c b/content/assignments/elf-loader/src/elf-loader.c new file mode 100644 index 0000000000..8e77d41ef0 --- /dev/null +++ b/content/assignments/elf-loader/src/elf-loader.c @@ -0,0 +1,102 @@ +// SPDX-License-Identifier: BSD-3-Clause + +#define _GNU_SOURCE + +#include +#include +#include +#include +#include +#include + +void *map_elf(const char *filename) +{ + // This part helps you store the content of the ELF file inside the buffer. + struct stat st; + void *file; + int fd; + + fd = open(filename, O_RDONLY); + if (fd < 0) { + perror("open"); + exit(1); + } + + fstat(fd, &st); + + file = mmap(NULL, st.st_size, PROT_READ, MAP_PRIVATE, fd, 0); + if (file == MAP_FAILED) { + perror("mmap"); + close(fd); + exit(1); + } + + return file; +} + +void load_and_run(const char *filename, int argc, char **argv, char **envp) +{ + // Contents of the ELF file are in the buffer: elf_contents[x] is the x-th byte of the ELF file. + void *elf_contents = map_elf(filename); + + /** + * TODO: ELF Header Validation + * Validate ELF magic bytes - "Not a valid ELF file" + exit code 3 if invalid. + * Validate ELF class is 64-bit (ELFCLASS64) - "Not a 64-bit ELF" + exit code 4 if invalid. + */ + + /** + * TODO: Load PT_LOAD segments + * For minimal syscall-only binaries. + * For each PT_LOAD segment: + * - Map the segments in memory. Permissions can be RWX for now. + */ + + /** + * TODO: Load Memory Regions with Correct Permissions + * For each PT_LOAD segment: + * - Set memory permissions according to program header p_flags (PF_R, PF_W, PF_X). + * - Use mprotect() or map with the correct permissions directly using mmap(). + */ + + /** + * TODO: Support Static Non-PIE Binaries with libc + * Must set up a valid process stack, including: + * - argc, argv, envp + * - auxv vector (with entries like AT_PHDR, AT_PHENT, AT_PHNUM, etc.) + * Note: Beware of the AT_RANDOM, AT_PHDR entries, the application will + * crash if you do not set them up properly. + */ + void *sp = NULL; + + /** + * TODO: Support Static PIE Executables + * Map PT_LOAD segments at a random load base. + * Adjust virtual addresses of segments and entry point by load_base. + * Stack setup (argc, argv, envp, auxv) same as above. + */ + + // TODO: Set the entry point and the stack pointer + void (*entry)() = NULL; + + // Transfer control + __asm__ __volatile__( + "mov %0, %%rsp\n" + "xor %%rbp, %%rbp\n" + "jmp *%1\n" + : + : "r"(sp), "r"(entry) + : "memory" + ); +} + +int main(int argc, char **argv, char **envp) +{ + if (argc < 2) { + fprintf(stderr, "Usage: %s \n", argv[0]); + exit(1); + } + + load_and_run(argv[1], argc - 1, &argv[1], envp); + return 0; +} diff --git a/content/assignments/elf-loader/tests/.gitignore b/content/assignments/elf-loader/tests/.gitignore new file mode 100644 index 0000000000..e69de29bb2 diff --git a/content/assignments/elf-loader/tests/Makefile b/content/assignments/elf-loader/tests/Makefile new file mode 100644 index 0000000000..401c5680e0 --- /dev/null +++ b/content/assignments/elf-loader/tests/Makefile @@ -0,0 +1,41 @@ +export SRC_PATH ?= $(realpath ../src) +export UTILS_PATH ?= $(realpath ../utils) + +CC=gcc +NASM=nasm +CFLAGS_NO_PIE=-g -static -fcf-protection=none -fno-PIC -o +CFLAGS=-g -static-pie -o +LDFLAGS_NO_PIE=-no-pie +LDFLAGS= +NASMFLAGS=-felf64 -o + +SNIPPETS_PATH=$(PWD)/snippets + +.PHONY: all src snippets clean_src clean_snippets check lint + +all: src snippets + +src: + $(MAKE) -C $(SRC_PATH) + +snippets: + $(MAKE) -C $(SNIPPETS_PATH) + +clean_snippets: + $(MAKE) -C $(SNIPPETS_PATH) clean + +clean_src: + $(MAKE) -C $(SRC_PATH) clean + +check: + $(MAKE) clean_src clean_snippets src snippets + ./run_tests.sh + +check-fast: + $(MAKE) clean_src clean_snippets src snippets + ./run_tests.sh + +lint: + -cd .. && checkpatch.pl -f src/*.c tests/snippets/*.c + -cd .. && checkpatch.pl -f checker/*.sh tests/*.sh + -cd .. && shellcheck checker/*.sh tests/*.sh diff --git a/content/assignments/elf-loader/tests/grade.sh b/content/assignments/elf-loader/tests/grade.sh new file mode 100755 index 0000000000..e71c0b2788 --- /dev/null +++ b/content/assignments/elf-loader/tests/grade.sh @@ -0,0 +1,136 @@ +#!/bin/bash +# SPDX-License-Identifier: BSD-3-Clause + +# Grade style based on build warnings and linter warnings / errors. +# Points are subtracted from the maximum amount of style points (10). +# - For 15 or more build warnings, all points (10) are subtracted. +# - For [10,15) build warnings, 6 points are subtracted. +# - For [5,10) build warnings, 4 points are subtracted. +# - For [1,5) build warnings, 2 points are subtracted. +# - For 25 ore more linter warnings / errors, all points (10) are subtracted. +# - For [15,25) linter warnings / errors, 6 points are subtracted. +# - For [7,15) linter warnings / errors, 4 points are subtracted. +# - For [1,7) linter warnings / errors, 2 points are subtracted. +# Final style points are between 0 and 10. Results cannot be negative. +# +# Result (grade) is stored in style_grade.out file. +# Collect summary in style_summary.out file. + +function grade_style() +{ + compiler_warn=$(< checker.out grep -v 'unused parameter' | grep -v 'unused variable' | \ + grep -v "discards 'const'" | grep -c '[0-9]\+:[0-9]\+: warning:') + + compiler_down=0 + if test "$compiler_warn" -ge 15; then + compiler_down=10 + elif test "$compiler_warn" -ge 10; then + compiler_down=6 + elif test "$compiler_warn" -ge 5; then + compiler_down=4 + elif test "$compiler_warn" -ge 1; then + compiler_down=2 + fi + + cpplint=$(< linter.out grep "Total errors found:" | rev | cut -d ' ' -f 1 | rev) + checkpatch_err=$(< linter.out grep 'total: [0-9]* errors' | grep -o '[0-9]* errors,' | \ + cut -d ' ' -f 1 | paste -s -d '+' | bc) + checkpatch_warn=$(< linter.out grep 'total: [0-9]* errors' | grep -o '[0-9]* warnings,' | \ + cut -d ' ' -f 1 | paste -s -d '+' | bc) + if test -z "$checkpatch_err"; then + checkpatch_err=0 + fi + if test -z "$checkpatch_warn"; then + checkpatch_warn=0 + fi + checkpatch=$((checkpatch_err + checkpatch_warn)) + checker_all=$((cpplint + checkpatch)) + + checker_down=0 + if test "$checker_all" -ge 25; then + checker_down=10 + elif test "$checker_all" -ge 15; then + checker_down=6 + elif test "$checker_all" -ge 7; then + checker_down=4 + elif test "$checker_all" -ge 1; then + checker_down=2 + fi + + full_down=$((compiler_down + checker_down)) + + if test "$full_down" -gt 10; then + full_down=10 + fi + style_grade=$((10 - full_down)) + + echo "$style_grade" > style_grade.out + + { + < linter.out grep -v 'unused parameter' | grep -v 'unused variable' | grep -v "discards 'const'" | \ + grep '[0-9]\+:[0-9]\+: warning:' + < linter.out grep "Total errors found: [1-9]" + < linter.out grep 'total: [1-9]* errors' + < linter.out grep 'total: 0 errors' | grep '[1-9][0-9]* warnings' + } > style_summary.out +} + +# Print grades: total, checker and style. +# Style grade is only awarded for assignments that have past 60 points +# of th checker grade. +print_results() +{ + checker_grade=$(< checker.out sed -n '/^Checker:/s/^.*[ \t]\+\([0-9\.]\+\)\/.*$/\1/p') + if test "$(echo "$checker_grade > 60" | bc)" -eq 1; then + style_grade=$(cat style_grade.out) + else + style_grade=0 + fi + final_grade=$(echo "scale=2; $checker_grade+$style_grade" | bc) + echo -e "\n\n### GRADE\n\n" + printf "Checker: %58s/ 90\n" "$checker_grade" + printf "Style: %60s/ 10\n" "$style_grade" + printf "Total: %60s/100\n" "$final_grade" + + echo -e "\n\n### STYLE SUMMARY\n\n" + cat style_summary.out +} + +run_interactive() +{ + echo -e "\n\n### CHECKER\n\n" + stdbuf -oL make check 2>&1 | stdbuf -oL sed 's/^Total:/Checker:/g' | tee checker.out + + echo -e "\n\n### LINTER\n\n" + stdbuf -oL make lint 2>&1 | tee linter.out + + grade_style + print_results +} + +run_non_interactive() +{ + make check 2>&1 | sed 's/^Total:/Checker:/g' > checker.out + make lint > linter.out 2>&1 + + grade_style + print_results + + echo -e "\n\n### CHECKER\n\n" + cat checker.out + + echo -e "\n\n### LINTER\n\n" + cat linter.out +} + +# In case of a command line argument disable interactive output. +# That is, do not show output as it generated. +# This is useful to collect all output and present final results at the +# beginning of the script output. +# This is because Moodle limits the output results, and the final results +# would otherwise not show up. +if test $# -eq 0; then + run_interactive +else + run_non_interactive +fi diff --git a/content/assignments/elf-loader/tests/ref/envp.ref b/content/assignments/elf-loader/tests/ref/envp.ref new file mode 100644 index 0000000000..e69de29bb2 diff --git a/content/assignments/elf-loader/tests/ref/error-bad-magic.ref b/content/assignments/elf-loader/tests/ref/error-bad-magic.ref new file mode 100644 index 0000000000..e69de29bb2 diff --git a/content/assignments/elf-loader/tests/ref/error-not-64.ref b/content/assignments/elf-loader/tests/ref/error-not-64.ref new file mode 100644 index 0000000000..e69de29bb2 diff --git a/content/assignments/elf-loader/tests/ref/no_pie.ref b/content/assignments/elf-loader/tests/ref/no_pie.ref new file mode 100644 index 0000000000..e69de29bb2 diff --git a/content/assignments/elf-loader/tests/ref/no_pie_argc.ref b/content/assignments/elf-loader/tests/ref/no_pie_argc.ref new file mode 100644 index 0000000000..b8626c4cff --- /dev/null +++ b/content/assignments/elf-loader/tests/ref/no_pie_argc.ref @@ -0,0 +1 @@ +4 diff --git a/content/assignments/elf-loader/tests/ref/no_pie_argv.ref b/content/assignments/elf-loader/tests/ref/no_pie_argv.ref new file mode 100644 index 0000000000..8ea56cd489 --- /dev/null +++ b/content/assignments/elf-loader/tests/ref/no_pie_argv.ref @@ -0,0 +1,4 @@ +4 +1 +2 +test diff --git a/content/assignments/elf-loader/tests/ref/no_pie_auxv.ref b/content/assignments/elf-loader/tests/ref/no_pie_auxv.ref new file mode 100644 index 0000000000..573541ac97 --- /dev/null +++ b/content/assignments/elf-loader/tests/ref/no_pie_auxv.ref @@ -0,0 +1 @@ +0 diff --git a/content/assignments/elf-loader/tests/ref/no_pie_envp.ref b/content/assignments/elf-loader/tests/ref/no_pie_envp.ref new file mode 100644 index 0000000000..dce8b5218f --- /dev/null +++ b/content/assignments/elf-loader/tests/ref/no_pie_envp.ref @@ -0,0 +1 @@ +ENV: test diff --git a/content/assignments/elf-loader/tests/ref/no_pie_hello.ref b/content/assignments/elf-loader/tests/ref/no_pie_hello.ref new file mode 100644 index 0000000000..af5626b4a1 --- /dev/null +++ b/content/assignments/elf-loader/tests/ref/no_pie_hello.ref @@ -0,0 +1 @@ +Hello, world! diff --git a/content/assignments/elf-loader/tests/ref/nolibc.ref b/content/assignments/elf-loader/tests/ref/nolibc.ref new file mode 100644 index 0000000000..e8d9a4003e --- /dev/null +++ b/content/assignments/elf-loader/tests/ref/nolibc.ref @@ -0,0 +1,2 @@ +Hello, world! +Hello from rodata! diff --git a/content/assignments/elf-loader/tests/ref/nolibc_no_rwx_rodata.ref b/content/assignments/elf-loader/tests/ref/nolibc_no_rwx_rodata.ref new file mode 100644 index 0000000000..cbf81240bb --- /dev/null +++ b/content/assignments/elf-loader/tests/ref/nolibc_no_rwx_rodata.ref @@ -0,0 +1 @@ +Hello, .rodata text! diff --git a/content/assignments/elf-loader/tests/ref/nolibc_no_rwx_text.ref b/content/assignments/elf-loader/tests/ref/nolibc_no_rwx_text.ref new file mode 100644 index 0000000000..e5d64ee90b --- /dev/null +++ b/content/assignments/elf-loader/tests/ref/nolibc_no_rwx_text.ref @@ -0,0 +1 @@ +Hello, .text test! diff --git a/content/assignments/elf-loader/tests/ref/pie.ref b/content/assignments/elf-loader/tests/ref/pie.ref new file mode 100644 index 0000000000..af5626b4a1 --- /dev/null +++ b/content/assignments/elf-loader/tests/ref/pie.ref @@ -0,0 +1 @@ +Hello, world! diff --git a/content/assignments/elf-loader/tests/run_tests.sh b/content/assignments/elf-loader/tests/run_tests.sh new file mode 100755 index 0000000000..63b789224e --- /dev/null +++ b/content/assignments/elf-loader/tests/run_tests.sh @@ -0,0 +1,106 @@ +#!/bin/bash +# SPDX-License-Identifier: BSD-3-Clause + +tests=( +"error-bad-magic|5" +"error-not-64|5" +"nolibc|20" +"nolibc_no_rwx_rodata|10" +"nolibc_no_rwx_text|10" +"no_pie_hello|5" +"no_pie_argc|5" +"no_pie_argv|5" +"no_pie_envp|5" +"no_pie_auxv|10" +"pie|10" +) + +loader="$(pwd)/../src/elf-loader" +snippets="$(pwd)/snippets/" +out="$(pwd)/out/" +ref="$(pwd)/ref/" +total=0 + +#set -o pipefail + +print_test() +{ + func="$1" + result="$2" + points="$3" + + if test "$points" -gt 999; then + points=999 + fi + + printf "%-32s " "${func:0:31}" + printf "........................" + if test "$result" -eq 0; then + printf " passed ... %3d\n" "$points" + total=$((total + points)) + else + printf " failed ... 0\n" + fi +} + +ret_expected() +{ + testname="$1" + + if [[ "$testname" =~ "no_rwx" ]]; then + return 139 + fi + + if [[ "$testname" =~ "error-bad-magic" ]]; then + return 3 + fi + + if [[ "$testname" =~ "error-not-64" ]]; then + return 4 + fi + + return 0 +} + +run_tests() +{ + if test ! -d "$out"; then + mkdir "$out" + fi + + for tst in "${tests[@]}"; do + test_name="$(echo "$tst" | cut -d'|' -f1)" + test_points="$(echo "$tst" | cut -d'|' -f2)" + + execute_test "$test_name" "$test_points" + done + + echo "" + echo -n "Total: " + echo -n " " + LC_ALL=C printf "%3d/100\n" "$total" +} + +execute_test() +{ + filename="$1" + points="$2" + outf="$filename.out" + reff="$filename.ref" + + setsid bash -c "ENV_TEST=test timeout -k 3 2 $loader $snippets/$filename 1 2 test > $out/$outf" >/dev/null 2>&1 & pid=$!; wait $pid; + + ret_code=$? + ret_expected "$filename" + ret_exp=$? + + if test "$ret_code" -ne "$ret_exp"; then + print_test "$filename" 1 "$points" + return + fi + + diff "$ref/$reff" "$out/$outf" > /dev/null + print_test "$filename" $? "$points" +} + +run_tests diff --git a/content/assignments/elf-loader/tests/snippets/Makefile b/content/assignments/elf-loader/tests/snippets/Makefile new file mode 100644 index 0000000000..a98796c127 --- /dev/null +++ b/content/assignments/elf-loader/tests/snippets/Makefile @@ -0,0 +1,46 @@ +CFLAGS_NO_PIE= -g -static -fcf-protection=none -fno-PIC -o +CFLAGS= -g -static-pie -o +LDFLAGS_NO_PIE= -no-pie +LDFLAGS= +NASMFLAGS= -felf64 -o + +all: nolibc_no_rwx_rodata nolibc_no_rwx_text nolibc no_pie_hello no_pie_argc no_pie_argv no_pie_auxv pie no_pie_envp + +nolibc_no_rwx_rodata.o: nolibc_no_rwx_rodata.asm + nasm $(NASMFLAGS) $@ $? + +nolibc_no_rwx_rodata: nolibc_no_rwx_rodata.o + ld -nostdlib -o $@ $? + +nolibc_no_rwx_text.o: nolibc_no_rwx_text.asm + nasm $(NASMFLAGS) $@ $? + +nolibc_no_rwx_text: nolibc_no_rwx_text.o + ld -nostdlib -o $@ $? + +nolibc.o: nolibc.asm + nasm $(NASMFLAGS) $@ $? + +nolibc: nolibc.o + ld -nostdlib -o $@ $? + +no_pie_hello: hello.c libc.a + gcc $(CFLAGS_NO_PIE) $@ $? $(LDFLAGS_NO_PIE) + +no_pie_argc: argc.c libc.a + gcc $(CFLAGS_NO_PIE) $@ $? $(LDFLAGS_NO_PIE) + +no_pie_argv: argv.c libc.a + gcc $(CFLAGS_NO_PIE) $@ $? $(LDFLAGS_NO_PIE) + +no_pie_auxv: auxv.c libc.a + gcc $(CFLAGS_NO_PIE) $@ $? $(LDFLAGS_NO_PIE) + +no_pie_envp: envp.c libc.a + gcc $(CFLAGS_NO_PIE) $@ $? $(LDFLAGS_NO_PIE) + +pie: hello.c libc.a + gcc $(CFLAGS) $@ $? $(LDFLAGS) + +clean: + -rm -f *.o pie no_pie_hello no_pie_argc no_pie_argv no_pie_auxv no_pie_envp nolibc_no_rwx_text nolibc_no_rwx_rodata nolibc diff --git a/content/assignments/elf-loader/tests/snippets/argc.c b/content/assignments/elf-loader/tests/snippets/argc.c new file mode 100644 index 0000000000..dba0932ce1 --- /dev/null +++ b/content/assignments/elf-loader/tests/snippets/argc.c @@ -0,0 +1,16 @@ +// SPDX-License-Identifier: BSD-3-Clause + +#include +#include +#include +#include + +int main(int argc, char **argv) +{ + printf("%d\n", argc); + fflush(stdout); + + syscall(SYS_exit_group, 0); + + return 0; // Should never be reached +} diff --git a/content/assignments/elf-loader/tests/snippets/argv.c b/content/assignments/elf-loader/tests/snippets/argv.c new file mode 100644 index 0000000000..715f29b949 --- /dev/null +++ b/content/assignments/elf-loader/tests/snippets/argv.c @@ -0,0 +1,18 @@ +// SPDX-License-Identifier: BSD-3-Clause + +#include +#include +#include +#include + +int main(int argc, char **argv) +{ + printf("%d\n", argc); + for (int i = 1; i < argc; i++) + printf("%s\n", argv[i]); + fflush(stdout); + + syscall(SYS_exit_group, 0); + + return 0; // Should never be reached +} diff --git a/content/assignments/elf-loader/tests/snippets/auxv.c b/content/assignments/elf-loader/tests/snippets/auxv.c new file mode 100644 index 0000000000..040e68dec5 --- /dev/null +++ b/content/assignments/elf-loader/tests/snippets/auxv.c @@ -0,0 +1,18 @@ +// SPDX-License-Identifier: BSD-3-Clause + +#include +#include +#include +#include + +int main(int argc, char **argv) +{ + unsigned long v = getauxval(AT_NULL); + + printf("%lu\n", v); + fflush(stdout); + + syscall(SYS_exit_group, 0); + + return 0; +} diff --git a/content/assignments/elf-loader/tests/snippets/envp.c b/content/assignments/elf-loader/tests/snippets/envp.c new file mode 100644 index 0000000000..3ad68eee43 --- /dev/null +++ b/content/assignments/elf-loader/tests/snippets/envp.c @@ -0,0 +1,21 @@ +// SPDX-License-Identifier: BSD-3-Clause + +#include +#include +#include +#include + +const int a = 1; + +int main(int argc, char **argv) +{ + char *env = malloc(30); + + env = getenv("ENV_TEST"); + printf("ENV: %s\n", env); + fflush(stdout); + + syscall(SYS_exit_group, 0); + + return 0; // Should never be reached +} diff --git a/content/assignments/elf-loader/tests/snippets/error-bad-magic b/content/assignments/elf-loader/tests/snippets/error-bad-magic new file mode 100755 index 0000000000000000000000000000000000000000..c85857673016cd10f319526ab89109638e1d4016 GIT binary patch literal 9040 zcmeHN%}N773{L<2Lu|!@1uxT{DoCvd!Gp+xB0cyH>(U&aP)TU_56qZZR#Gty{ua-MEB$*RzJ; zhND4H>X=qg1yli5Kow90Q~^~$6;K6K0aZX1PzC0;+&2pbDr0s(>n>3aA3AfGVI0r~<0MA6B3@48zD=$I-~d z(I7~J<{DUT&)i3GI6(KuK{p3wn9lM^>y{{i+$l)hJ>ySj^LKU*Y+EdXTr^02%No=Q_4V{@h>V4p!2hP_Kg> zq7U~^cuWg0EiN?CqI%p3H~8WQjC1q*u-N+nu#0I_Uvla5{g=Lr{J!Vw7uM!)5VVbR t-W}N95&9oj*l#edBFfrKS!gt!*UOS<(|XyliOzTRmt4P}O-4Ik{{dkqR%-wN literal 0 HcmV?d00001 diff --git a/content/assignments/elf-loader/tests/snippets/error-not-64 b/content/assignments/elf-loader/tests/snippets/error-not-64 new file mode 100755 index 0000000000000000000000000000000000000000..48b0c21f6fb0518fba3dd4778cd4176dd4bab7b6 GIT binary patch literal 15428 zcmeHOYit}>6~43HCQIWaYbP{m9GY=Ss=hqyR~_0A8b6bOlQ@Y}2q~S+?#_BQ+1**~ z%qDhKic_nlp+s_NfJ#sl@dJVsB!u!K0+qVBs#H`&hy;Q^JVYuSB?Y5~D$uIS_uZZA zOk7vczi5ti_MCIicVB1j%-rj__e=eQ1Dd7@MG;XaXbnw9W1Wy|H`Y9zLKjQKouV0X z5t8WZ90vw`5E;l(q><~uCt;sF7LActz$Vj&WXTXhATK$F3}pI6-1kEYw1GT@Oyrw_ zejUas_`zqOl5fbLK*CG#gX74%63lk9{0Kx0{uq4Z8Q5g{SqAwu@TbvsavyB6@(sP= zrDHnsqi*tg_{hq?8-AGYqs2&JxAxE@SH@)gfc43!Ujq`{8H) zGDST|EK5-%l-rd7Nqbws)}gcBDB{8L8pLuT%XTDmqV4O+VybR<7r%@^bpuyU zS*n-*1ZZ^wS5R5{E}e(j_U0qU|2%PVc4&Cd%&kZ~HhmJs9>U-5hfYi!2V`jG9B@~k zS-XDv(%*o{Oa}j^gGOd@lCWUXxj3rZ%pY(4Iph{-Ytz2tZ%kZSnPn=Q9e-`wIC1t4 zdMD>JKSAbiA@caoU!Ga_^1mb6`Cni2^qjSlVo+<|W&yxh3M8r6?j z`HHRQ+z0dx>6^t+_Niy_pNSXVt5$IxtQV${Ltd-PNI5qBpdp03K5TYKVFbbmgb@fM z5Jn)3Kp25A0$~Kg2!s&`BM?U5zeRwbLir6n52VS!6*M-tx}3*v4Lgv zpI*Mii;jv1*KE4GeSPrLGZ7($5eOp?Mj(tp7=bVXVFbbmgb@fM5Jn)3Kp24!6@i2J z%GwI%T9o^U>zNyS&asJDoIyIjVRH|869m87uAzi2*SpaF`Ciqzf<5BgPyO%{*jo%) z4|y}>XTeLrTpxcFOyc_bD)Mv7G1IwDZUC8ePx{qgorqqXxc zKUpsMYdfbI7$Hgs*ZqDm6x>zd`!)K00A_l@8Z|7pAQD-pU?*J9&RE4nsq7rGQ=Xg1 zSgA_hvphRtxjENMWUX@gs8zOGTqom61zX1LGGKNhZzWy+w6tzac$RzEO`z*On5 z&M79u&hEWC^me1I%V=#A+IWPmU<-uSfWJm9zNjwN6x$G6fvXoWM1!`4moE#Bh>e74 z?<`x2UwJ+PQP<|$4yR!xP)*)N)47Sgqg{3i4jfdFHLcnI`9z|*xj7oC8;fXKb916D z8fnyS-@kuzb5vUw!3K6-$r~w{5*7cbvc0ckeCd!HMi0lu7w4|mQgIt>MZ|Lq|H$`8 z#8Pn&?Ad_*DcBtE%tQY$q};gxsfpF%ASA~(1nd>!%aFWBXcPY&l{tT-_%UstSU>ks zJfXr2{421zlY#xxVE+>G`G9|Z`+pDr=~JqH-VN7@<^L6?8owIuo>|&Ho_ACoD&qac zT$;E=+!_cdW7h@y6ym`l+Zb6Y@~~srxk0>+yKrzG{tK`fANu3)r(yHCK)W%rOdNu( z^3#7bkY5MpDcJRX{)WgZ@f_^n8Hw-0UO89(YVm#8lV~4=Y+nfq&*HSxM*Lo)&nq>|bJZs{;wuUu2?Vw$>X z4(#dP(Qo$e>@!Va_U+%YXl5sM|?BC{lxApB2=GMVoJ>7%mu7QEQ{ln&Pch6uy za?C%hvPGRuktb3{Khb53*q&KRnO?S1JZvOK{nIRSICS1lTb?CkcTpgX+9?(oHymk! z&vneKRZQpY_nn$ShlH8Vn`Jxir0`&55ne83mU6&mN@n&bic7lgT+zcjdR4a>Jv^%1 zv4UqMA-!@HcxF6}3E_iS(BsF1#_xf-;@yNZwFyTX+iLWW8yeACVBEz#sp>r6(<;W-*cGkb}*)(!0tCpiS0%|#wj&iL+ zE`>TBl!YHdR!}Pp@el?EVgcg|O|_Gi5#+OqBN(;5nJZ=-rCZ5l**>By9K5n|oBXfD zxdCr0zU<_gHyC_xE259|aWKz-5hu@7rhQc|FB4SC8fH@B# zj&qQ+P&nUF^)j7&7|eMKb`$q3{8#%3~jBPk@Q*0duZ& z4LG)&xLd#q_f_a@E9;TZI)HTQJ`qRayG`MAJQH=`*gi!po7k^`6^_s1AvpFk!6)KK z&w~|??}#&aeW~vt^s9Qm(8tMy=m$L!fy?8g=3*0!; z6shulhJ zKZxV`yV(yY?_vPQclcG{=EslidozIJyElf5#(ESHLwXC$bk@svWE?oOne9c*=f#G&&Zl4HQf@i6bjK!cUa*;f#E@xI#pNo7;G{{T15L*@Vg literal 0 HcmV?d00001 diff --git a/content/assignments/elf-loader/tests/snippets/hello.c b/content/assignments/elf-loader/tests/snippets/hello.c new file mode 100644 index 0000000000..b4898075b4 --- /dev/null +++ b/content/assignments/elf-loader/tests/snippets/hello.c @@ -0,0 +1,16 @@ +// SPDX-License-Identifier: BSD-3-Clause + +#include +#include +#include +#include + +int main(int argc, char **argv) +{ + printf("Hello, world!\n"); + fflush(stdout); + + syscall(SYS_exit_group, 0); + + return 0; // Should never be reached +} diff --git a/content/assignments/elf-loader/tests/snippets/nolibc.asm b/content/assignments/elf-loader/tests/snippets/nolibc.asm new file mode 100644 index 0000000000..f8fbe1ac5e --- /dev/null +++ b/content/assignments/elf-loader/tests/snippets/nolibc.asm @@ -0,0 +1,31 @@ +; SPDX-License-Identifier: BSD-3-Clause + +section .data + msg db "Hello, world!", 10 ; Message + newline + msglen equ $ - msg ; Length of the message + +section .rodata + msg_rodata db "Hello from rodata!", 10 ; Message + newline + msglen_rodata equ $ - msg_rodata ; Length of the message + +section .text + global _start + +_start: + ; write(stdout=1, msg, msglen) + mov rax, 1 ; syscall: write + mov rdi, 1 ; file descriptor: stdout + mov rsi, msg ; pointer to message + mov rdx, msglen ; message length + syscall + + mov rax, 1 + mov rdi, 1 + mov rsi, msg_rodata + mov rdx, msglen_rodata + syscall + + ; exit(0) + mov rax, 60 ; syscall: exit + xor rdi, rdi ; status = 0 + syscall diff --git a/content/assignments/elf-loader/tests/snippets/nolibc_no_rwx_rodata.asm b/content/assignments/elf-loader/tests/snippets/nolibc_no_rwx_rodata.asm new file mode 100644 index 0000000000..a5e6b39fcc --- /dev/null +++ b/content/assignments/elf-loader/tests/snippets/nolibc_no_rwx_rodata.asm @@ -0,0 +1,24 @@ +section .data + msg db "Hello, .rodata text!", 10 ; Message + newline + msglen equ $ - msg ; Length of the message + +section .rodata + ro_msg db "test" + +section .text + global _start + +_start: + ; write(stdout=1, msg, msglen) + mov rax, 1 ; syscall: write + mov rdi, 1 ; file descriptor: stdout + mov rsi, msg ; pointer to message + mov rdx, msglen ; message length + syscall + + mov BYTE [ro_msg], 0 + + ; exit(0) + mov rax, 60 ; syscall: exit + xor rdi, rdi ; status = 0 + syscall diff --git a/content/assignments/elf-loader/tests/snippets/nolibc_no_rwx_text.asm b/content/assignments/elf-loader/tests/snippets/nolibc_no_rwx_text.asm new file mode 100644 index 0000000000..ee6f9488cf --- /dev/null +++ b/content/assignments/elf-loader/tests/snippets/nolibc_no_rwx_text.asm @@ -0,0 +1,24 @@ +section .data + msg db "Hello, .text test!", 10 ; Message + newline + msglen equ $ - msg ; Length of the message + +section .rodata + ro_msg db "test" + +section .text + global _start + +_start: + ; write(stdout=1, msg, msglen) + mov rax, 1 ; syscall: write + mov rdi, 1 ; file descriptor: stdout + mov rsi, msg ; pointer to message + mov rdx, msglen ; message length + syscall + + mov BYTE [_start], 0 + + ; exit(0) + mov rax, 60 ; syscall: exit + xor rdi, rdi ; status = 0 + syscall diff --git a/gen-zip.py b/gen-zip.py index a456f804a6..dcb94a51f5 100644 --- a/gen-zip.py +++ b/gen-zip.py @@ -54,7 +54,7 @@ def find_directories_by_name(dirname): matches = [] for root, dirs, _ in os.walk("."): for d in dirs: - if d == dirname: + if d == dirname and "drills" in root: matches.append(os.path.join(root, d)) return matches