docs: Fill the INSTALL_AND_USAGE docs

Author: iusztinpaul

6-bonus-superlinked-rag/README.md

@@ -46,7 +46,9 @@ make local-start-superlinked
 ```
 
 > [!IMPORTANT]
-> Before starting, make sure that you have your `.env` file filled with everything required to run the system. 
+> Before starting, ensure you have your `.env` file filled with everything required to run the system. 
+>
+> For more details on setting up the local infrastructure, you can check out the course's main [INSTALL_AND_USAGE](https://github.com/decodingml/llm-twin-course/blob/main/INSTALL_AND_USAGE.md) document.
 
 To stop the local infrastructure, run:
 ```shell
@@ -71,3 +73,6 @@ To query the vector DB, run:
 ```shell
 make ... # TO BE ADDED
 ```
+
+> [!IMPORTANT]
+> You can check out the main [INSTALL_AND_USAGE](https://github.com/decodingml/llm-twin-course/blob/main/INSTALL_AND_USAGE.md) document of the course for more details on an end-to-end flow.

INSTALL_AND_USAGE.md

@@ -18,13 +18,17 @@ Go to the root of the repository, copy our `.env.example` file and fill it with
 cp .env.example .env
 ```
 
-## Run
+## Supported commands
 
-### Check all possible `Make` commands
+We will use `GNU Make` to install and run our application.
+
+To see all our supported commands, run the following:
 ```shell
 make help
 ```
 
+## Set up the infrastructure
+
 ### Spin up the infrastructure
 
 Now, the whole infrastructure can be spun up using a simple Make command:
@@ -33,18 +37,77 @@ Now, the whole infrastructure can be spun up using a simple Make command:
 make local-start
 ```
 
-Behind the scenes it will build and run all the Docker images defined in the [docker-compose.yml[(https://github.com/decodingml/llm-twin-course/blob/main/docker-compose.yml)] file.
+Behind the scenes it will build and run all the Docker images defined in the [docker-compose.yml](https://github.com/decodingml/llm-twin-course/blob/main/docker-compose.yml) file.
 
+## Read this before starting 🚨 
 
 > [!CAUTION]
-> For `Mongo`` to work with multiple replicas on MacOs or linux systems you have to add the following lines of code to `/etc/hosts`:
+> For `Mongo` to work with multiple replicas (as we use it in our Docker setup) on `macOS` or `Linux` systems, you have to add the following lines of code to `/etc/hosts`:
 >
 > ```
 > 127.0.0.1       mongo1
 > 127.0.0.1       mongo2 
 > 127.0.0.1       mongo3
 > ```
 >
-> From what we know on Windows, it works out-of-the-box.
->
->
+> From what we know, on `Windows`, it `works out-of-the-box`.
+
+> [!WARNING]
+> For `arm` users (e.g., `M1/M2/M3 macOS devices`), go to your Docker desktop application and enable `Use Rosetta for x86_64/amd64 emulation on Apple Silicon` from the Settings. There is a checkbox you have to check.
+> Otherwise, your Docker containers will crash.
+
+### Tear down the infrastructure
+
+Run the following `Make` command to tear down all your docker containers:
+
+```shell
+make local-stop
+```
+
+## Run an end-to-end flow
+
+Now that we have configured our credentials and started our infrastructure let's look at how to run an end-to-end flow of the LLM Twin application.
+
+> [!IMPORTANT]
+> Note that we won't go into the details of the system here. To fully understand it, check out our free article series, which explains everything step-by-step: [LLM Twin articles series](https://medium.com/decodingml/llm-twin-course/home).
+
+### Step 1: Crawlers
+
+Trigger the crawler to collect data and add it to the MongoDB:
+
+```shell
+make local-test-github
+# or make local-test-medium
+``` 
+
+After the data is added to Mongo, the CDC component will be triggered, which will populate the RabbitMQ with the event.
+
+### Step 2: Feature engineering & Vector DB
+
+Check that the feature pipeline works and the vector DB is successfully populated.
+
+To check the `feature pipeline`, check the logs of the `llm-twin-bytewax` Docker container by running:
+```shell
+docker logs llm-twin-bytewax
+```
+You should see logs reflecting the cleaning, chunking, and embedding operations (without any errors, of course).
+
+To check that the Qdrant `vector DB` is populated successfully, go to its dashboard at localhost:6333/dashboard. There, you should see the repositories or article collections created and populated.
+
+> [!NOTE]
+> If using the cloud version of Qdrant, go to your Qdrant account and cluster to see the same thing as in the local dashboard.
+
+### Step 3: RAG retrieval step
+
+Now that we have some data in our vector DB, let's test out the RAG retriever:
+
+### Step 4: Generate the instruct dataset
+
+
+### Step 5: Fine-tuning
+
+
+### Step 6: Inference
+
+
+

README.md

@@ -1,6 +1,6 @@
 <div align="center">
     <h2>LLM Twin Course: Building Your Production-Ready AI Replica</h2>
-    <h1>An End-to-End Framework for Production-Ready LLM & RAG Systems by Building Your LLM Twin</h1>
+    <h1>Learn to build an end-to-end system for production-ready LLM & RAG systems by building your LLM Twin</h1>
     <h3>From data gathering to productionizing LLMs using LLMOps good practices.</h3>
     <i>by <a href="https://github.com/iusztinpaul">Paul Iusztin</a>, <a href="https://github.com/alexandruvesa">Alexandru Vesa</a> and <a href="https://github.com/Joywalker">Alexandru Razvant</a></i>
 </div>
@@ -142,7 +142,21 @@ If you plan to run the code while reading it, you have to know that we use sever
 11. [Build a scalable RAG ingestion pipeline using 74.3% less code](https://medium.com/decodingml/build-a-scalable-rag-ingestion-pipeline-using-74-3-less-code-ac50095100d6)
 12. Refactoring the 4 Advanced RAG Algorithms using Superlinked
 
+## Install & Usage
+
+To understand how to install and run the LLM Twin code, go to the [INSTALL_AND_USAGE](https://github.com/decodingml/llm-twin-course/blob/main/INSTALL_AND_USAGE.md) dedicated document.
+
+> [!NOTE]
+> Even though you can run everything solely using the [INSTALL_AND_USAGE](https://github.com/decodingml/llm-twin-course/blob/main/INSTALL_AND_USAGE.md) dedicated document, we recommend that you read the articles to understand the LLM Twin system and design choices fully.
+
+### Bonus Superlinked series
+
+The bonus Superlinked series has an extra dedicated [README](https://github.com/decodingml/llm-twin-course/blob/main/6-bonus-superlinked-rag/README.md) that you can access under the [6-bonus-superlinked-rag](https://github.com/decodingml/llm-twin-course/tree/main/6-bonus-superlinked-rag) directory.
+
+Here we explain all the changes made to the code to run it with the improved RAG layer powered by [Superlinked](https://rebrand.ly/superlinked-github).
+
 ## Meet your teachers!
+
 The course is created under the [Decoding ML](https://decodingml.substack.com/) umbrella by:
 
 <table>