chore(docs): adjust rag example documentation (#67)

* adjust rag example documentation * update docs * more info on usage * adjust tf init, terraform output with chdir flag and more information about products * fix ragas version * add pricing info * adjusting RAG procedure * adjust rag populate tfvars command * adjusting rag example * Remove empty section * adjust with more automated steps * add more info * add description about deployment options
GoogleCloudPlatform · Oct 8, 2024 · 37a32e7 · 37a32e7
1 parent ba52535
commit 37a32e7
Show file tree

Hide file tree

Showing 3 changed files with 243 additions and 50 deletions.
diff --git a/examples/genai-rag-multimodal/README.md b/examples/genai-rag-multimodal/README.md
@@ -12,15 +12,26 @@ The main modifications to the original example include:
 
 - Adaptations to comply with Cloud Foundation Toolkit security measures.
 - Installation of additional libraries in the Conda environment.
-- Use of Vertex AI Workbench to run the notebook with a custom Service Account.
+- Use of Vertex AI Workbench to run the notebook with a custom Service Account in a secure environment.
 - Implementation of Vector Search on Vertex AI with [Private Service Connect](https://cloud.google.com/vpc/docs/private-service-connect).
 
+For more information about the technologies used in this example, please refer to the following resources:
+
+- [Vertex AI Workbench Introduction](https://cloud.google.com/vertex-ai/docs/workbench/introduction)
+- [Vertex AI Vector Search Overview](https://cloud.google.com/vertex-ai/docs/vector-search/overview)
+- [Ragas Documentation](https://docs.ragas.io/en/stable/)
+
+After ensuring all requirements are satisfied you can follow one of the two deployment options:
+
+1. [**Using Machine Learning Infra Pipeline**](#deploying-infrastructure-using-machine-learning-infra-pipeline): This is a robust option suitable for production environments and continuous deployment scenarios.
+2. [**Using Terraform Locally**](#deploying-infrastructure-using-terraform-locally): This is a better option for one-time testing purposes where you know you will delete the example later.
+
 ## Requirements
 
 - Terraform v1.7.5
 - [Authenticated Google Cloud SDK 469.0.0](https://cloud.google.com/sdk/docs/authorizing)
 
-### Provision Infrastructure with Terraform
+### Terraform Variables Configuration
 
 - Update the `terraform.tfvars` file with values from your environment.
 
@@ -33,59 +44,100 @@ The main modifications to the original example include:
   ```
 
 - Assuming you are deploying the example on top of the development environment, the following instructions will provide you more insight on how to retrieve these values:
-  - **NETWORK-PROJECT-ID**: Run `terraform output -raw restricted_host_project_id` on `gcp-networks` repository, inside the development environment directory and branch.
-  - **NETWORK-NAME**: Run `terraform output -raw restricted_network_name` on `gcp-networks` repository, inside the development environment directory and branch.
-  - **MACHINE-LEARNING-PROJECT-ID**: Run `terraform output -raw machine_learning_project_id` on `gcp-projects` repository, inside the Machine Learning business unit directory and on the development branch.
-  - **KMS-PROJECT-ID**, **ML-ENV-KEYRING**, **ML-ENV-KEY**: Run `terraform output machine_learning_kms_keys` on `gcp-projects` repository, inside the Machine Learning business unit directory and on the development branch.
+  - **NETWORK-PROJECT-ID**: Run `terraform -chdir="envs/development" output -raw restricted_host_project_id` on `gcp-networks` repository at the development branch. Please note that if you have not initialized the environment you will need to run `./tf-wrapper.sh init development` on the directory.
+  - **NETWORK-NAME**: Run `terraform -chdir="envs/development" output -raw restricted_network_name` on `gcp-networks` repository at the development branch. Please note that if you have not initialized the environment you will need to run `./tf-wrapper.sh init development` on the directory.
+  - **MACHINE-LEARNING-PROJECT-ID**: Run `terraform -chdir="ml_business_unit/development" output -raw machine_learning_project_id` on `gcp-projects` repository, at the development branch. Please note that if you have not initialized the environment you will need to run `./tf-wrapper.sh init development` on the directory.
+  - **KMS-PROJECT-ID**, **ML-ENV-KEYRING**, **ML-ENV-KEY**: Run `terraform -chdir="ml_business_unit/development" output machine_learning_kms_keys` on `gcp-projects` repository, at the development branch. Please note that if you have not initialized the environment you will need to run `./tf-wrapper.sh init development` on the directory.
   - **REGION**: The chosen region.
 
-### Allow file download from Google Notebook Examples Bucket on VPC-SC Perimeter
+  - Optionally, you may follow the series of steps below to automatically cre the `terraform.tfvars` file:
+    - **IMPORTANT:** Please note that the steps below are assuming you are checked out on the same level as `terraform-google-enterprise-genai/` and the other repos (`gcp-bootstrap`, `gcp-org`, `gcp-projects`...).
+    - Retrieve values from terraform outputs to bash variables:
 
-When running the Notebook, you will reach a step that downloads an example PDF file from a bucket, you need to add the egress rule below on the VPC-SC perimeter to allow the operation.
-
-```yaml
-- egressFrom:
-    identities:
-    - serviceAccount:rag-notebook-runner@<INSERT_YOUR_MACHINE_LEARNING_PROJECT_ID_HERE>.iam.gserviceaccount.com
-  egressTo:
-      operations:
-      - methodSelectors:
-      - method: google.storage.buckets.list
-      - method: google.storage.buckets.get
-      - method: google.storage.objects.get
-      - method: google.storage.objects.list
-      serviceName: storage.googleapis.com
-      resources:
-      - projects/200612033880 # Google Cloud Example Project
-```
+      ```bash
+      (cd gcp-networks && git checkout development && ./tf-wrapper.sh init development)
+
+      export restricted_host_project_id=$(terraform -chdir="gcp-networks/envs/development" output -raw restricted_host_project_id)
+
+      export restricted_network_name=$(terraform -chdir="gcp-networks/envs/development" output -raw restricted_network_name)
+
+      (cd gcp-projects && git checkout development && ./tf-wrapper.sh init development)
+
+      export machine_learning_project_id=$(terraform -chdir="gcp-projects/ml_business_unit/development" output -raw machine_learning_project_id)
+
+      export machine_learning_kms_keys_json=$(terraform -chdir="gcp-projects/ml_business_unit/development" output -json machine_learning_kms_keys)
+      ```
+
+    - Extract the kms key from the `json` variable by using `jq`:
+
+      ```bash
+      export machine_learning_kms_keys=$(echo $machine_learning_kms_keys_json | jq -r ".\"$region\".id")
+      ```
+
+    - Create region environment variable (if you are not using `us-central1`, remember to change the value below):
+
+      ```bash
+      export region="us-central1"
+      ```
+
+    - Validate if the variables values are correct:
+
+      ```bash
+      echo region=$region
+      echo restricted_host_project_id=$restricted_host_project_id
+      echo restricted_network_name=$restricted_network_name
+      echo machine_learning_project_id=$machine_learning_project_id
+      echo machine_learning_kms_keys=$machine_learning_kms_keys
+      ```
+
+    - Populate `terraform.tfvars` with the following command:
+
+      ```bash
+      cat > terraform-google-enterprise-genai/examples/genai-rag-multimodal/terraform.tfvars <<EOF
+      kms_key                   = "$machine_learning_kms_keys"
+      network                   = "projects/$restricted_host_project_id/global/networks/$restricted_network_name"
+      subnet                    = "projects/$restricted_host_project_id/regions/$region/subnetworks/sb-d-shared-restricted-$region"
+      machine_learning_project  = "$machine_learning_project_id"
+      vector_search_vpc_project = "$restricted_host_project_id"
+      EOF
+      ```
+
+    - Validate if all values are correct in `terraform.tfvars`
+
+      ```bash
+      cat terraform-google-enterprise-genai/examples/genai-rag-multimodal/terraform.tfvars
+      ```
 
 ## Deploying infrastructure using Machine Learning Infra Pipeline
 
-### Required Permissions for pipeline Service Account
+### Required Permissions and VPC-SC adjustments for pipeline Service Account
 
-- Give `roles/compute.networkUser` to the Service Account that runs the Pipeline.
+- The Service Account that runs the Pipeline must have `roles/compute.networkUser` on the Shared VPC Host Project, you can give this role by running the command below:
 
   ```bash
+  (cd gcp-projects && git checkout production && ./tf-wrapper.sh init shared)
   SERVICE_ACCOUNT=$(terraform -chdir="./gcp-projects/ml_business_unit/shared" output -json terraform_service_accounts | jq -r '."ml-machine-learning"')
 
-  gcloud projects add-iam-policy-binding <INSERT_HOST_VPC_NETWORK_PROJECT_HERE> --member="serviceAccount:$SERVICE_ACCOUNT" --role="roles/compute.networkUser"
+  gcloud projects add-iam-policy-binding  $restricted_host_project_id --member="serviceAccount:$SERVICE_ACCOUNT" --role="roles/compute.networkUser"
   ```
 
-- Add the following ingress rule to the Service Perimeter.
-
-  ```yaml
-  ingressPolicies:
-  - ingressFrom:
-      identities:
-      - serviceAccount:<SERVICE_ACCOUNT>
-      sources:
-      - accessLevel: '*'
-    ingressTo:
-      operations:
-      - serviceName: '*'
-      resources:
-      - '*'
-  ```
+- Add the build service account in the development VPC-SC perimeter.
+  - Retrieve the service account value for your environment:
+
+    ```bash
+    echo "serviceAccount:$SERVICE_ACCOUNT"
+    ```
+  
+  - Add "serviceAccount:<YOUR_SA_HERE>" to `perimeter_additional_members` field in `common.auto.tfvars` at `gcp-networks` repository on the development branch.
+
+  - Commit and push the result by running the commands below:
+
+    ```bash
+    cd gcp-networks
+    git add common.auto.tfvars
+    git commit -m "Add machine learning build SA to perimeter"
+    git push origin development
+    ```
 
 ### Deployment steps
 
@@ -136,7 +188,8 @@ When running the Notebook, you will reach a step that downloads an example PDF f
 
 - Create a file named `genai_example.tf` under `ml_business_unit/development` path that calls the module.
 
-  ```terraform
+  ```bash
+  cat > ml_business_unit/development/genai_example.tf <<EOF
   module "genai_example" {
     source = "../../modules/genai-rag-multimodal"
 
@@ -146,8 +199,34 @@ When running the Notebook, you will reach a step that downloads an example PDF f
     machine_learning_project  = var.machine_learning_project
     vector_search_vpc_project = var.vector_search_vpc_project
   }
+  EOF
   ```
 
+- Verify if `backend.tf` file exists at `ml-machine-learning/ml_business_unit/development`.
+  - If there is a `backend.tf` file, proceed with the next step (commit and push) and ignore the sub-steps below.
+  - If there is no `backend.tf` file, follow the sub-steps below:
+    - Create the file by running the command below:
+
+      ```bash
+      cat > ml_business_unit/development/backend.tf <<EOF
+      terraform {
+        backend "gcs" {
+          bucket = "UPDATE_APP_INFRA_BUCKET"
+          prefix = "terraform/app-infra/ml_business_unit/development"
+        }
+      }
+      EOF
+      ```
+
+    - Run the command below to update `UPDATE_APP_INFRA_BUCKET` placeholder:
+
+      ```bash
+      export backend_bucket=$(terraform -chdir="../gcp-projects/ml_business_unit/shared/" output -json state_buckets | jq '."ml-machine-learning"' --raw-output)
+      echo "backend_bucket = ${backend_bucket}"
+
+      for i in `find -name 'backend.tf'`; do sed -i "s/UPDATE_APP_INFRA_BUCKET/${backend_bucket}/" $i; done
+      ```
+
 - Commit and push
 
   ```terraform
@@ -159,16 +238,125 @@ When running the Notebook, you will reach a step that downloads an example PDF f
 
 ## Deploying infrastructure using terraform locally
 
-Run `terraform init && terraform apply -auto-approve`.
+- Only proceed with these steps if you have not deployed [using cloudbuild](#deploying-infrastructure-using-machine-learning-infra-pipeline).
+- Run `terraform init` inside this directory.
+- Run `terraform apply` inside this directory.
+
+## Post-Deployment
+
+### Allow file download from Google Notebook Examples Bucket on VPC-SC Perimeter
+
+When running the Notebook, you will reach a step that downloads an example PDF file from a bucket, you need to add the egress rule below on the VPC-SC perimeter to allow the operation. You can do this by adding this rule to `egress_rule` variable on `gcp-networks/envs/development/development.auto.tfvars` on the development branch.
+
+> NOTE: If you are deploying this example on top of an existing foundation instance, the variable name might be `egress_policies`.
+
+```terraform
+{
+  "from" = {
+    "identity_type" = ""
+    "identities" = [
+      "serviceAccount:rag-notebook-runner@<INSERT_YOUR_MACHINE_LEARNING_PROJECT_ID_HERE>.iam.gserviceaccount.com"
+    ]
+  },
+  "to" = {
+    "resources" = ["projects/200612033880"]  # Google Cloud Example Project
+    "operations" = {
+        "storage.googleapis.com" = {
+        "methods" = [
+            "google.storage.buckets.list",
+            "google.storage.buckets.get",
+            "google.storage.objects.get",
+            "google.storage.objects.list",
+          ]
+        }
+    }
+  }
+},
+```
+
+> **IMPORTANT**: If you are planning to delete the notebook-runner service account at any moment, make sure you remove this policy before deleting it.
 
 ## Usage
 
 Once all the requirements are set up, you can start by running and adjusting the notebook step-by-step.
 
-To run the notebook, open the Google Cloud Console on Vertex AI Workbench, open JupyterLab and upload the notebook (`multimodal_rag_langchain.ipynb`) to it.
+To run the notebook, open the Google Cloud Console on Vertex AI Workbench (`https://console.cloud.google.com/vertex-ai/workbench/instances?referrer=search&project=<MACHINE_LEARNING_PROJECT_ID>`), click open JupyterLab on the created instance.
+
+After clicking "open JupyterLab" button, you will be taken to an interactive JupyterLab Workspace, you can upload the notebook (`multimodal_rag_langchain.ipynb`) in this repo to it. Once the notebook is uploaded to the environment, run it cell-by-cell to see process of building a RAG chain. The notebook contains placeholders variables that must be replaced, you may follow the next section instructions to automatically replace this placeholders using `sed` command.
 
 ### Optional: Use `terraform output` and bash command to fill in fields in the notebook
 
+#### Infra Pipeline (Cloud Build)
+
+If you ran using Cloud Build, proceed with the steps below to use `terraform output`.
+
+- Update `outputs.tf` file on `ml-machine-learning/ml_business_unit/development` and add the following values to it, if the file does not exist create it:
+
+  ```terraform
+  output "private_endpoint_ip_address" {
+    value       = module.genai_example.private_endpoint_ip_address
+  }
+
+  output "host_vpc_project_id" {
+    value       = module.genai_example.host_vpc_project_id
+  }
+
+  output "host_vpc_network" {
+    value       = module.genai_example.host_vpc_network
+  }
+
+  output "notebook_project_id" {
+    value       = module.genai_example.notebook_project_id
+  }
+
+  output "vector_search_bucket_name" {
+    value       = module.genai_example.vector_search_bucket_name
+  }
+  ```
+
+- Run `./tf-wrapper.sh init development` on `ml-machine-learning`.
+
+- Run `cd ml_business_unit/development && terraform refresh`, to refresh the outputs.
+
+- Extract values from `terraform output` and validate. You must run the commands below at `ml-machine-learning/ml_business_unit/development`.
+
+  ```bash
+  export private_endpoint_ip_address=$(terraform output -raw private_endpoint_ip_address)
+  echo private_endpoint_ip_address=$private_endpoint_ip_address
+
+  export host_vpc_project_id=$(terraform output -raw host_vpc_project_id)
+  echo host_vpc_project_id=$host_vpc_project_id
+
+  export notebook_project_id=$(terraform output -raw notebook_project_id)
+  echo notebook_project_id=$notebook_project_id
+
+  export vector_search_bucket_name=$(terraform output -raw vector_search_bucket_name)
+  echo vector_search_bucket_name=$vector_search_bucket_name
+
+  export host_vpc_network=$(terraform output -raw host_vpc_network)
+  echo host_vpc_network=$host_vpc_network
+  ```
+
+- Search and Replace using `sed` command at `terraform-google-enterprise-genai/examples/genai-rag-multimodal`.
+
+  ```bash
+  cd ../../../terraform-google-enterprise-genai/examples/genai-rag-multimodal
+
+  sed -i "s/<INSERT_PRIVATE_IP_VALUE_HERE>/$private_endpoint_ip_address/g" multimodal_rag_langchain.ipynb
+
+  sed -i "s/<INSERT_HOST_VPC_PROJECT_ID>/$host_vpc_project_id/g" multimodal_rag_langchain.ipynb
+
+  sed -i "s/<INSERT_NOTEBOOK_PROJECT_ID>/$notebook_project_id/g" multimodal_rag_langchain.ipynb
+
+  sed -i "s/<INSERT_BUCKET_NAME>/$vector_search_bucket_name/g" multimodal_rag_langchain.ipynb
+
+  sed -i "s:<INSERT_HOST_VPC_NETWORK>:$host_vpc_network:g" multimodal_rag_langchain.ipynb
+  ```
+
+#### Terraform Locally
+
+If you ran terraform locally, proceed with the steps below to use `terraform output`.
+
 You can save some time adjusting the notebook by running the commands below:
 
 - Extract values from `terraform output` and validate.
@@ -204,6 +392,10 @@ You can save some time adjusting the notebook by running the commands below:
   sed -i "s:<INSERT_HOST_VPC_NETWORK>:$host_vpc_network:g" multimodal_rag_langchain.ipynb
   ```
 
+## Notes
+
+- Some, but not exclusively, of the billable components deployed are: Vertex AI Workbench Instance, Private Service Connect Endpoint and Vector Search Endpoint.
+
 ## Known Issues
 
 - `Error: Error creating Instance: googleapi: Error 400: value_to_check(https://compute.googleapis.com/compute/v1/projects/...) is not found`.

diff --git a/examples/genai-rag-multimodal/multimodal_rag_langchain.ipynb b/examples/genai-rag-multimodal/multimodal_rag_langchain.ipynb
@@ -708,7 +708,8 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "* Retrieve the value of the service attachment and execute the command below on your local machine:"
+    "* Retrieve the value of the service attachment and execute the `gcloud` command below on your local machine:\n",
+    "* NOTE: If you don't have permission to run the command below, you may need to open a PR to `gcp-networks` repository adding the forwarding rule or ask a network engineer to create this forwarding rule."
    ]
   },
   {
@@ -720,7 +721,7 @@
     "NETWORK=\"<INSERT_HOST_VPC_NETWORK>\"\n",
     "NETWORK_PROJECT_ID=\"<INSERT_HOST_VPC_PROJECT_ID>\"\n",
     "\n",
-    "!gcloud compute forwarding-rules create vector-search-endpoint \\\n",
+    "!echo gcloud compute forwarding-rules create vector-search-endpoint \\\n",
     "    --network={NETWORK} \\\n",
     "    --address=vector-search-endpoint \\\n",
     "    --target-service-attachment={SERVICE_ATTACHMENT} \\\n",
@@ -1024,7 +1025,7 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "%pip install ragas"
+    "%pip install ragas==0.1.9"
    ]
   },
   {

diff --git a/examples/genai-rag-multimodal/outputs.tf b/examples/genai-rag-multimodal/outputs.tf
@@ -25,8 +25,8 @@ output "host_vpc_project_id" {
 }
 
 output "host_vpc_network" {
-  description = "This is the Self-link of the Host VPC network"
-  value       = google_workbench_instance.instance.gce_setup[0].network_interfaces[0].network
+  description = "This is the self-link of the Host VPC network, without the URL prefix (i.e. https://)"
+  value       = var.network
 }
 
 output "notebook_project_id" {