diff --git a/examples/genai-rag-multimodal/README.md b/examples/genai-rag-multimodal/README.md index 63f1f0a3..598ddf41 100644 --- a/examples/genai-rag-multimodal/README.md +++ b/examples/genai-rag-multimodal/README.md @@ -12,15 +12,26 @@ The main modifications to the original example include: - Adaptations to comply with Cloud Foundation Toolkit security measures. - Installation of additional libraries in the Conda environment. -- Use of Vertex AI Workbench to run the notebook with a custom Service Account. +- Use of Vertex AI Workbench to run the notebook with a custom Service Account in a secure environment. - Implementation of Vector Search on Vertex AI with [Private Service Connect](https://cloud.google.com/vpc/docs/private-service-connect). +For more information about the technologies used in this example, please refer to the following resources: + +- [Vertex AI Workbench Introduction](https://cloud.google.com/vertex-ai/docs/workbench/introduction) +- [Vertex AI Vector Search Overview](https://cloud.google.com/vertex-ai/docs/vector-search/overview) +- [Ragas Documentation](https://docs.ragas.io/en/stable/) + +After ensuring all requirements are satisfied you can follow one of the two deployment options: + +1. [**Using Machine Learning Infra Pipeline**](#deploying-infrastructure-using-machine-learning-infra-pipeline): This is a robust option suitable for production environments and continuous deployment scenarios. +2. [**Using Terraform Locally**](#deploying-infrastructure-using-terraform-locally): This is a better option for one-time testing purposes where you know you will delete the example later. + ## Requirements - Terraform v1.7.5 - [Authenticated Google Cloud SDK 469.0.0](https://cloud.google.com/sdk/docs/authorizing) -### Provision Infrastructure with Terraform +### Terraform Variables Configuration - Update the `terraform.tfvars` file with values from your environment. @@ -33,59 +44,100 @@ The main modifications to the original example include: ``` - Assuming you are deploying the example on top of the development environment, the following instructions will provide you more insight on how to retrieve these values: - - **NETWORK-PROJECT-ID**: Run `terraform output -raw restricted_host_project_id` on `gcp-networks` repository, inside the development environment directory and branch. - - **NETWORK-NAME**: Run `terraform output -raw restricted_network_name` on `gcp-networks` repository, inside the development environment directory and branch. - - **MACHINE-LEARNING-PROJECT-ID**: Run `terraform output -raw machine_learning_project_id` on `gcp-projects` repository, inside the Machine Learning business unit directory and on the development branch. - - **KMS-PROJECT-ID**, **ML-ENV-KEYRING**, **ML-ENV-KEY**: Run `terraform output machine_learning_kms_keys` on `gcp-projects` repository, inside the Machine Learning business unit directory and on the development branch. + - **NETWORK-PROJECT-ID**: Run `terraform -chdir="envs/development" output -raw restricted_host_project_id` on `gcp-networks` repository at the development branch. Please note that if you have not initialized the environment you will need to run `./tf-wrapper.sh init development` on the directory. + - **NETWORK-NAME**: Run `terraform -chdir="envs/development" output -raw restricted_network_name` on `gcp-networks` repository at the development branch. Please note that if you have not initialized the environment you will need to run `./tf-wrapper.sh init development` on the directory. + - **MACHINE-LEARNING-PROJECT-ID**: Run `terraform -chdir="ml_business_unit/development" output -raw machine_learning_project_id` on `gcp-projects` repository, at the development branch. Please note that if you have not initialized the environment you will need to run `./tf-wrapper.sh init development` on the directory. + - **KMS-PROJECT-ID**, **ML-ENV-KEYRING**, **ML-ENV-KEY**: Run `terraform -chdir="ml_business_unit/development" output machine_learning_kms_keys` on `gcp-projects` repository, at the development branch. Please note that if you have not initialized the environment you will need to run `./tf-wrapper.sh init development` on the directory. - **REGION**: The chosen region. -### Allow file download from Google Notebook Examples Bucket on VPC-SC Perimeter + - Optionally, you may follow the series of steps below to automatically cre the `terraform.tfvars` file: + - **IMPORTANT:** Please note that the steps below are assuming you are checked out on the same level as `terraform-google-enterprise-genai/` and the other repos (`gcp-bootstrap`, `gcp-org`, `gcp-projects`...). + - Retrieve values from terraform outputs to bash variables: -When running the Notebook, you will reach a step that downloads an example PDF file from a bucket, you need to add the egress rule below on the VPC-SC perimeter to allow the operation. - -```yaml -- egressFrom: - identities: - - serviceAccount:rag-notebook-runner@.iam.gserviceaccount.com - egressTo: - operations: - - methodSelectors: - - method: google.storage.buckets.list - - method: google.storage.buckets.get - - method: google.storage.objects.get - - method: google.storage.objects.list - serviceName: storage.googleapis.com - resources: - - projects/200612033880 # Google Cloud Example Project -``` + ```bash + (cd gcp-networks && git checkout development && ./tf-wrapper.sh init development) + + export restricted_host_project_id=$(terraform -chdir="gcp-networks/envs/development" output -raw restricted_host_project_id) + + export restricted_network_name=$(terraform -chdir="gcp-networks/envs/development" output -raw restricted_network_name) + + (cd gcp-projects && git checkout development && ./tf-wrapper.sh init development) + + export machine_learning_project_id=$(terraform -chdir="gcp-projects/ml_business_unit/development" output -raw machine_learning_project_id) + + export machine_learning_kms_keys_json=$(terraform -chdir="gcp-projects/ml_business_unit/development" output -json machine_learning_kms_keys) + ``` + + - Extract the kms key from the `json` variable by using `jq`: + + ```bash + export machine_learning_kms_keys=$(echo $machine_learning_kms_keys_json | jq -r ".\"$region\".id") + ``` + + - Create region environment variable (if you are not using `us-central1`, remember to change the value below): + + ```bash + export region="us-central1" + ``` + + - Validate if the variables values are correct: + + ```bash + echo region=$region + echo restricted_host_project_id=$restricted_host_project_id + echo restricted_network_name=$restricted_network_name + echo machine_learning_project_id=$machine_learning_project_id + echo machine_learning_kms_keys=$machine_learning_kms_keys + ``` + + - Populate `terraform.tfvars` with the following command: + + ```bash + cat > terraform-google-enterprise-genai/examples/genai-rag-multimodal/terraform.tfvars < --member="serviceAccount:$SERVICE_ACCOUNT" --role="roles/compute.networkUser" + gcloud projects add-iam-policy-binding $restricted_host_project_id --member="serviceAccount:$SERVICE_ACCOUNT" --role="roles/compute.networkUser" ``` -- Add the following ingress rule to the Service Perimeter. - - ```yaml - ingressPolicies: - - ingressFrom: - identities: - - serviceAccount: - sources: - - accessLevel: '*' - ingressTo: - operations: - - serviceName: '*' - resources: - - '*' - ``` +- Add the build service account in the development VPC-SC perimeter. + - Retrieve the service account value for your environment: + + ```bash + echo "serviceAccount:$SERVICE_ACCOUNT" + ``` + + - Add "serviceAccount:" to `perimeter_additional_members` field in `common.auto.tfvars` at `gcp-networks` repository on the development branch. + + - Commit and push the result by running the commands below: + + ```bash + cd gcp-networks + git add common.auto.tfvars + git commit -m "Add machine learning build SA to perimeter" + git push origin development + ``` ### Deployment steps @@ -136,7 +188,8 @@ When running the Notebook, you will reach a step that downloads an example PDF f - Create a file named `genai_example.tf` under `ml_business_unit/development` path that calls the module. - ```terraform + ```bash + cat > ml_business_unit/development/genai_example.tf < ml_business_unit/development/backend.tf < NOTE: If you are deploying this example on top of an existing foundation instance, the variable name might be `egress_policies`. + +```terraform +{ + "from" = { + "identity_type" = "" + "identities" = [ + "serviceAccount:rag-notebook-runner@.iam.gserviceaccount.com" + ] + }, + "to" = { + "resources" = ["projects/200612033880"] # Google Cloud Example Project + "operations" = { + "storage.googleapis.com" = { + "methods" = [ + "google.storage.buckets.list", + "google.storage.buckets.get", + "google.storage.objects.get", + "google.storage.objects.list", + ] + } + } + } +}, +``` + +> **IMPORTANT**: If you are planning to delete the notebook-runner service account at any moment, make sure you remove this policy before deleting it. ## Usage Once all the requirements are set up, you can start by running and adjusting the notebook step-by-step. -To run the notebook, open the Google Cloud Console on Vertex AI Workbench, open JupyterLab and upload the notebook (`multimodal_rag_langchain.ipynb`) to it. +To run the notebook, open the Google Cloud Console on Vertex AI Workbench (`https://console.cloud.google.com/vertex-ai/workbench/instances?referrer=search&project=`), click open JupyterLab on the created instance. + +After clicking "open JupyterLab" button, you will be taken to an interactive JupyterLab Workspace, you can upload the notebook (`multimodal_rag_langchain.ipynb`) in this repo to it. Once the notebook is uploaded to the environment, run it cell-by-cell to see process of building a RAG chain. The notebook contains placeholders variables that must be replaced, you may follow the next section instructions to automatically replace this placeholders using `sed` command. ### Optional: Use `terraform output` and bash command to fill in fields in the notebook +#### Infra Pipeline (Cloud Build) + +If you ran using Cloud Build, proceed with the steps below to use `terraform output`. + +- Update `outputs.tf` file on `ml-machine-learning/ml_business_unit/development` and add the following values to it, if the file does not exist create it: + + ```terraform + output "private_endpoint_ip_address" { + value = module.genai_example.private_endpoint_ip_address + } + + output "host_vpc_project_id" { + value = module.genai_example.host_vpc_project_id + } + + output "host_vpc_network" { + value = module.genai_example.host_vpc_network + } + + output "notebook_project_id" { + value = module.genai_example.notebook_project_id + } + + output "vector_search_bucket_name" { + value = module.genai_example.vector_search_bucket_name + } + ``` + +- Run `./tf-wrapper.sh init development` on `ml-machine-learning`. + +- Run `cd ml_business_unit/development && terraform refresh`, to refresh the outputs. + +- Extract values from `terraform output` and validate. You must run the commands below at `ml-machine-learning/ml_business_unit/development`. + + ```bash + export private_endpoint_ip_address=$(terraform output -raw private_endpoint_ip_address) + echo private_endpoint_ip_address=$private_endpoint_ip_address + + export host_vpc_project_id=$(terraform output -raw host_vpc_project_id) + echo host_vpc_project_id=$host_vpc_project_id + + export notebook_project_id=$(terraform output -raw notebook_project_id) + echo notebook_project_id=$notebook_project_id + + export vector_search_bucket_name=$(terraform output -raw vector_search_bucket_name) + echo vector_search_bucket_name=$vector_search_bucket_name + + export host_vpc_network=$(terraform output -raw host_vpc_network) + echo host_vpc_network=$host_vpc_network + ``` + +- Search and Replace using `sed` command at `terraform-google-enterprise-genai/examples/genai-rag-multimodal`. + + ```bash + cd ../../../terraform-google-enterprise-genai/examples/genai-rag-multimodal + + sed -i "s//$private_endpoint_ip_address/g" multimodal_rag_langchain.ipynb + + sed -i "s//$host_vpc_project_id/g" multimodal_rag_langchain.ipynb + + sed -i "s//$notebook_project_id/g" multimodal_rag_langchain.ipynb + + sed -i "s//$vector_search_bucket_name/g" multimodal_rag_langchain.ipynb + + sed -i "s::$host_vpc_network:g" multimodal_rag_langchain.ipynb + ``` + +#### Terraform Locally + +If you ran terraform locally, proceed with the steps below to use `terraform output`. + You can save some time adjusting the notebook by running the commands below: - Extract values from `terraform output` and validate. @@ -204,6 +392,10 @@ You can save some time adjusting the notebook by running the commands below: sed -i "s::$host_vpc_network:g" multimodal_rag_langchain.ipynb ``` +## Notes + +- Some, but not exclusively, of the billable components deployed are: Vertex AI Workbench Instance, Private Service Connect Endpoint and Vector Search Endpoint. + ## Known Issues - `Error: Error creating Instance: googleapi: Error 400: value_to_check(https://compute.googleapis.com/compute/v1/projects/...) is not found`. diff --git a/examples/genai-rag-multimodal/multimodal_rag_langchain.ipynb b/examples/genai-rag-multimodal/multimodal_rag_langchain.ipynb index 4c138eed..cef283e4 100644 --- a/examples/genai-rag-multimodal/multimodal_rag_langchain.ipynb +++ b/examples/genai-rag-multimodal/multimodal_rag_langchain.ipynb @@ -708,7 +708,8 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "* Retrieve the value of the service attachment and execute the command below on your local machine:" + "* Retrieve the value of the service attachment and execute the `gcloud` command below on your local machine:\n", + "* NOTE: If you don't have permission to run the command below, you may need to open a PR to `gcp-networks` repository adding the forwarding rule or ask a network engineer to create this forwarding rule." ] }, { @@ -720,7 +721,7 @@ "NETWORK=\"\"\n", "NETWORK_PROJECT_ID=\"\"\n", "\n", - "!gcloud compute forwarding-rules create vector-search-endpoint \\\n", + "!echo gcloud compute forwarding-rules create vector-search-endpoint \\\n", " --network={NETWORK} \\\n", " --address=vector-search-endpoint \\\n", " --target-service-attachment={SERVICE_ATTACHMENT} \\\n", @@ -1024,7 +1025,7 @@ "metadata": {}, "outputs": [], "source": [ - "%pip install ragas" + "%pip install ragas==0.1.9" ] }, { diff --git a/examples/genai-rag-multimodal/outputs.tf b/examples/genai-rag-multimodal/outputs.tf index 1cd4ae7a..dcf6aee0 100644 --- a/examples/genai-rag-multimodal/outputs.tf +++ b/examples/genai-rag-multimodal/outputs.tf @@ -25,8 +25,8 @@ output "host_vpc_project_id" { } output "host_vpc_network" { - description = "This is the Self-link of the Host VPC network" - value = google_workbench_instance.instance.gce_setup[0].network_interfaces[0].network + description = "This is the self-link of the Host VPC network, without the URL prefix (i.e. https://)" + value = var.network } output "notebook_project_id" {