feat(examples): Vertex Machine Learning Pipeline

2-environments/README.md

@@ -196,7 +196,7 @@ You will be doing this procedure for each environment (`development`, `non-produ
     Make sure your git is checked out to the development branch by running `git checkout development` on `GCP_ENVIRONMENTS_PATH`.
 
     ```bash
-    (cd $GCP_ENVIRONMENTS_PATH && git checkout development)
+    (cd $GCP_ENVIRONMENTS_PATH && git checkout development && cd $GCP_ENVIRONMENTS_PATH/envs/development && terraform init)
     ```
 
 2. Retrieve the bucket name and project id from terraform outputs.
@@ -244,7 +244,7 @@ You will be doing this procedure for each environment (`development`, `non-produ
     Make sure your git is checked out to the `non-production` branch by running `git checkout non-production` on `GCP_ENVIRONMENTS_PATH`.
 
     ```bash
-    (cd $GCP_ENVIRONMENTS_PATH && git checkout non-production)
+    (cd $GCP_ENVIRONMENTS_PATH && git checkout non-production && cd $GCP_ENVIRONMENTS_PATH/envs/non-production && terraform init)
     ```
 
 2. Retrieve the bucket name and project id from terraform outputs.
@@ -292,7 +292,7 @@ You will be doing this procedure for each environment (`development`, `non-produ
     Make sure your git is checked out to the `production` branch by running `git checkout production` on `GCP_ENVIRONMENTS_PATH`.
 
     ```bash
-    (cd $GCP_ENVIRONMENTS_PATH && git checkout production)
+    (cd $GCP_ENVIRONMENTS_PATH && git checkout production && cd $GCP_ENVIRONMENTS_PATH/envs/production && terraform init)
     ```
 
 2. Retrieve the bucket name and project id from terraform outputs.

5-app-infra/README.md

@@ -222,20 +222,17 @@ Once pushed, the pipeline build logs can be accessed by navigating to the artifa
 #### Configuring Cloud Source Repository of Artifact Application
 
 1. Grab the Artifact Project ID
-
    ```bash
    export ARTIFACT_PROJECT_ID=$(terraform -chdir="gcp-projects/ml_business_unit/shared" output -raw common_artifacts_project_id)
    echo ${ARTIFACT_PROJECT_ID}
    ```
 
 1. Clone the freshly minted Cloud Source Repository that was created for this project.
-
    ```bash
    gcloud source repos clone publish-artifacts --project=${ARTIFACT_PROJECT_ID}
    ```
 
 1. Enter the repo folder and copy over the artifact files from `5-app-infra/source_repos/artifact-publish` folder.
-
    ```bash
    cd publish-artifacts
    git checkout -b main
@@ -254,7 +251,6 @@ Once pushed, the pipeline build logs can be accessed by navigating to the artifa
    ```
 
 1. `cd` out of the `publish-artifacts` repository.
-
    ```bash
    cd ..
    ```
@@ -271,7 +267,6 @@ Although Service Catalog itself must be manually deployed, the modules which wil
 The resoning behind utilizing one repository with two deployment methodologies is due to how close interactive (`development`) and operational environments are.
 
 The repository has the structure (truncated for brevity):
-
    ```
    ml_business_unit
    ├── development
@@ -407,13 +402,11 @@ The pipeline also listens for changes made to `plan`, `development`, `non-produc
    ```
 
 1. Clone the freshly minted Cloud Source Repository that was created for this project.
-
    ```bash
    gcloud source repos clone service-catalog --project=${SERVICE_CATALOG_PROJECT_ID}
    ```
 
 1. Enter the repo folder and copy over the service catalogs files from `5-app-infra/source_repos/service-catalog` folder.
-
    ```bash
    cd service-catalog/
    cp -RT ../terraform-google-enterprise-genai/5-app-infra/source_repos/service-catalog/ .
@@ -422,7 +415,6 @@ The pipeline also listens for changes made to `plan`, `development`, `non-produc
    ```
 
 1. Commit changes and push main branch to the new repo.
-
    ```bash
    git add modules
    git commit -m 'Initialize Service Catalog Build Repo'
@@ -431,7 +423,6 @@ The pipeline also listens for changes made to `plan`, `development`, `non-produc
    ```
 
 1. `cd` out of the `service_catalog` repository.
-
    ```bash
    cd ..
    ```
@@ -547,20 +538,17 @@ unset GOOGLE_IMPERSONATE_SERVICE_ACCOUNT
 1. The next instructions assume that you are at the same level of the `terraform-google-enterprise-genai` folder.
 
 1. Grab the Artifact Project ID
-
    ```bash
    export ARTIFACT_PROJECT_ID=$(terraform -chdir="terraform-google-enterprise-genai/4-projects/ml_business_unit/shared" output -raw common_artifacts_project_id)
    echo ${ARTIFACT_PROJECT_ID}
    ```
 
 1. Clone the freshly minted Cloud Source Repository that was created for this project.
-
    ```bash
    gcloud source repos clone publish-artifacts --project=${ARTIFACT_PROJECT_ID}
    ```
 
 1. Enter the repo folder and copy over the artifact files from `5-app-infra/source_repos/artifact-publish` folder.
-
    ```bash
    cd publish-artifacts
    git checkout -b main
@@ -579,7 +567,6 @@ unset GOOGLE_IMPERSONATE_SERVICE_ACCOUNT
    ```
 
 1. `cd` out of the `publish-artifacts` repository.
-
    ```bash
    cd ..
    ```
@@ -698,13 +685,11 @@ After executing this stage, unset the `GOOGLE_IMPERSONATE_SERVICE_ACCOUNT` envir
    ```
 
 1. Clone the freshly minted Cloud Source Repository that was created for this project.
-
    ```bash
    gcloud source repos clone service-catalog --project=${SERVICE_CATALOG_PROJECT_ID}
    ```
 
 1. Enter the repo folder and copy over the service catalogs files from `5-app-infra/source_repos/service-catalog` folder.
-
    ```bash
    cd service-catalog/
    git checkout -b main
@@ -715,7 +700,6 @@ After executing this stage, unset the `GOOGLE_IMPERSONATE_SERVICE_ACCOUNT` envir
    ```
 
 1. Commit changes and push main branch to the new repo.
-
    ```bash
    git add modules
    git commit -m 'Initialize Service Catalog Build Repo'
@@ -724,7 +708,6 @@ After executing this stage, unset the `GOOGLE_IMPERSONATE_SERVICE_ACCOUNT` envir
    ```
 
 1. `cd` out of the `service-catalog` repository.
-
    ```bash
    cd ..
    ```

5-app-infra/projects/artifact-publish/ml_business_unit/shared/README.md

@@ -4,7 +4,6 @@
 | Name | Description | Type | Default | Required |
 |------|-------------|------|---------|:--------:|
 | instance\_region | The region where compute instance will be created. A subnetwork must exists in the instance region. | `string` | n/a | yes |
-| log\_bucket | Log bucket to be used by Service Catalog Bucket | `string` | n/a | yes |
 | remote\_state\_bucket | Backend bucket to load remote state information from previous steps. | `string` | n/a | yes |
 
 ## Outputs

5-app-infra/projects/service-catalog/ml_business_unit/shared/README.md

@@ -4,6 +4,7 @@
 | Name | Description | Type | Default | Required |
 |------|-------------|------|---------|:--------:|
 | instance\_region | The region where compute instance will be created. A subnetwork must exists in the instance region. | `string` | n/a | yes |
+| log\_bucket | Log bucket to be used by Service Catalog Bucket | `string` | n/a | yes |
 | remote\_state\_bucket | Backend bucket to load remote state information from previous steps. | `string` | n/a | yes |
 
 ## Outputs

5-app-infra/source_repos/artifact-publish/images/vertexpipeline:v2/Dockerfile

@@ -13,4 +13,4 @@
 # limitations under the License.
 #
 FROM tensorflow/tensorflow:2.8.0
-RUN pip install tensorflow-io==0.25.0 protobuf==3.20.0 google-cloud-bigquery==3.13.0 pandas==2.0.3 db-dtypes==1.2.0 google-cloud-aiplatform==1.36.0 google-cloud-storage==2.14.0 kfp google-cloud-pipeline-components
+RUN pip install tensorflow-io==0.25.0 protobuf==3.20.0 google-cloud-bigquery==3.13.0 pandas==2.0.3 db-dtypes==1.2.0 google-cloud-aiplatform==1.36.0 google-cloud-storage==2.14.0 kfp==2.7.0 google-cloud-pipeline-components