crc qe virtualized migration #111
There are no files selected for viewing
Empty file.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,44 @@ | ||
| # Overview | ||
|
|
||
| This pipeline allows to run the qe phase on externally build distribution of crc. | ||
| The target host will be self provisioned. | ||
|
|
||
| ## Params | ||
|
|
||
| TBC | ||
|
|
||
| ## Required tasks | ||
|
|
||
| This pipeline relies on several tasks which should be deployed to allow deploy its spec | ||
|
|
||
| ```yaml | ||
| oc apply -f catalog/task/gather-run-info/0.1/gather-run-info.yaml | ||
| oc apply -f catalog/task/gather-crc-info/0.1/gather-crc-info.yaml | ||
| oc apply -f catalog/task/crc-preparer/0.2/crc-preparer.yaml | ||
| oc apply -f catalog/task/crc-qe-cli/0.1/crc-qe-cli.yaml | ||
| oc apply -f catalog/task/infra-fedora-provision/0.1/infra-fedora-provision.yaml | ||
| oc apply -f catalog/task/infra-fedora-decommission/0.1/infra-fedora-decommission.yaml | ||
| oc apply -f catalog/task/infra-rhel-provision/0.1/infra-rhel-provision.yaml | ||
| oc apply -f catalog/task/infra-rhel-decommission/0.1/infra-rhel-decommission.yaml | ||
| #REVIEW | ||
| oc apply -f catalog/task/gather-run-info/0.1/gather-run-info.yaml | ||
| oc apply -f catalog/task/gather-host-info/0.1/gather-host-info.yaml | ||
| oc apply -f catalog/task/gather-crc-info/0.1/gather-crc-info.yaml | ||
| oc apply -f catalog/task/gather-s3-info/0.1/gather-s3-info.yaml | ||
| oc apply -f catalog/task/asset-checker-http/0.1/asset-checker-http.yaml | ||
| oc apply -f catalog/task/crc-preparer/0.2/crc-preparer.yaml | ||
| oc apply -f catalog/task/crc-builder-tray/0.3/crc-builder-tray-mac-universal.yaml | ||
| oc apply -f catalog/task/crc-builder-tray/0.3/crc-builder-tray-x64.yaml | ||
| oc apply -f catalog/task/crc-builder-installer/0.3/crc-builder-installer.yaml | ||
| oc apply -f orchestrator/catalog/task/crc-qe-cli/0.1/crc-qe-cli.yaml | ||
| oc apply -f catalog/task/s3-sink-workspace/0.3/s3-sink-workspace.yaml | ||
| oc apply -f catalog/task/url-generator/0.1/url-generator.yamls | ||
| ``` | ||
|
|
||
| ### workspace | ||
|
|
||
| The task uses a workspace to copy files which are required across the pipeline. As an example | ||
| the host key is copied to the workspace and the path is exposed through `workspace-resources-path`. | ||
|
|
||
| Then any other task across the pipeline which uses the workspace can access the key to use for ssh into | ||
| the target host. | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,320 @@ | ||
| --- | ||
| apiVersion: tekton.dev/v1beta1 | ||
| kind: Pipeline | ||
| metadata: | ||
| name: crc-qe-virtualized | ||
| labels: | ||
| app.kubernetes.io/version: "0.1" | ||
| redhat.com/product: openshift-local | ||
| redhat.com/phase: qe | ||
| annotations: | ||
| tekton.dev/pipelines.minVersion: "0.24.x" | ||
| tekton.dev/categories: qe | ||
| tekton.dev/tags: openshift-local, qe, nested virualization | ||
| tekton.dev/displayName: "qe for openshift local on linux nested virtualization" | ||
| tekton.dev/platforms: "linux/amd64" | ||
| spec: | ||
| description: >- | ||
| This pipeline will run the testing for openshift local on | ||
| a specific linux version based on nested virtualization | ||
|
|
||
| workspaces: | ||
| - name: pipelines-data | ||
|
|
||
| params: | ||
| # Fedora related params | ||
| - name: fedora-version | ||
| default: '' | ||
| # Rhel related params | ||
| - name: rhel-version | ||
| default: '' | ||
| - name: repo-baseos-url | ||
| description: baseos repo url to setup on provisioned machine | ||
| default: '' | ||
| - name: repo-appstream-url | ||
| description: appstream repo url to setup on provisioned machine | ||
| default: '' | ||
| - name: rh-account-secret | ||
| description: ocp secret holding the credentials for a valid rh user to subscribe VM. Secret should be accessible to this task. | ||
| default: credentials-rh-subs-crcqe-prod | ||
| # Common params | ||
| - name: target-platform | ||
| default: linux | ||
| - name: image-id | ||
| description: internal id for image on virtualization provider (Openstack image ID) | ||
| default: '' | ||
| - name: downloadable-url | ||
| description: full base url to download ditributables and shasumsfiles | ||
| default: https://mirror.openshift.com/pub/openshift-v4/clients/crc/latest | ||
| - name: shasum-file | ||
| description: file name for shasum to check distributable | ||
| default: sha256sum.txt | ||
| # QE params | ||
| - name: e2e-tag | ||
| description: tags to select e2e scnearios. Default empty values which means all scnearios | ||
| default: "''" | ||
| - name: qe-worspace-subpath | ||
| description: subpath on workspace where storing ephemeral qe results | ||
| default: qe-results | ||
|
Comment on lines
+56
to
+58
There was a problem hiding this comment. Resolve parameter naming inconsistency. Line 56 defines a parameter named - name: qe-worspace-subpath
- description: subpath on workspace where storing ephemeral qe results
+ description: subpath on workspace where storing ephemeral QE results
default: qe-results
...
- name: worspace-qe-subpath
+ value: $(params.qe-workspace-subpath)
- value: $(params.qe-worspace-subpath)Also applies to: 224-226 🤖 Prompt for AI Agents |
||
| - name: run-e2e | ||
| description: Control if e2e tests are executed. (true or false) | ||
| default: 'true' | ||
| - name: run-integration | ||
| description: Control if integration tests are executed. (true or false) | ||
| default: 'true' | ||
| # S3 params | ||
| - name: s3-data-secret | ||
| default: datalake-aws | ||
| - name: s3-bucket | ||
| default: crcqe-asia | ||
| - name: s3-folder-path | ||
| description: target folder to store results. (Example release/2.9.0) | ||
|
|
||
| results: | ||
| - name: results-url | ||
| description: url to get results for the execution | ||
| value: $(tasks.url-generator.results.url) | ||
| - name: product-version | ||
| description: product version tested for this interop | ||
| value: $(tasks.crc-info.results.crc-version) | ||
| - name: qe-duration | ||
| description: amount of time in seconds for qe execution (testing) | ||
| value: $(tasks.qe.results.duration) | ||
|
|
||
| tasks: | ||
| - name: correlate | ||
| taskRef: | ||
| name: gather-run-info | ||
| workspaces: | ||
| - name: pipelines-data | ||
| workspace: pipelines-data | ||
| - name: s3-info | ||
| runAfter: | ||
| - correlate | ||
| taskRef: | ||
| name: gather-s3-info | ||
| params: | ||
| - name: s3-data-secret | ||
| value: $(params.s3-data-secret) | ||
| - name: bucket | ||
| value: $(params.s3-bucket) | ||
| - name: folder-path | ||
| # only one have value fedora-version or rhel-version | ||
| value: $(params.s3-folder-path)/$(params.fedora-version)$(params.rhel-version) | ||
| - name: crc-info | ||
| runAfter: | ||
| - correlate | ||
| taskRef: | ||
| name: gather-crc-info | ||
| params: | ||
| - name: platform | ||
| value: $(params.target-platform) | ||
| - name: provision-fedora | ||
| runAfter: | ||
| - s3-info | ||
| - crc-info | ||
| taskRef: | ||
| name: infra-fedora-provision | ||
| when: | ||
| - input: $(params.fedora-version) | ||
| operator: notin | ||
| values: [''] | ||
| params: | ||
| - name: id | ||
| value: crc-qe-virtualized-fedora | ||
| - name: fedora-version | ||
| value: $(params.fedora-version) | ||
| - name: image-id | ||
| value: $(params.image-id) | ||
| - name: correlation | ||
| value: $(tasks.correlate.results.correlation) | ||
| - name: workspace-resources-path | ||
| value: $(tasks.correlate.results.workspace-resources-path) | ||
| workspaces: | ||
| - name: pipelines-data | ||
| workspace: pipelines-data | ||
| timeout: "20m" | ||
| - name: provision-rhel | ||
| runAfter: | ||
| - s3-info | ||
| - crc-info | ||
| when: | ||
| - input: $(params.rhel-version) | ||
| operator: notin | ||
| values: [''] | ||
| taskRef: | ||
| name: infra-rhel-provision | ||
| params: | ||
| - name: id | ||
| value: crc-qe-virtualized-rhel | ||
| - name: rhel-version | ||
| value: $(params.rhel-version) | ||
| - name: repo-baseos-url | ||
| value: $(params.repo-baseos-url) | ||
| - name: repo-appstream-url | ||
| value: $(params.repo-appstream-url) | ||
| - name: rh-account-secret | ||
| value: $(params.rh-account-secret) | ||
| - name: image-id | ||
| value: $(params.image-id) | ||
| - name: correlation | ||
| value: $(tasks.correlate.results.correlation) | ||
| - name: workspace-resources-path | ||
| value: $(tasks.correlate.results.workspace-resources-path) | ||
| workspaces: | ||
| - name: pipelines-data | ||
| workspace: pipelines-data | ||
| timeout: "45m" | ||
| - name: sync-provision | ||
| runAfter: | ||
| - s3-info | ||
| - crc-info | ||
| taskRef: | ||
| name: infra-syncer | ||
| params: | ||
| - name: workspace-resources-path | ||
| value: $(tasks.correlate.results.workspace-resources-path) | ||
| workspaces: | ||
| - name: pipelines-data | ||
| workspace: pipelines-data | ||
| timeout: "20m" | ||
| - name: install | ||
| runAfter: | ||
| - sync-provision | ||
| taskRef: | ||
| name: crc-preparer | ||
| params: | ||
| - name: platform | ||
| value: $(params.target-platform) | ||
| - name: ip | ||
| value: $(tasks.sync-provision.results.host) | ||
| - name: username | ||
| value: $(tasks.sync-provision.results.username) | ||
| - name: key | ||
| value: $(tasks.sync-provision.results.key-filename) | ||
| - name: workspace-resources-path | ||
| value: $(tasks.correlate.results.workspace-resources-path) | ||
| # Preparing crc version is only used for target path storage | ||
| - name: downloadable-version | ||
| value: $(tasks.crc-info.results.crc-version) | ||
| - name: downloadable-url | ||
| value: $(params.downloadable-url) | ||
| - name: shasum-file | ||
| value: $(params.shasum-file) | ||
| workspaces: | ||
| - name: pipelines-data | ||
| workspace: pipelines-data | ||
| timeout: "1h" | ||
| - name: qe | ||
| runAfter: | ||
| - install | ||
| taskRef: | ||
| name: crc-qe-cli | ||
| params: | ||
| - name: platform | ||
| value: $(params.target-platform) | ||
| - name: ip | ||
| value: $(tasks.sync-provision.results.host) | ||
| - name: username | ||
| value: $(tasks.sync-provision.results.username) | ||
| - name: key | ||
| value: $(tasks.sync-provision.results.key-filename) | ||
| - name: workspace-resources-path | ||
| value: $(tasks.correlate.results.workspace-resources-path) | ||
| - name: worspace-qe-subpath | ||
| value: $(params.qe-worspace-subpath) | ||
| # This is used to run e2e and integration containers | ||
| # Need to integrate a build from main and make it accessible | ||
| # For the time been we are good with latest released version | ||
| - name: crc-version | ||
| value: $(tasks.crc-info.results.crc-version) | ||
| - name: run-e2e | ||
| value: $(params.run-e2e) | ||
| - name: e2e-tag | ||
| value: $(params.e2e-tag) | ||
| - name: run-integration | ||
| value: $(params.run-integration) | ||
| workspaces: | ||
| - name: pipelines-data | ||
| workspace: pipelines-data | ||
| timeout: "5h" | ||
| - name: datalake-sink | ||
| runAfter: | ||
| - qe | ||
| taskRef: | ||
| name: s3-sink-workspace | ||
| params: | ||
| - name: s3-url | ||
| value: $(tasks.s3-info.results.upload-url) | ||
| - name: s3-access-key | ||
| value: $(tasks.s3-info.results.access-key) | ||
| - name: s3-secret-key | ||
| value: $(tasks.s3-info.results.secret-key) | ||
| - name: s3-folder-path | ||
| value: $(tasks.s3-info.results.upload-path) | ||
| - name: workspace-path | ||
| value: $(tasks.correlate.results.workspace-resources-path) | ||
| - name: workspace-subpath | ||
| value: $(params.qe-worspace-subpath) | ||
| # Do not delete to keep tf state to decomission on final | ||
| # This task should be splitted to allow only cleaning up | ||
| - name: workspace-cleanup | ||
| value: 'false' | ||
| workspaces: | ||
| - name: pipelines-data | ||
| workspace: pipelines-data | ||
| timeout: "15m" | ||
| - name: url-generator | ||
| taskRef: | ||
| name: url-generator | ||
| runAfter: | ||
| - datalake-sink | ||
| params: | ||
| - name: base-url | ||
| value: $(tasks.s3-info.results.download-url) | ||
| - name: context | ||
| value: $(params.qe-worspace-subpath) | ||
| - name: asset-name | ||
| value: e2e-non-ux.xml | ||
| finally: | ||
| - name: decomission-fedora | ||
| when: | ||
| - input: $(params.fedora-version) | ||
| operator: notin | ||
| values: [''] | ||
| taskRef: | ||
| name: infra-fedora-decommission | ||
| params: | ||
| - name: infra-id | ||
| # This is the value composed on infra provision for project | ||
| # if we keep the link with provision task if it fails this one will not run | ||
| value: crc-qe-virtualized-fedora-$(tasks.correlate.results.correlation) | ||
| - name: fedora-version | ||
| value: $(params.fedora-version) | ||
| - name: workspace-resources-path | ||
| value: $(tasks.correlate.results.workspace-resources-path) | ||
| workspaces: | ||
| - name: pipelines-data | ||
| workspace: pipelines-data | ||
| timeout: "15m" | ||
| - name: decomission-rhel | ||
| when: | ||
| - input: $(params.rhel-version) | ||
| operator: notin | ||
| values: [''] | ||
| taskRef: | ||
| name: infra-rhel-decommission | ||
| params: | ||
| - name: infra-id | ||
| # This is the value composed on infra provision for project | ||
| # if we keep the link with provision task if it fails this one will not run | ||
| value: crc-qe-virtualized-rhel-$(tasks.correlate.results.correlation) | ||
| - name: rhel-version | ||
| value: $(params.rhel-version) | ||
| - name: workspace-resources-path | ||
| value: $(tasks.correlate.results.workspace-resources-path) | ||
| workspaces: | ||
| - name: pipelines-data | ||
| workspace: pipelines-data | ||
| timeout: "15m" | ||
|
|
||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,22 @@ | ||
| --- | ||
| apiVersion: tekton.dev/v1beta1 | ||
| kind: PipelineRun | ||
| metadata: | ||
| generateName: crc-qe-virtualized-fedora37- | ||
| spec: | ||
| pipelineRef: | ||
| name: crc-qe-virtualized | ||
| params: | ||
| - name: fedora-version | ||
| value: Fedora-Cloud-Base-37 | ||
| - name: downloadable-url | ||
| value: https://download.eng.bos.redhat.com/staging-cds/developer/crc/2.9.0-1/staging | ||
| - name: shasum-file | ||
| value: sha256sum.txt | ||
| - name: s3-folder-path | ||
| value: 'release/2.9.0' | ||
| workspaces: | ||
| - name: pipelines-data | ||
| persistentVolumeClaim: | ||
| claimName: pipelines-data | ||
| timeout: "5h" |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Resolve incomplete documentation and task list duplication.
The "#REVIEW" marker (line 23) indicates incomplete review. Additionally, the task list contains apparent duplicates (gather-run-info, gather-crc-info, and crc-preparer appear in both lines 15–22 and 24–27), and line 35 has a typo: "url-generator.yamls" should be "url-generator.yaml". Please clarify the intended task list and remove the marker before merge.
🤖 Prompt for AI Agents