rancher-turtles-e2e

What tests are doing:

1. Create the infra stack ( GCP runner, cert-manager, rancher )
2. Install the Turtles chart with locally built latest chart
3. Deploy the Turtles UI extension
4. Test the Turtles menu, namespaces import features
5. Perform CAPI setup prerequisites
6. Create & Import CAPI cluster using fleet by cluster, namespace annotation
7. Install App on imported CAPI cluster
8. Scale the imported CAPI cluster
9. Remove & Delete the imported CAPI cluster
10. Migration test from 2.12 to 2.13 to test turtles migration from an external chart(2.12) to system integrated chart( 2.13). These tests are only supported with dev=true options; dev=true is applicable to 2.13.
11. Upgrade tests from 2.13 to 2.14 to test Turtles & CAPI upgrade from v1.10 to v1.12.
12. Add Feature Switch test for 2.13 to test switch between embedded-cluster-api and turtles features switch.

Running the tests locally

Pre-requisites

1. Install Rancher.
2. Install Rancher Turtles chart.
3. Install CAPI UI Extension.

Running the test

1. cd tests/cypress/latest
2. Install Cypress and its dependencies: npm install
3. Export the following ENV VAR: RANCHER_URL (format: https://<FQDN>/dashboard), RANCHER_PASSWORD, RANCHER_USER, GREPTAGS=@install [@short @full @upgrade @migration | @vsphere ], and provider specific env var:
   1. CAPA - AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY
   2. CAPG - GCP_CREDENTIALS and GCP_PROJECT
   3. CAPZ - AZURE_TENANT_ID, AZURE_CLIENT_ID, AZURE_CLIENT_SECRET, and AZURE_SUBSCRIPTION_ID.
   4. CAPV - VSPHERE_ENDPOINTS_LIST (workflow only, otherwise hardcoded endpoint_ip is part of the following var), VSPHERE_SECRETS_JSON_BASE64, see here for reference.
4. Start Cypress: ./node_modules/cypress/bin/cypress open -C cypress.config.ts or npx cypress open --env grepTags="@install @short"

The Cypress GUI should now be visible.

---

Test structure

We primarily categorize our tests using tags such as short, full, vsphere, and install. Tests tagged with short are local (docker-based) tests, while those tagged with vsphere are specific to vSphere. Tests tagged with full are cloud provider-based tests. The install tag is used for initial setup tests ('install' tag is also to be included in setup tests title).

Additional tags that are supported:

Tag	Test
@install	Initial test setup (install rancher, rancher turtles, rancher turtles providers and CAPI UI Extension
@short	Docker Provider tests
@full	Cloud Providers (CAPA, CAPG, CAPZ) tests
@vsphere	VSphere (CAPV) Provider tests
@capXk	Provider X (X=Docker, VSphere, Google, Azure, AWS) & Kubeadm
@capXr	Provider X (X=Docker, VSphere, Google, Azure, AWS) & RKE2
@capgke	CAPG GKE
@capaeks	CAPA EKS
@capzaks	CAPZ AKS
@migration	Migration from 2.12 (Externally-managed Rancher Turtles) to 2.13 (System integration Rancher Turtles)
@switch	(2.13 Only) Switch from Turtles to Embedded-CAPI and back
@upgrade	Upgrade from 2.13 to 2.14

Running tests using Cypress grep

We have implemented tags for more precise selection of tests using a Cypress plugin called cypress-grep

Note: the title can be either at describe, context or it level.

By default, daily runs will run test with the tags@install, @short

To use locally use the tag --env grepTags=tag along with the npx command

For example:

npx cypress run -C cypress.config.ts  --env grepTags="@short" cypress/e2e/*.spec.ts