Merge pull request #1481 from Gkrumbach07/testing-arch

Testing arch added to docs
opendatahub-io · Jul 25, 2023 · c19babd · c19babd
2 parents a1b119b + f3c2691
commit c19babd
Show file tree

Hide file tree

Showing 2 changed files with 252 additions and 1 deletion.
diff --git a/docs/architecture.md b/docs/architecture.md
@@ -79,4 +79,255 @@ When building new client features, there are a few things worth noting about the
 
 ### Testing Structure
 
-TBD - This is new functionality that is being flushed out right now.
+![testing structure](./meta/testing.png)
+
+#### Test Structure
+
+Tests can be divided into the following categories: unit, integration, accessibility, and end to end testing. To keep organized of the different types of tests, there will be a test folder at the root of the frontend project with the following structure.
+
+
+```
+/frontend/tests
+	/integration => ComponentName.stories.tsx, ComponentName.spec.ts
+	/unit => functionName.test.ts
+	/e2e => storyName.spec.ts
+```
+
+Some nesting can be used to organize testing groups together. For example, the _projects_ page has screens for _details_, _projects_, and, _spawner_ which can be all grouped together under a projects folder. 
+
+
+#### Testing Types
+
+##### Unit Testing
+
+Unit tests cover util functions and other non React based functions. These tests are stored  in the `/unit `folder and can be organized into folders depending on their parent page and/or screen. Use Jest to test each function using `describe` to group together the utils file and the specific function. Then each test is described using `it`. Some functions are very basic and don't need a test. Use your best judgment if a test is needed.
+
+_Example_
+``` ts
+describe('Project view screen utils', () => {
+  describe('getDisplayNameFromK8sResource', () => {
+    it('returns the display name from resource metadata annotations', () => {
+      // Test 1
+    });
+    it('returns the resource name if display name is not present', () => {
+      // Test 2
+    });
+  });
+  // Write similar test cases for other functions in the utils file
+  describe('getSecretDescription', () => {
+    // Test case for getSecretDescription function
+  });
+});
+```
+
+##### Integration Testing
+
+Integration tests will be conducted on specific screens in the app using storybook stories and the playwright testing framework.
+
+
+###### Test setup steps
+
+1. Identify view components that don't require any props and create a story around the entire screen component that handles the business logic.
+2. Create a default export story configuration for the selected component and its child stories. Specify the API mocks using the `msw` parameter.
+
+Configure API mocks using Mock Service Worker (MSW) to simulate API calls. Only mock GET requests for fetching k8s resources; do not mock POST, PUT, or DELETE calls as the tests are run locally.
+
+
+``` ts
+export default {
+  component: ProjectView,
+  parameters: {
+    msw: {
+      handlers: [
+        rest.get(
+       '/api/k8s/apis/route.openshift.io/v1/namespaces/test-project/routes/test-notebook',
+          (req, res, ctx) => res(ctx.json(mockRouteK8sResource({}))),
+        ),
+      ],
+    },
+  },
+} as Meta<typeof ProjectView>;
+```
+
+
+3. Each exported member of the file (except the default export) represents a story and can have various parameters. For example, the a11y parameter can be used to set a custom root element for accessibility tests, such as modals. Optionally setup each test by defining the `play` function,  such as opening a modal, to ensure correct accessibility testing.
+
+``` tsx
+export const EditModel = {
+  render: Template,
+  parameters: {
+    a11y: {
+      // need to select modal as root
+      element: '.pf-c-backdrop',
+    },
+  },
+  play: async ({ canvasElement }) => {
+    // load page and wait until settled
+    const canvas = within(canvasElement);
+    await canvas.findByText('Test Inference Service', undefined, { timeout: 5000 });
+
+    // user flow for editing a project
+    await userEvent.click(canvas.getByLabelText('Actions', { selector: 'button' }));
+    await userEvent.click(canvas.getByText('Edit', { selector: 'button' }));
+  },
+};
+```
+
+4. Write the Playwright test by creating a new file in the same directory with the same name as the component file but with the `.spec.ts` extension. Test all edge cases for the component in the Playwright test, focusing on individual component behavior rather than longer flows that will be covered in end-to-end (e2e) testing. A typical tests could look like this:
+
+``` tsx
+import { test, expect } from '@playwright/test';
+
+test('Create project', async ({ page }) => {
+  await page.goto(
+  './iframe.html?id=tests-stories-pages-projects-projectview--create-project&viewMode=story',
+  );
+
+  // wait for page to load
+  await page.waitForSelector('text=Create data science project');
+
+  // Test that can submit on valid form
+  await page.getByLabel('Name *', { exact: true }).click();
+  await page.getByLabel('Name *', { exact: true }).fill('Test Project');
+  await page.getByLabel('Description').click();
+  await page.getByLabel('Description').fill('Test project description');
+  await expect(page.getByRole('button', { name: 'Create' })).toBeEnabled();
+});
+```
+
+5. Use the `page.goto()` function to navigate to the specific storybook story URL with viewMode=story which can be found by pressing the _open in new tab icon_ in the top right of a specific storybook story in its UI.
+
+To run storybook UI: `cd ./frontend && npm run storybook`
+
+``` ts
+await page.goto('./iframe.html?id=tests-stories-pages-projects-projectview--create-project&viewMode=story');
+```
+
+6. Wait for the page to load and the story to settle before performing any assertions or actions. Use `page.waitForSelector()` to wait for a specific element to appear as an indication of the story being loaded.
+
+``` ts
+await page.waitForSelector('text=Create data science project');
+```
+
+
+7. Perform tests and assertions using Playwright's API, interacting with elements and verifying their states.
+
+``` ts
+await page.getByLabel('Name *', { exact: true }).click();
+await page.getByLabel('Name *', { exact: true }).fill('Test Project');
+await page.getByLabel('Description').click();
+await page.getByLabel('Description').fill('Test project description');
+await expect(page.getByRole('button', { name: 'Create' })).toBeEnabled();
+```
+
+Note: Adjust the selectors in the code according to the specific component and testing requirements. Use the [Playwright API](https://playwright.dev/docs/locators) to find the appropriate selectors.
+
+1. To execute these tests you can run:
+
+``` bash
+npm run test:integration
+```
+
+This will either attach to an already running instance of storybook UI, or start up a new instance at port `6006.`
+
+##### Accessibility Testing
+
+Accessibility testing is covered automatically by an a11y storybook plugin. Tests will fail if there is an error determined by a11y. Run these tests with
+
+
+``` bash
+cd ./frontend && npm run storybook
+npm run test:accessibility
+```
+
+
+
+##### E2E Testing
+
+For end to end testing we will use Playwright just as we did with e2e testing. These are probably the easiest tests to write as there is a handy [VSCode plugin](https://marketplace.visualstudio.com/items?itemName=ms-playwright.playwright) to generate these tests for us. An e2e test covers a wider range of use cases and is not restricted to one screen or page. These tests should be grouped by user stories. Each file is an area to test such as pipelines or projects, and each test is a user story.
+
+For example, a possible e2e test might be under `projects.spec.ts` and a test called “Create, edit, and destroy a project” which would create a project, edit the project, and then delete the project. This test is nice because it does the cleanup for us. But there may be times when this is not the case, so make sure to [teardown](https://playwright.dev/docs/api-testing#setup-and-teardown) any hanging resources. 
+
+_Example_
+
+
+``` ts
+const cleanTestProject = async (page: Page) => {
+ await page.goto('/projects');
+ const count = await page.getByRole('link', { name: process.env.TEST_PROJECT_NAME }).count();
+ if (count > 0) {
+   await page
+     .locator('tr', {
+       has: page.locator(`text="${process.env.TEST_PROJECT_NAME}"`),
+     })
+     .getByRole('button', { name: 'Actions' })
+     .click();
+   await page.getByRole('menuitem', { name: 'Delete project' }).click();
+   await page
+     .getByRole('textbox', { name: 'Delete modal input' })
+     .fill(`${process.env.TEST_PROJECT_NAME}`);
+   await page.getByRole('button', { name: 'Delete project' }).click();
+ }
+};
+
+test.beforeEach(async ({ page }) => await cleanTestProject(page));
+test.afterEach(async ({ page }) => await cleanTestProject(page));
+```
+
+
+Remember to try to keep duplicate code to a minimum by utilizing setup and teardown calls.
+
+A full e2e test may look like this. Notice how it completes the user flow by deleting any resources it created. The teardown should be a fallback if the tests fail midway through.
+
+
+``` ts
+test('Create, edit, and delete a project', async ({ page }) => {
+ await page.goto('');
+
+ // create project
+ await page.getByRole('link', { name: 'Data Science Projects' }).click();
+ await page.getByRole('button', { name: 'Create data science project' }).click();
+ await page.getByLabel('Name *', { exact: true }).fill(`${process.env.TEST_PROJECT_NAME}`);
+ await page.getByRole('button', { name: 'Create' }).click();
+ expect(await page.getByText('Error creating project').count()).toBe(0);
+ expect(
+   await page
+     .locator('tr', { has: page.locator(`text="${process.env.TEST_PROJECT_NAME}"`) })
+     .count(),
+ ).toBe(0);
+ await page.getByRole('link', { name: 'Data Science Projects', exact: true }).click();
+
+ // edit project
+ await page
+   .locator('tr', { has: page.locator(`text="${process.env.TEST_PROJECT_NAME}"`) })
+   .getByRole('button', { name: 'Actions' })
+   .click();
+ await page.getByRole('menuitem', { name: 'Edit project' }).click();
+ await page.getByLabel('Name *', { exact: true }).fill(`${process.env.TEST_PROJECT_NAME}-name`);
+ await page.getByRole('button', { name: 'Update' }).click();
+ expect(
+   await page
+     .locator('tr', { has: page.locator(`text="${process.env.TEST_PROJECT_NAME}-name"`) })
+     .count(),
+ ).toBe(0);
+
+ // delete project
+ await page
+   .locator('tr', { has: page.locator(`text="${process.env.TEST_PROJECT_NAME}-name"`) })
+   .getByRole('button', { name: 'Actions' })
+   .click();
+ await page.getByRole('menuitem', { name: 'Delete project' }).click();
+ await page
+   .getByRole('textbox', { name: 'Delete modal input' })
+   .fill(`${process.env.TEST_PROJECT_NAME}-name`);
+ await page.getByRole('button', { name: 'Delete project' }).click();
+ expect(
+   await page
+     .locator('tr', { has: page.locator(`text="${process.env.TEST_PROJECT_NAME}"`) })
+     .count(),
+ ).toBe(0);
+});
+```
+
+
+> **Note**: Storybook was evaluated for reuse here, however it ended up causing problems because we would have needed a story for the whole application in some instances which defeats the purpose of using stories. The use case that would work would be to use storybook as a way to encapsulate the UI in smaller chunks and then have Playwright run user flows on those stories. Essentially pushing the _play_ function to a playwright test. The benefit of this is that we could use the VSCode plugin to make creating user flows faster. 
diff --git a/docs/meta/testing.png b/docs/meta/testing.png