Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Task/APPENG-2738: NIM Cypress E2E tests #3298

Open
wants to merge 33 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 20 commits
Commits
Show all changes
33 commits
Select commit Hold shift + click to select a range
9947f96
test: added mocks and interceptors for nim cypress tests
TomerFi Oct 2, 2024
ba0f98c
Merging the base test cases of Daniel and Lokesh's code to the Tomer'…
lokeshrangineni Oct 2, 2024
d103247
Reverting the mock changes which are not needed.
lokeshrangineni Oct 2, 2024
b1e48d3
nim-specific classes and functions, partial rework of deployment test…
dmartinol Oct 2, 2024
2c7152d
Checking in the cypress test covers to validate all the scenarios to …
lokeshrangineni Oct 3, 2024
2424ab3
Added mockNimServingRuntimeTemplate mock function
dmartinol Oct 3, 2024
8175aa6
mocking deploy respurces (WIP)
dmartinol Oct 3, 2024
1d80e39
Completed deployment tests
dmartinol Oct 3, 2024
b662f9f
Added new test cases to validate if the nim is not enabled
lokeshrangineni Oct 3, 2024
47306a1
test: added test cases for deleting model from the project models tab
TomerFi Oct 3, 2024
34ab9df
* Added new test cases to validate the model serving menu item when t…
lokeshrangineni Oct 3, 2024
e57074a
test: cleanups
TomerFi Oct 4, 2024
b5c58a8
adding const for modal dialog title
dmartinol Oct 4, 2024
db769a1
test cases for list of models in different pages/tabs
dmartinol Oct 4, 2024
6d82381
linting fixes
dmartinol Oct 4, 2024
5de4b14
removed experimentalStudio option
dmartinol Oct 4, 2024
a765cb9
test: split nim tests per pages (#13)
TomerFi Oct 4, 2024
e23d11a
testing more buttons in modal dialog (#14)
dmartinol Oct 4, 2024
ca6ef78
Added a test case to address there is a failure in loading Nvidia Nim…
lokeshrangineni Oct 4, 2024
b1c2492
Added a test case to address there is a failure in loading Nvidia Nim…
lokeshrangineni Oct 4, 2024
c480d4e
test: fix linting errors
TomerFi Oct 4, 2024
baf470e
test: fix review change requests 1 - see body
TomerFi Oct 7, 2024
7da5544
test: fix review change requests 2 - see body
TomerFi Oct 7, 2024
eba0293
test: create page objects for verifying nim model tabels
TomerFi Oct 7, 2024
ec116f6
Update frontend/src/__tests__/cypress/cypress/tests/mocked/projects/m…
TomerFi Oct 7, 2024
bdc58ab
test: fix linting errors
TomerFi Oct 7, 2024
eaa0e47
test: rewrite test case for global model list nim
TomerFi Oct 8, 2024
dd7ac09
test: fix linting errors
TomerFi Oct 8, 2024
7c06f11
test: rewrite test case for project overview list nim
TomerFi Oct 8, 2024
aed9685
test: rewrite test case for project models tab list nim
TomerFi Oct 8, 2024
1cafc5d
test: cleanups
TomerFi Oct 8, 2024
cb01605
test: rewrite test cases nim enablement
TomerFi Oct 9, 2024
bd71dc4
test: final touchups
TomerFi Oct 9, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion frontend/src/__mocks__/mockDashboardConfig.ts
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
import { DashboardConfigKind, KnownLabels } from '~/k8sTypes';
import { NotebookSize } from '~/types';

type MockDashboardConfigType = {
export type MockDashboardConfigType = {
disableInfo?: boolean;
disableSupport?: boolean;
disableClusterManager?: boolean;
Expand Down
132 changes: 132 additions & 0 deletions frontend/src/__mocks__/mockNimResource.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,132 @@
import {
ConfigMapKind,
InferenceServiceKind,
PersistentVolumeClaimKind,
SecretKind,
ServingRuntimeKind,
TemplateKind,
} from '~/k8sTypes';
import { ServingRuntimeAPIProtocol, ServingRuntimePlatform } from '~/types';
import { mockConfigMap } from './mockConfigMap';
import { mockServingRuntimeK8sResource } from './mockServingRuntimeK8sResource';
import { mockInferenceServiceK8sResource } from './mockInferenceServiceK8sResource';
import { mockServingRuntimeTemplateK8sResource } from './mockServingRuntimeTemplateK8sResource';
import { mockSecretK8sResource } from './mockSecretK8sResource';
import { mockPVCK8sResource } from './mockPVCK8sResource';

export const mockNimImages = (): ConfigMapKind =>
mockConfigMap({
name: 'nvidia-nim-images-data',
namespace: 'opendatahub',
data: {
alphafold2:
'{' +
' "name": "alphafold2",' +
' "displayName": "AlphaFold2",' +
' "shortDescription": "A widely used model for predicting the 3D structures of proteins from their amino acid sequences.",' +
' "namespace": "nim/deepmind",' +
' "tags": [' +
' "1.0.0"' +
' ],' +
' "latestTag": "1.0.0",' +
' "updatedDate": "2024-08-27T01:51:55.642Z"' +
' }',
'arctic-embed-l':
'{' +
' "name": "arctic-embed-l",' +
' "displayName": "Snowflake Arctic Embed Large Embedding",' +
' "shortDescription": "NVIDIA NIM for GPU accelerated Snowflake Arctic Embed Large Embedding inference",' +
' "namespace": "nim/snowflake",' +
' "tags": [' +
' "1.0.1",' +
' "1.0.0"' +
' ],' +
' "latestTag": "1.0.1",' +
' "updatedDate": "2024-07-27T00:38:40.927Z"' +
' }',
},
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perhaps to make this easier to read just make it a normal JS object and do JSON.stringify if you need a single string value.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Resolved here: baf470e

});

export const mockNimInferenceService = (): InferenceServiceKind => {
const inferenceService = mockInferenceServiceK8sResource({
name: 'test-name',
modelName: 'test-name',
displayName: 'Test Name',
kserveInternalLabel: true,
resources: {
limits: { cpu: '2', memory: '8Gi' },
requests: { cpu: '1', memory: '4Gi' },
},
});
delete inferenceService.metadata.labels?.name;
delete inferenceService.metadata.creationTimestamp;
delete inferenceService.metadata.generation;
delete inferenceService.metadata.resourceVersion;
delete inferenceService.metadata.uid;
if (inferenceService.spec.predictor.model?.modelFormat) {
inferenceService.spec.predictor.model.modelFormat.name = 'arctic-embed-l';
}
delete inferenceService.spec.predictor.model?.modelFormat?.version;
delete inferenceService.spec.predictor.model?.storage;
delete inferenceService.status;

return inferenceService;
};

export const mockNimServingRuntime = (): ServingRuntimeKind => {
const servingRuntime = mockServingRuntimeK8sResource({
name: 'test-name',
displayName: 'Test Name',
});
if (servingRuntime.metadata.annotations) {
servingRuntime.metadata.annotations['opendatahub.io/template-display-name'] = 'NVIDIA NIM';
servingRuntime.metadata.annotations['opendatahub.io/template-name'] = 'nvidia-nim-runtime';
}

return servingRuntime;
};

export const mockNimServingRuntimeTemplate = (): TemplateKind => {
const templateMock = mockServingRuntimeTemplateK8sResource({
name: 'nvidia-nim-serving-template',
displayName: 'NVIDIA NIM',
platforms: [ServingRuntimePlatform.SINGLE],
apiProtocol: ServingRuntimeAPIProtocol.REST,
namespace: 'opendatahub',
});
if (templateMock.metadata.annotations != null) {
templateMock.metadata.annotations['opendatahub.io/dashboard'] = 'true';
}

return templateMock;
};

export const mockNvidiaNimAccessSecret = (): SecretKind => {
const secret = mockSecretK8sResource({
name: 'nvidia-nim-access',
});
delete secret.data;
secret.data = {};
secret.data.api_key = 'api-key'; // eslint-disable-line camelcase
secret.data.configMapName = 'bnZpZGlhLW5pbS12YWxpZGF0aW9uLXJlc3VsdA==';

return secret;
};

export const mockNvidiaNimImagePullSecret = (): SecretKind => {
const secret = mockSecretK8sResource({
name: 'nvidia-nim-image-pull',
});
delete secret.data;
secret.data = {};
secret.data['.dockerconfigjson'] = 'ZG9ja2VyY29uZmlnCg==';

return secret;
};

export const mockNimModelPVC = (): PersistentVolumeClaimKind => {
const pvc = mockPVCK8sResource({
name: 'nim-pvc',
});
return pvc;
};
1 change: 1 addition & 0 deletions frontend/src/__tests__/cypress/cypress.config.ts
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@ import { env, cypressEnv, BASE_URL } from '~/__tests__/cypress/cypress/utils/tes
const resultsDir = `${env.CY_RESULTS_DIR || 'results'}/${env.CY_MOCK ? 'mocked' : 'e2e'}`;

export default defineConfig({
experimentalStudio: false,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Was this needed? Changing base configs I imagine is not necessary here...

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it was helpful to debug by running the one testcase and add some html code by enabling this flag. it is disabled now if you want I am happy to take it out.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Resolved here: baf470e

experimentalMemoryManagement: true,
// Use relative path as a workaround to https://github.com/cypress-io/cypress/issues/6406
reporter: '../../../node_modules/cypress-multi-reporters',
Expand Down
45 changes: 45 additions & 0 deletions frontend/src/__tests__/cypress/cypress/pages/nimModelDialog.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
import { Modal } from '~/__tests__/cypress/cypress/pages/components/Modal';

class NIMDeployModal extends Modal {
constructor(private edit = false) {
super(`${edit ? 'Edit' : 'Deploy'} model with NVIDIA NIM`);
}

findSubmitButton() {
return this.findFooter().findByTestId('modal-submit-button');
}

findModelNameInput() {
return this.find().findByTestId('model-deployment-name-section');
}

findNIMToDeploy() {
return this.find().findByTestId('nim-model-list-selection');
}

findNimStorageSizeInput() {
return cy.get('[data-testid="pvc-size"] input');
}

findStorageSizeMinusButton() {
return this.find().findByTestId('pvc-size').findByRole('button', { name: 'Minus' });
}

findStorageSizePlusButton() {
return this.find().findByTestId('pvc-size').findByRole('button', { name: 'Plus' });
}

findNimModelReplicas() {
return cy.get('[id="model-server-replicas"]');
}

findNimModelReplicasMinusButton() {
return this.find().find('button[aria-label="Minus"]').eq(1);
}

findNimModelReplicasPlusButton() {
return this.find().find('button[aria-label="Plus"]').eq(1);
}
}

export const nimDeployModal = new NIMDeployModal();
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
import {
initInterceptsToEnableNim,
validateNimInmferenceModelsTable,
} from '~/__tests__/cypress/cypress/utils/nimUtils';
import { mockNimInferenceService, mockNimServingRuntime } from '~/__mocks__/mockNimResource';
import {
InferenceServiceModel,
ServingRuntimeModel,
} from '~/__tests__/cypress/cypress/utils/models';
import { mockK8sResourceList } from '~/__mocks__';
import { modelServingGlobal } from '~/__tests__/cypress/cypress/pages/modelServing';

describe('NIM Models Deployments', () => {
it('should be listed in the global models list', () => {
initInterceptsToEnableNim({ hasAllModels: false });
cy.interceptK8sList(InferenceServiceModel, mockK8sResourceList([mockNimInferenceService()]));
cy.interceptK8sList(ServingRuntimeModel, mockK8sResourceList([mockNimServingRuntime()]));

modelServingGlobal.visit('test-project');

validateNimInmferenceModelsTable();
});

it('should only be allowed to be deleted, no edit', () => {
initInterceptsToEnableNim({});
cy.interceptK8sList(InferenceServiceModel, mockK8sResourceList([mockNimInferenceService()]));
cy.interceptK8sList(ServingRuntimeModel, mockK8sResourceList([mockNimServingRuntime()]));

modelServingGlobal.visit('test-project');
modelServingGlobal.getModelRow('Test Name').get('button[aria-label="Kebab toggle"]').click();

modelServingGlobal
.getModelRow('Test Name')
.get('button[role="menuitem"]')
.should('have.length', 1);
modelServingGlobal
.getModelRow('Test Name')
.get('button[role="menuitem"]')
.contains('Delete')
.should('exist');
});
});
Loading
Loading