Add integration tests for the remote inference feature #1073

ryanbogan · 2023-07-11T17:31:18Z

Description

Adds integration tests for the remote inference feature

Check List

New functionality includes testing.
- All tests pass
New functionality has been documented.
- New functionality has javadoc added
Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Signed-off-by: Ryan Bogan <[email protected]>

ylwu-amzn · 2023-07-11T17:57:23Z

Take a look at the failed cases?

31 tests completed, 4 failed, 12 skipped

Tests with failures:
 - org.opensearch.ml.rest.RestMLRemoteInferenceIT.testOpenAIModerationsModel
 - org.opensearch.ml.rest.RestMLRemoteInferenceIT.testOpenAIEditsModel
 - org.opensearch.ml.rest.RestMLRemoteInferenceIT.testPredictRemoteModel
 - org.opensearch.ml.rest.RestMLRemoteInferenceIT.testCohereGenerateTextModel

ryanbogan · 2023-07-11T17:58:44Z

Those are expected failures until the keys are added to GitHub secrets on the ml-commons repository

ylwu-amzn · 2023-07-11T18:04:22Z

Those are expected failures until the keys are added to GitHub secrets on the ml-commons repository

Got it , thanks. Who will help add the keys to Github secrets?

ryanbogan · 2023-07-11T18:07:57Z

Those are expected failures until the keys are added to GitHub secrets on the ml-commons repository

Got it , thanks. Who will help add the keys to Github secrets?

It has to be someone with admin permissions on the repository.

peterzhuamazon · 2023-07-11T21:06:47Z

I am adding the key sections to the github workflows after talking to @ryanbogan and @ylwu-amzn offline and make required change in their accounts.

peterzhuamazon · 2023-07-11T21:08:01Z

Remove dco.yml as it is replaced by dco apps already.

Signed-off-by: Ryan Bogan <[email protected]>

plugin/src/test/java/org/opensearch/ml/rest/RestMLRemoteInferenceIT.java

Signed-off-by: Ryan Bogan <[email protected]>

* Add remote inference integration tests for OpenAI and Cohere Signed-off-by: Ryan Bogan <[email protected]> * Change edits model test Signed-off-by: Ryan Bogan <[email protected]> * Change loop number in edits test Signed-off-by: Ryan Bogan <[email protected]> * Change edits model test again Signed-off-by: Ryan Bogan <[email protected]> * Fix spotless Signed-off-by: Ryan Bogan <[email protected]> * Refactoring Signed-off-by: Ryan Bogan <[email protected]> * Fix typo Signed-off-by: Ryan Bogan <[email protected]> * Remove env vars Signed-off-by: Peter Zhu <[email protected]> * Removing unnecessary comments Signed-off-by: Ryan Bogan <[email protected]> * Rebase with 2.x and remove set up for master key Signed-off-by: Ryan Bogan <[email protected]> * Remove print line Signed-off-by: Ryan Bogan <[email protected]> * Add throttling check Signed-off-by: Ryan Bogan <[email protected]> * Minor change for assert statement Signed-off-by: Ryan Bogan <[email protected]> * Test changes Signed-off-by: Ryan Bogan <[email protected]> * Uncomment delete connector test Signed-off-by: Ryan Bogan <[email protected]> --------- Signed-off-by: Ryan Bogan <[email protected]> Signed-off-by: Peter Zhu <[email protected]> Co-authored-by: Peter Zhu <[email protected]> (cherry picked from commit 3e56ad5)

* Add remote inference integration tests for OpenAI and Cohere Signed-off-by: Ryan Bogan <[email protected]> * Change edits model test Signed-off-by: Ryan Bogan <[email protected]> * Change loop number in edits test Signed-off-by: Ryan Bogan <[email protected]> * Change edits model test again Signed-off-by: Ryan Bogan <[email protected]> * Fix spotless Signed-off-by: Ryan Bogan <[email protected]> * Refactoring Signed-off-by: Ryan Bogan <[email protected]> * Fix typo Signed-off-by: Ryan Bogan <[email protected]> * Remove env vars Signed-off-by: Peter Zhu <[email protected]> * Removing unnecessary comments Signed-off-by: Ryan Bogan <[email protected]> * Rebase with 2.x and remove set up for master key Signed-off-by: Ryan Bogan <[email protected]> * Remove print line Signed-off-by: Ryan Bogan <[email protected]> * Add throttling check Signed-off-by: Ryan Bogan <[email protected]> * Minor change for assert statement Signed-off-by: Ryan Bogan <[email protected]> * Test changes Signed-off-by: Ryan Bogan <[email protected]> * Uncomment delete connector test Signed-off-by: Ryan Bogan <[email protected]> --------- Signed-off-by: Ryan Bogan <[email protected]> Signed-off-by: Peter Zhu <[email protected]> Co-authored-by: Peter Zhu <[email protected]> (cherry picked from commit 3e56ad5) Signed-off-by: Yaliang Wu <[email protected]>

* Add remote inference integration tests for OpenAI and Cohere * Change edits model test * Change loop number in edits test * Change edits model test again * Fix spotless * Refactoring * Fix typo * Remove env vars * Removing unnecessary comments * Rebase with 2.x and remove set up for master key * Remove print line * Add throttling check * Minor change for assert statement * Test changes * Uncomment delete connector test --------- (cherry picked from commit 3e56ad5) Signed-off-by: Ryan Bogan <[email protected]> Signed-off-by: Peter Zhu <[email protected]> Signed-off-by: Yaliang Wu <[email protected]> Co-authored-by: Ryan Bogan <[email protected]> Co-authored-by: Peter Zhu <[email protected]>

…oject#1073) * Add remote inference integration tests for OpenAI and Cohere Signed-off-by: Ryan Bogan <[email protected]> * Change edits model test Signed-off-by: Ryan Bogan <[email protected]> * Change loop number in edits test Signed-off-by: Ryan Bogan <[email protected]> * Change edits model test again Signed-off-by: Ryan Bogan <[email protected]> * Fix spotless Signed-off-by: Ryan Bogan <[email protected]> * Refactoring Signed-off-by: Ryan Bogan <[email protected]> * Fix typo Signed-off-by: Ryan Bogan <[email protected]> * Remove env vars Signed-off-by: Peter Zhu <[email protected]> * Removing unnecessary comments Signed-off-by: Ryan Bogan <[email protected]> * Rebase with 2.x and remove set up for master key Signed-off-by: Ryan Bogan <[email protected]> * Remove print line Signed-off-by: Ryan Bogan <[email protected]> * Add throttling check Signed-off-by: Ryan Bogan <[email protected]> * Minor change for assert statement Signed-off-by: Ryan Bogan <[email protected]> * Test changes Signed-off-by: Ryan Bogan <[email protected]> * Uncomment delete connector test Signed-off-by: Ryan Bogan <[email protected]> --------- Signed-off-by: Ryan Bogan <[email protected]> Signed-off-by: Peter Zhu <[email protected]> Co-authored-by: Peter Zhu <[email protected]>

* Add remote inference integration tests for OpenAI and Cohere Signed-off-by: Ryan Bogan <[email protected]> * Change edits model test Signed-off-by: Ryan Bogan <[email protected]> * Change loop number in edits test Signed-off-by: Ryan Bogan <[email protected]> * Change edits model test again Signed-off-by: Ryan Bogan <[email protected]> * Fix spotless Signed-off-by: Ryan Bogan <[email protected]> * Refactoring Signed-off-by: Ryan Bogan <[email protected]> * Fix typo Signed-off-by: Ryan Bogan <[email protected]> * Remove env vars Signed-off-by: Peter Zhu <[email protected]> * Removing unnecessary comments Signed-off-by: Ryan Bogan <[email protected]> * Rebase with 2.x and remove set up for master key Signed-off-by: Ryan Bogan <[email protected]> * Remove print line Signed-off-by: Ryan Bogan <[email protected]> * Add throttling check Signed-off-by: Ryan Bogan <[email protected]> * Minor change for assert statement Signed-off-by: Ryan Bogan <[email protected]> * Test changes Signed-off-by: Ryan Bogan <[email protected]> * Uncomment delete connector test Signed-off-by: Ryan Bogan <[email protected]> --------- Signed-off-by: Ryan Bogan <[email protected]> Signed-off-by: Peter Zhu <[email protected]> Co-authored-by: Peter Zhu <[email protected]>

ryanbogan added 7 commits July 11, 2023 07:44

Add remote inference integration tests for OpenAI and Cohere

8ffe4cb

Signed-off-by: Ryan Bogan <[email protected]>

Change edits model test

fe21a5f

Signed-off-by: Ryan Bogan <[email protected]>

Change loop number in edits test

d10e4d3

Signed-off-by: Ryan Bogan <[email protected]>

Change edits model test again

2c4fac6

Signed-off-by: Ryan Bogan <[email protected]>

Fix spotless

505087a

Signed-off-by: Ryan Bogan <[email protected]>

Refactoring

6ce0f7c

Signed-off-by: Ryan Bogan <[email protected]>

Fix typo

e2d28f5

Signed-off-by: Ryan Bogan <[email protected]>

ylwu-amzn previously approved these changes Jul 11, 2023

View reviewed changes

peterzhuamazon dismissed ylwu-amzn’s stale review via 38d0195 July 11, 2023 21:08

peterzhuamazon force-pushed the write_remote_inference_integ_tests branch from 51d9f25 to e2d28f5 Compare July 11, 2023 23:37

peterzhuamazon added backport 2.9 release v2.9.0 v2.9.0 test ci infra labels Jul 12, 2023

peterzhuamazon mentioned this pull request Jul 12, 2023

Add test keys to CI related workflows #1099

Merged

5 tasks

peterzhuamazon had a problem deploying to ml-commons-cicd-env July 12, 2023 18:42 — with GitHub Actions Failure

peterzhuamazon had a problem deploying to ml-commons-cicd-env July 12, 2023 18:42 — with GitHub Actions Error

peterzhuamazon had a problem deploying to ml-commons-cicd-env July 12, 2023 18:42 — with GitHub Actions Failure

peterzhuamazon had a problem deploying to ml-commons-cicd-env July 12, 2023 18:48 — with GitHub Actions Failure

peterzhuamazon had a problem deploying to ml-commons-cicd-env July 12, 2023 18:48 — with GitHub Actions Error

ryanbogan temporarily deployed to ml-commons-cicd-env July 12, 2023 23:34 — with GitHub Actions Inactive

Minor change for assert statement

dc027bb

Signed-off-by: Ryan Bogan <[email protected]>

ryanbogan temporarily deployed to ml-commons-cicd-env July 12, 2023 23:34 — with GitHub Actions Inactive

peterzhuamazon mentioned this pull request Jul 12, 2023

[RELEASE] Release version 2.9.0 opensearch-project/opensearch-build#3616

Closed

42 tasks

Test changes

0ca24cf

Signed-off-by: Ryan Bogan <[email protected]>

ryanbogan temporarily deployed to ml-commons-cicd-env July 12, 2023 23:48 — with GitHub Actions Inactive

Zhangxunmt reviewed Jul 12, 2023

View reviewed changes

plugin/src/test/java/org/opensearch/ml/rest/RestMLRemoteInferenceIT.java Outdated Show resolved Hide resolved

ryanbogan added 2 commits July 13, 2023 00:04

Merge branch '2.x' into write_remote_inference_integ_tests

f913882

Uncomment delete connector test

d43f3a4

Signed-off-by: Ryan Bogan <[email protected]>

ryanbogan temporarily deployed to ml-commons-cicd-env July 13, 2023 00:06 — with GitHub Actions Inactive

Zhangxunmt approved these changes Jul 13, 2023

View reviewed changes

ylwu-amzn approved these changes Jul 13, 2023

View reviewed changes

ylwu-amzn merged commit 3e56ad5 into opensearch-project:2.x Jul 13, 2023
9 checks passed

opensearch-trigger-bot bot mentioned this pull request Jul 13, 2023

[Backport 2.9] Add integration tests for the remote inference feature #1109

Merged

peterzhuamazon mentioned this pull request Jul 17, 2023

Migrate Keys from ml-commons AWS secret manager opensearch-project/opensearch-build#3793

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add integration tests for the remote inference feature #1073

Add integration tests for the remote inference feature #1073

ryanbogan commented Jul 11, 2023

ylwu-amzn commented Jul 11, 2023

ryanbogan commented Jul 11, 2023

ylwu-amzn commented Jul 11, 2023

ryanbogan commented Jul 11, 2023

peterzhuamazon commented Jul 11, 2023 •

edited

Loading

peterzhuamazon commented Jul 11, 2023

Add integration tests for the remote inference feature #1073

Add integration tests for the remote inference feature #1073

Conversation

ryanbogan commented Jul 11, 2023

Description

Check List

ylwu-amzn commented Jul 11, 2023

ryanbogan commented Jul 11, 2023

ylwu-amzn commented Jul 11, 2023

ryanbogan commented Jul 11, 2023

peterzhuamazon commented Jul 11, 2023 • edited Loading

peterzhuamazon commented Jul 11, 2023

peterzhuamazon commented Jul 11, 2023 •

edited

Loading