Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

rename direct attack sim #3705

Merged
merged 8 commits into from
Sep 5, 2024
Merged

rename direct attack sim #3705

merged 8 commits into from
Sep 5, 2024

Conversation

MilesHolland
Copy link
Member

Renames the JailbreakAdversarialSimulator to DirectAttackSimulator in accordance with the relevant Spec.

Copy link

github-actions bot commented Aug 30, 2024

promptflow-evals test result

 12 files  ±  0   12 suites  ±0   1h 50m 46s ⏱️ + 1h 30m 24s
 54 tests  -  62   51 ✅  -  65   3 💤 + 3  0 ❌ ±0 
648 runs   - 744  612 ✅  - 780  36 💤 +36  0 ❌ ±0 

Results for commit b782931. ± Comparison against base commit d5cc19c.

This pull request removes 116 and adds 54 tests. Note that renamed tests count towards both.
tests.evals.unittests.test_batch_run_context.TestBatchRunContext ‑ test_batch_timeout_custom
tests.evals.unittests.test_batch_run_context.TestBatchRunContext ‑ test_batch_timeout_default
tests.evals.unittests.test_batch_run_context.TestBatchRunContext ‑ test_with_codeclient
tests.evals.unittests.test_batch_run_context.TestBatchRunContext ‑ test_with_pfclient
tests.evals.unittests.test_built_in_evaluator.TestBuiltInEvaluators ‑ test_fluency_evaluator
tests.evals.unittests.test_built_in_evaluator.TestBuiltInEvaluators ‑ test_fluency_evaluator_empty_string
tests.evals.unittests.test_built_in_evaluator.TestBuiltInEvaluators ‑ test_fluency_evaluator_non_string_inputs
tests.evals.unittests.test_chat_evaluator.TestChatEvaluator ‑ test_conversation_validation_invalid_citations
tests.evals.unittests.test_chat_evaluator.TestChatEvaluator ‑ test_conversation_validation_missing_role
tests.evals.unittests.test_chat_evaluator.TestChatEvaluator ‑ test_conversation_validation_normal
…
tests.evals.e2etests.test_builtin_evaluators.TestBuiltInEvaluators ‑ test_composite_evaluator_chat[False-True]
tests.evals.e2etests.test_builtin_evaluators.TestBuiltInEvaluators ‑ test_composite_evaluator_chat[True-True]
tests.evals.e2etests.test_builtin_evaluators.TestBuiltInEvaluators ‑ test_composite_evaluator_content_safety
tests.evals.e2etests.test_builtin_evaluators.TestBuiltInEvaluators ‑ test_composite_evaluator_content_safety_chat[False-False]
tests.evals.e2etests.test_builtin_evaluators.TestBuiltInEvaluators ‑ test_composite_evaluator_content_safety_chat[True-False]
tests.evals.e2etests.test_builtin_evaluators.TestBuiltInEvaluators ‑ test_composite_evaluator_qa[False]
tests.evals.e2etests.test_builtin_evaluators.TestBuiltInEvaluators ‑ test_composite_evaluator_qa[True]
tests.evals.e2etests.test_builtin_evaluators.TestBuiltInEvaluators ‑ test_composite_evaluator_qa_for_nans
tests.evals.e2etests.test_builtin_evaluators.TestBuiltInEvaluators ‑ test_composite_evaluator_qa_with_openai_config[False]
tests.evals.e2etests.test_builtin_evaluators.TestBuiltInEvaluators ‑ test_composite_evaluator_qa_with_openai_config[True]
…

♻️ This comment has been updated with latest results.

luigiw
luigiw previously approved these changes Aug 30, 2024
diondrapeck
diondrapeck previously approved these changes Aug 30, 2024
luigiw
luigiw previously approved these changes Aug 30, 2024
singankit
singankit previously approved these changes Aug 30, 2024
@minthigpen minthigpen self-requested a review September 5, 2024 17:16
@MilesHolland MilesHolland merged commit 7774d6e into main Sep 5, 2024
65 of 77 checks passed
@MilesHolland MilesHolland deleted the rename-jailbreak-to-upia branch September 5, 2024 17:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants