Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Rename to personal attributes #40078

Open
wants to merge 2 commits into
base: main
Choose a base branch
from
Open

Conversation

w-javed
Copy link
Contributor

@w-javed w-javed commented Mar 14, 2025

No description provided.

@Copilot Copilot bot review requested due to automatic review settings March 14, 2025 17:16
@w-javed w-javed requested a review from a team as a code owner March 14, 2025 17:16
@github-actions github-actions bot added the Evaluation Issues related to the client library for Azure AI Evaluation label Mar 14, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR renames the evaluator from "ISAEvaluator" to "PersonalAttributesEvaluator" and updates its associated metrics, constants, and test references.

  • Renames the evaluator class and updates its documentation and metric identifier.
  • Adjusts constants and configuration to reflect the new personal attributes naming.
  • Updates tests and simulator scenarios to use the new naming convention.

Reviewed Changes

Copilot reviewed 15 out of 15 changed files in this pull request and generated no comments.

Show a summary per file
File Description
sdk/evaluation/azure-ai-evaluation/azure/ai/evaluation/_evaluators/_personal_attributes/init.py Adds an init file for the new PersonalAttributesEvaluator.
sdk/evaluation/azure-ai-evaluation/azure/ai/evaluation/init.py Replaces references to ISAEvaluator with PersonalAttributesEvaluator.
sdk/evaluation/azure-ai-evaluation/azure/ai/evaluation/_evaluators/_personal_attributes/_personal_attributes.py No functional changes; updates documentation and evaluator identifier.
sdk/evaluation/azure-ai-evaluation/azure/ai/evaluation/_evaluators/_common/_base_rai_svc_eval.py Updates task mapping to use PERSONAL_ATTRIBUTES.
sdk/evaluation/azure-ai-evaluation/azure/ai/evaluation/_common/rai_service.py Updates metric handling and constant usage for the new evaluator.
sdk/evaluation/azure-ai-evaluation/azure/ai/evaluation/simulator/_adversarial_scenario.py Renames the adversarial scenario enum with the new evaluator name.
sdk/evaluation/azure-ai-evaluation/_common/constants.py Replaces ISA constant with PERSONAL_ATTRIBUTES constant.
sdk/evaluation/azure-ai-evaluation/tests/e2etests/* Updates tests to reference PersonalAttributesEvaluator and associated metric names.
sdk/evaluation/azure-ai-evaluation/CHANGELOG.md Reflects the new PersonalAttributesEvaluator and updated descriptions.
Comments suppressed due to low confidence (3)

sdk/evaluation/azure-ai-evaluation/azure/ai/evaluation/_common/rai_service.py:298

  • Consider renaming or updating the constant 'INFERENCE_OF_SENSITIVE_ATTRIBUTES' to match the new 'PersonalAttributes' naming for consistency across the codebase.
if metric_name == EvaluationMetrics.PERSONAL_ATTRIBUTES and INFERENCE_OF_SENSITIVE_ATTRIBUTES in batch_response[0]:

sdk/evaluation/azure-ai-evaluation/azure/ai/evaluation/simulator/_adversarial_scenario.py:32

  • [nitpick] The underlying string value 'adv_isa' has not been updated to reflect the new naming convention; consider changing it to 'adv_personal_attributes' for clarity.
ADVERSARIAL_PERSONAL_ATTRIBUTES = "adv_isa"

sdk/evaluation/azure-ai-evaluation/azure/ai/evaluation/_common/constants.py:43

  • [nitpick] The constant value for PERSONAL_ATTRIBUTES still uses the legacy term 'inference sensitive attributes'; consider updating it to better reflect the personal attributes context.
PERSONAL_ATTRIBUTES = "inference sensitive attributes"
@azure-sdk
Copy link
Collaborator

API change check

APIView has identified API level changes in this PR and created following API reviews.

azure-ai-evaluation

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Evaluation Issues related to the client library for Azure AI Evaluation
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants