Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Come with a developer guide on adding new validators in Pythia #7

Open
cloudronin opened this issue Sep 19, 2024 · 1 comment
Open
Assignees
Labels
documentation Improvements or additions to documentation
Milestone

Comments

@cloudronin
Copy link
Member

Currently Pythia ships with a handful of validators, we need to come up with a guide that explain how additional custom validators can be built in Pythia using a descriptive developer guide

@cloudronin cloudronin added the documentation Improvements or additions to documentation label Sep 19, 2024
@cloudronin cloudronin added this to the v0.1 milestone Sep 19, 2024
@pratacosmin
Copy link
Collaborator

The next steps explain how a new validator can be added using the current format. This change require a new image build/deployment.

Add new validator

1. Add the actual implementation under pythia/validators/

Create a file with the validator implementation following the template

### validator implementation template
def validate(text):
    try:
        #some config
        err_msg, is_valid, risk_score = validate_text(text)

        return {
            "isValid": is_valid,
            "errorMessage": err_msg,
            "riskScore": risk_score

        }
    except Exception as e:
        print(e)
        return {
            "isValid": False,
            "errorMessage": str(e),
            "riskScore": 1
        }

2. Create a method with the validator name to be called

This part will allow us to call specific validators using only the name and also allow us to prepare
the input and also how the validator is user (input, output or both base on the config)
Add the method in the ValidatorCall clss in validator_call.py

from pythia.validators import validator_impl as validator_method

class ValidatorCall:
    def detect_pii(self, validator, **kwargs):
        question = get_question(kwargs=kwargs)
        input_data_reference = get_input_data(validator=validator, kwargs=kwargs)
        output_data_response = get_output_data(validator=validator, kwargs=kwargs)
        validator_results = []
        if input_data_reference:
            validator_result = validator_method.validate(text=input_data_reference)
            validator_result["validatedField"] = validator["input"]
            validator_result["validator"] = validator
            validator_results.append(validator_result)
        if output_data_response:
            validator_result = validator_method.validate(text=output_data_response)
            validator_result["validatedField"] = validator["output"]
            validator_result["validator"] = validator
            validator_results.append(validator_result)
        return validator_results

3. Add the validator to the config file

We control what validators are executed using the config.yaml file therefore we have to add the new created
validator the file and enable him. Also we can specify what field to be use for the input or output ( null if no support)

    - name: validator_name
      description: "Custom Validaotr"
      source: custom
      enabled: true
      input: input_reference
      output: input_response

@cloudronin cloudronin modified the milestones: v0.1, v0.2 Sep 27, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
documentation Improvements or additions to documentation
Projects
None yet
Development

No branches or pull requests

3 participants