Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] Unintended arguments are being forwarded to boto3.client #1646

Open
drooms-sandrus opened this issue Nov 13, 2024 · 0 comments
Open

[BUG] Unintended arguments are being forwarded to boto3.client #1646

drooms-sandrus opened this issue Nov 13, 2024 · 0 comments
Assignees
Labels
bug Something isn't working

Comments

@drooms-sandrus
Copy link

Bug Description
It's not possible to set **kwargs intended for trulens.core.feedback.endpoint.Endpoint via trulens.providers.bedrock.provider.Bedrock constructor, if no client argument is passed.

The **kwargs passed into Bedrock are being forwarded to BedrockEndpoint, and from there to its super class Endpoint, as intended. The issue is that client_kwargs (set inside BedrockEndpoint constructor) are a super set of kwargs:

client_kwargs = {k: v for k, v in kwargs.items()}

Further down below, a check is performed, whether the client argument is contained in kwargs:

if "client" in kwargs:
    ...

If not, a boto3 client is instantiated using client_kwargs:

self.client = boto3.client(
    service_name="bedrock-runtime", **client_kwargs
)

Since boto3.client expects different arguments than Endpoint, it may raise an error depending on which arguments were passed.

I noticed this issue, when I intended to instantiate Bedrock from a configuration that is stored in a file. Since all involved classes extend from pydantic BaseModel, following should be possible once the mentioned issue is fixed:

# config can be stored in external configuration, e.g. in a .yaml file
config = {
    "model_id": "anthropic.claude-3-5-sonnet-20240620-v1:0",
    "region_name": "eu-central-1",
    "pace": {
        "marks_per_second": 0.5,
        "seconds_per_period": 60,
    },
}
bedrock = Bedrock.model_validate(config)

To Reproduce

from trulens.core.utils.pace import Pace
from trulens.providers.bedrock import Bedrock

bedrock = Bedrock(
    model_id="anthropic.claude-3-5-sonnet-20240620-v1:0",
    region_name="eu-central-1",
    pace=Pace(marks_per_second=0.5, seconds_per_period=60),
)

Expected behavior
I expect the code above to work once pace is being forwarded only to Endpoint (and not to boto3.client as is happening now), and recommend to solve it similarly to how it was done in Huggingface pipelines (see here).

Relevant Logs/Tracebacks

Traceback (most recent call last):
  File "/home/user/projects/project/test.py", line 4, in <module>
    bedrock = Bedrock(
              ^^^^^^^^
  File "/home/user/miniconda3/envs/my-env/lib/python3.12/site-packages/trulens/providers/bedrock/provider.py", line 50, in __init__
    self_kwargs["endpoint"] = bedrock_endpoint.BedrockEndpoint(
                              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/user/miniconda3/envs/my-env/lib/python3.12/site-packages/trulens/providers/bedrock/endpoint.py", line 197, in __init__
    self.client = boto3.client(
                  ^^^^^^^^^^^^^
  File "/home/user/miniconda3/envs/my-env/lib/python3.12/site-packages/boto3/__init__.py", line 92, in client
    return _get_default_session().client(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: Session.client() got an unexpected keyword argument 'pace'

Environment:

  • OS: Ubuntu 22.04
  • Python Version: 3.12.5
  • TruLens version: 1.2.6
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

3 participants