Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Enhancement] Generated remote model interfaces should include detailed model outputs #2945

Open
ohltyler opened this issue Sep 13, 2024 · 2 comments
Assignees
Labels
enhancement New feature or request

Comments

@ohltyler
Copy link
Member

The current generated model output interfaces just include the boilerplate. The actual useful model outputs will be under dataAsMap, but this is always defined as a generic object with no sub-properties.

"inference_results": {
    "type": "array",
    "items": {
        "type": "object",
        "properties": {
            "output": {
                "type": "array",
                "items": {
                    "properties": {
                        "name": {
                            "type": "string",
                            "description": "This is a test description field"
                        },
                        "dataAsMap": {
                            "type": "object",
                            "description": "This is a test description field"
                        }
                    }
                },
                "description": "This is a test description field"
            },
            "status_code": {
                "type": "integer",
                "description": "This is a test description field"
            }
        }
    },
    "description": "This is a test description field"
}

The dataAsMap field should be further defined to include the actual model outputs

@ohltyler ohltyler added the enhancement New feature or request label Sep 13, 2024
@b4sjoo
Copy link
Collaborator

b4sjoo commented Sep 16, 2024

@ohltyler Can you check https://github.com/opensearch-project/ml-commons/pull/2689/files to see if you are satisfied with the current granularity for our output interface

@b4sjoo b4sjoo removed the untriaged label Sep 16, 2024
@ohltyler
Copy link
Member Author

Got it, thanks for the reference! I've tested with some other blueprints, and it seems to work as expected. I think the confusion I was having was around how much flexibility can be made to the connectors, before it stops producing the detailed interfaces. The documentation states:

The predefined model interface is generated based on the connector blueprint and the model's metadata, so you must strictly follow the blueprint when creating the connector in order to avoid errors.

For example, I had some blueprints that had a supported model defined in-line, but not included as one of the parameters. Or some models that weren't part of the list I'm seeing defined here.

From that code, it looks like a blueprint will be generated automatically if & only if:

  1. There is a service_name and model defined under parameters
  2. The values of service_name / model are included in the switch statement
  3. All other parts of the blueprint (the protocol/credentials/actions/request body etc.) can be anything

If so, can we update the documentation to reflect this a bit more, on what flexibility and what restrictions are put in place? It would be helpful to know how users can still have custom request bodies, for example, but still leverage automated model interfaces.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
Status: In Progress
Development

No branches or pull requests

2 participants