Accept passthrough headers in agent execute

[jiapingzeng]

Description

Create MCP client on each execute request + forward headers to MCP connector

Related Issues

Resolves #[Issue number to be closed when this PR is merged]

Check List

• New functionality includes testing.
• New functionality has been documented.
• API changes companion pull request created.
• Commits are signed per the DCO using --signoff.
• Public documentation issue/PR created.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Accepted headers:

- opensearch-url
- aws-region
- aws-access-key-id
- aws-secret-access-key
- aws-session-token
- aws-service-name

Sample request:

POST {{endpoint}}/_plugins/_ml/agents/{{agent_id}}/_execute
--header 'aws-access-key-id: xxx' \
--header 'aws-secret-access-key: xxx' \
--header 'aws-session-token: xxx' \
--header 'opensearch-url: https://xxx.aos.us-east-1.on.aws' \
--header 'aws-region: us-east-1' \
--header 'aws-service-name: es' \

--header 'Content-Type: application/json'
{
    "parameters": {
        "question": "list indices"
    },
    "verbose": "true"
}

Response:

{
    "inference_results": [
        {
            "output": [
                {
                    "name": "memory_id",
                    "result": "F7vALZoBCqjo4z06S5j0"
                },
                {
                    "name": "parent_interaction_id",
                    "result": "GbvALZoBCqjo4z06TZiW"
                },
                {
                    "name": "response",
                    "dataAsMap": {
                        "response": "Here are the indices in your OpenSearch cluster:\n\n## User Indices\n1. **tmdb** - 8,516 documents, green health\n...",
                        "additional_info": {}
                    }
                }
            ]
        }
    ]
}

[Zhangxunmt]

spotless needs to apply

[Hailong-am]

can we change it to x-amzn-datasource-endpoint and add a new header x-amzn-datasource-type

[jiapingzeng]

discussed with team today, we will accept these headers in ml-commons to match opensearch-mcp-server-py

- opensearch-url
- aws-region
- aws-access-key-id
- aws-secret-access-key
- aws-session-token
- aws-service-name

[saratvemulapalli]

+1 we should generic headers which opensearch mcp server accepts.

[codecov]

Codecov Report

❌ Patch coverage is 68.91892% with 23 lines in your changes missing coverage. Please review.
✅ Project coverage is 80.15%. Comparing base (c243f8a) to head (7ad1a16).

Files with missing lines	Patch %	Lines
.../java/org/opensearch/ml/utils/RestActionUtils.java	70.00%	3 Missing and 6 partials ⚠️
...nsearch/ml/engine/algorithms/agent/AgentUtils.java	83.33%	4 Missing and 1 partial ⚠️
...engine/algorithms/remote/McpConnectorExecutor.java	0.00%	2 Missing and 2 partials ⚠️
...hms/remote/McpStreamableHttpConnectorExecutor.java	0.00%	2 Missing and 2 partials ⚠️
.../opensearch/ml/rest/RestMLExecuteStreamAction.java	50.00%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff            @@
##               main    #4364   +/-   ##
=========================================
  Coverage     80.15%   80.15%           
- Complexity    10227    10245   +18     
=========================================
  Files           858      858           
  Lines         44496    44547   +51     
  Branches       5145     5158   +13     
=========================================
+ Hits          35664    35708   +44     
- Misses         6668     6669    +1     
- Partials       2164     2170    +6     

Flag	Coverage Δ
ml-commons	80.15% <68.91%> (+<0.01%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.