Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cannot connect to aws opensearch serverless #1159

Open
1 task
adhikari23 opened this issue Jul 8, 2024 · 2 comments
Open
1 task

Cannot connect to aws opensearch serverless #1159

adhikari23 opened this issue Jul 8, 2024 · 2 comments
Labels

Comments

@adhikari23
Copy link

adhikari23 commented Jul 8, 2024

Describe the bug
Cannot connect to aws opensearch serverles. Here is the code snippet.

from haystack_integrations.document_stores.opensearch import OpenSearchDocumentStore
from haystack_integrations.components.embedders.amazon_bedrock import AmazonBedrockTextEmbedder
from requests_aws4auth import AWS4Auth
from opensearchpy import OpenSearch, RequestsHttpConnection
from boto3 import Session
from haystack import Pipeline

service = 'aoss'
credentials = Session().get_credentials()
awsauth = AWS4Auth(credentials.access_key, credentials.secret_key,
                   "us-east-1", service, session_token=credentials.token)

embedder = AmazonBedrockTextEmbedder(model="amazon.titan-embed-text-v1")
docstore = OpenSearchDocumentStore(
    hosts = <opensearch serverless endpoint>,
    index = "jps-test-index",
    http_auth= awsauth,
    timeout=300,
    use_ssl=True,
    verify_certs=True,
    connection_class=RequestsHttpConnection,
    engine = "faiss"
    # return_embedding=True

)
query_pipeline = Pipeline()
query_pipeline.add_component("text_embedder", embedder)
query_pipeline.add_component("retriever", OpenSearchEmbeddingRetriever(document_store=docstore))
query_pipeline.connect("text_embedder.embedding", "retriever.query_embedding")

query = "How many languages are there?"

result = query_pipeline.run({"text_embedder": {"text": query}})

print(result['retriever']['documents'][0])

Error message

  File "/root/bosai/genai/eric-bosaiapps-genai-poc/haystack-demo/venv/lib/python3.10/site-packages/opensearchpy/transport.py", line 416, in perform_request
    status, headers_response, data = connection.perform_request(
  File "/root/bosai/genai/eric-bosaiapps-genai-poc/haystack-demo/venv/lib/python3.10/site-packages/opensearchpy/connection/http_requests.py", line 241, in perform_request
    self._raise_error(
  File "/root/bosai/genai/eric-bosaiapps-genai-poc/haystack-demo/venv/lib/python3.10/site-packages/opensearchpy/connection/base.py", line 315, in _raise_error
    raise HTTP_EXCEPTIONS.get(status_code, TransportError)(
opensearchpy.exceptions.NotFoundError: NotFoundError(404, '')

Expected behavior
Retrieved documents

Additional context
Add any other context about the problem here, like document types / preprocessing steps / settings of reader etc.

To Reproduce
Steps to reproduce the behavior

FAQ Check

System:

  • OS: linux
  • GPU/CPU: cpu
  • Haystack version (commit or version number):
    amazon-bedrock-haystack 0.9.2
    haystack-ai 2.2.4
    opensearch-haystack 0.7.1
  • DocumentStore:
  • Reader:
  • Retriever:
@adhikari23 adhikari23 changed the title Cannot connect to aws opensearch serverles Cannot connect to aws opensearch serverless Jul 8, 2024
@davidsbatista
Copy link
Contributor

davidsbatista commented Jul 10, 2024

Hi @adhikari23 , I formatted your code posting to allow for a better reading.

First thing is that it seems that you are missing an import:

from haystack_integrations.components.retrievers.opensearch import OpenSearchEmbeddingRetrieve

Looking at your error message it seems that there's a problem connection to your OpenSearch server. Can you connect to it in isolation, i.e.: outside of haystack?

@adhikari23
Copy link
Author

Yes, I am able to connect to the opensearch server outside haystack.

@anakin87 anakin87 transferred this issue from deepset-ai/haystack Oct 28, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

3 participants