Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Langchain doesn't work on AWS Lambda with either faiss or hnswlib #1930

Closed
dmb0058 opened this issue Jul 11, 2023 · 27 comments
Closed

Langchain doesn't work on AWS Lambda with either faiss or hnswlib #1930

dmb0058 opened this issue Jul 11, 2023 · 27 comments
Labels
auto:bug Related to a bug, vulnerability, unexpected error with an existing feature

Comments

@dmb0058
Copy link

dmb0058 commented Jul 11, 2023

The current version is broken on AWS lambda, both with Faiss and HNSWlib. I've spent the last few days building test cases on both platforms, but haven't dived into the langchain code.

The typical outcome is:

"Could not import faiss-node. Please install faiss-node as a dependency with, e.g. npm install -S faiss-node.\n\nError: libgomp.so.1: cannot open shared object file: No such file or directory"

Same with HNSWlib (substitute hnswlib for faiss in the error above).

Also see

#943
#1764

Hope this helps - langchain is amazing, and if it worked on AWS it would be even better :)

David
@dosubot dosubot bot added the auto:bug Related to a bug, vulnerability, unexpected error with an existing feature label Jul 11, 2023
@dosubot
Copy link

dosubot bot commented Jul 11, 2023

Answer generated by a 🤖

Answer

I understand that you're experiencing issues with LangChain on AWS Lambda, specifically when using either the faiss-node or hnswlib-node libraries. The error message you're encountering suggests that the AWS Lambda environment is missing a required shared library, libgomp.so.1, which is part of the GCC libraries.

Here's a potential solution that involves creating a Lambda Layer with the missing shared libraries and adding it to your AWS Lambda function:

  1. On your local machine, create a directory, for instance, lambda-layer.
  2. Within this directory, create another one named lib.
  3. You'll need to install the necessary libraries in this directory. This can be done by running a Docker container that uses the Amazon Linux image, installing the libraries there, and then copying them to your local machine. Here's how you can do it:
docker run -v $(pwd):/lambda-layer -it amazonlinux:2
yum install -y gcc
cp /usr/lib64/libgomp.so.1 /lambda-layer/lib/
exit
  1. Next, create a zip file of the lib directory:
cd lambda-layer
zip -r lib.zip lib
  1. Upload this zip file as a new Lambda Layer in the AWS Lambda console.
  2. Finally, add this layer to your Lambda function.

This approach should provide your function with the necessary shared libraries to run faiss-node and hnswlib-node.

Please note that this is one potential solution and there might be other ways to address this issue. If you find this solution to be correct and believe that it's a bug that could impact other users, we encourage you to make a pull request.

If this solution doesn't align with your experience, it might be worth checking if you're running the latest version of LangChain, as updates often include bug fixes and improvements.

Sources

This response is meant to be useful, save you time, and share context. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

@dmb0058
Copy link
Author

dmb0058 commented Jul 11, 2023 via email

@dmb0058
Copy link
Author

dmb0058 commented Jul 11, 2023 via email •
edited
Loading

@jacoblee93
Copy link
Collaborator

Pinging @ewfian - is this something fixable in faiss-node?

@ewfian
Copy link
Contributor

ewfian commented Jul 12, 2023

@jacoblee93 There is also a related issue on faiss-node side. I'll do some research and try to find a solution.

@dmb0058
Copy link
Author

dmb0058 commented Jul 12, 2023

Happy to test any new versions if it helps :)

@lika85456
Copy link

There is a document on how to run hnswlib-node on AWS lambda. I managed to deploy it using this dockerfile:

FROM public.ecr.aws/lambda/nodejs:18 as builder
WORKDIR /usr/app

# install python
RUN yum install -y python3 make gcc-c++ && \
    yum clean all && \
    rm -rf /var/cache/yum

# install pnpm
RUN npm i -g pnpm

# install dependencies
COPY package.json pnpm-lock.yaml ./
RUN pnpm install

# build
COPY index.ts ./
RUN pnpm run build


FROM public.ecr.aws/lambda/nodejs:18
WORKDIR ${LAMBDA_TASK_ROOT}

COPY foo.dat ./
COPY --from=builder /usr/app ./
COPY --from=builder /usr/app/dist/* ./

CMD ["index.handler"]

Looking for a lambda layer solution, since I don't want to deploy using Docker images.

@dmb0058 dmb0058 mentioned this issue Jul 13, 2023
Closed
@dmb0058
Copy link
Author

dmb0058 commented Jul 13, 2023

Amazon linux 2 image seems to only include /lib64/libm.so.6: version `GLIBC_2.26'

Any idea where I might find 2.27 just to see if adding it to my missing libraries layer would work? If not I'll spin up an EC2 server and build one.

@dmb0058 dmb0058 mentioned this issue Jul 14, 2023
Open
@dmb0058
Copy link
Author

dmb0058 commented Jul 14, 2023 •
edited
Loading

...
{ "errorType": "Error", "errorMessage": "Could not import faiss-node. Please install faiss-node as a dependency with, e.g. npm install -S faiss-node.\n\nError: /lib64/libm.so.6: version GLIBC_2.27' not found (required by /opt/nodejs/node_modules/faiss-node/build/Release/faiss-node.node)", "trace": [ "Error: Could not import faiss-node. Please install faiss-node as a dependency with, e.g. npm install -S faiss-node.", "", "Error: /lib64/libm.so.6: version GLIBC_2.27' not found (required by /opt/nodejs/node_modules/faiss-node/build/Release/faiss-node.node)", " at FaissStore.importFaiss (/opt/nodejs/node_modules/langchain/dist/vectorstores/faiss.cjs:236:19)", " at async readIndex (/opt/nodejs/node_modules/langchain/dist/vectorstores/faiss.cjs:141:37)", " at async Promise.all (index 1)", " at async FaissStore.load (/opt/nodejs/node_modules/langchain/dist/vectorstores/faiss.cjs:144:51)", " at async query (/var/task/index.js:112:20)", " at async exports.handler (/var/task/index.js:20:9)" ] }
…

I think the problem here might be something to do with the fact that AWS Lambda runs a distro called "Amazon Linux 2", which has glibc 2.26. So in the error above it seems to be saying that faiss-node requires glibc 2.27. If that's the case it might explain why there are unmet dependencies. Unfortunately the only options would appear to be to wait for AWS to update the version of glibc or to get a 'downgraded' version of faiss-node for Lambda.

Or I might be barking at the moon :)

@campoblanco
Copy link

I'm facing the same issue, and haven't been able to solve it. I would really like to be able to use FAISS and LangChain in a lambda

@awadhwanan
Copy link

There is a document on how to run hnswlib-node on AWS lambda. I managed to deploy it using this dockerfile:

FROM public.ecr.aws/lambda/nodejs:18 as builder
WORKDIR /usr/app

# install python
RUN yum install -y python3 make gcc-c++ && \
    yum clean all && \
    rm -rf /var/cache/yum

# install pnpm
RUN npm i -g pnpm

# install dependencies
COPY package.json pnpm-lock.yaml ./
RUN pnpm install

# build
COPY index.ts ./
RUN pnpm run build


FROM public.ecr.aws/lambda/nodejs:18
WORKDIR ${LAMBDA_TASK_ROOT}

COPY foo.dat ./
COPY --from=builder /usr/app ./
COPY --from=builder /usr/app/dist/* ./

CMD ["index.handler"]

Looking for a lambda layer solution, since I don't want to deploy using Docker images.

@lika85456 This solution fails if you try to do anything with vectorStore
https://js.langchain.com/docs/modules/chains/popular/vector_db_qa

@ewfian
Copy link
Contributor

ewfian commented Jul 21, 2023

There is some update on faiss-node side, you can try it out, contact me if any questions
ewfian/faiss-node#25 (comment)

@dmb0058
Copy link
Author

dmb0058 commented Jul 21, 2023 via email

@ewfian
Copy link
Contributor

ewfian commented Jul 22, 2023

@dmb0058 I have made a demo for langchain. Please confirm whether it works properly by uploading this zip directly to AWS.
langchain-faiss-aws-demo.zip
It should correctly output the total count in the index, which is 1.
Snipaste_2023-07-22_14-34-47

If you want your project to work, please remember that the files(node_modules/faiss-node/build/Release) in the zip to upload to AWS should replace with the files I provided here: ewfian/faiss-node#25 (comment)

@ewfian ewfian mentioned this issue Jul 22, 2023
Closed
@iAmCodeHead
Copy link

facing similar issue.

@ewfian
Copy link
Contributor

ewfian commented Jul 23, 2023

Here is a new demo
Function: langchain-aws-demo.zip
Layer: faiss-node-v0.2.2-aws-layer-x64.zip

@2441630833
Copy link

2441630833 commented Aug 14, 2023 •
edited
Loading

The current version is broken on AWS lambda, both with Faiss and HNSWlib. I've spent the last few days building test cases on both platforms, but haven't dived into the langchain code.

The typical outcome is:

"Could not import faiss-node. Please install faiss-node as a dependency with, e.g. npm install -S faiss-node.\n\nError: libgomp.so.1: cannot open shared object file: No such file or directory"

Same with HNSWlib (substitute hnswlib for faiss in the error above).

Also see

#943 #1764

Hope this helps - langchain is amazing, and if it worked on AWS it would be even better :)

David

Hi, I encountered this issue too. I suspect the issue arises from langchain depends on the third-party modules faiss-node or hnswlib, I solved this issue by:

  1. Remove the faiss-node dependency from your package.json file.
  2. Modify your Dockerfile to install the faiss-node dependency directly.

If you're using HNSWlib, simply replace faiss with hnswlib in your Dockerfile.

By handling the installation independently in the Dockerfile instead of through package.json, you might avoid the issue. I hope this helps!
图片

@dosubot DosuBot
Copy link

dosubot bot commented Nov 13, 2023

Hi, @dmb0058! I'm Dosu, and I'm helping the langchainjs team manage their backlog. I wanted to let you know that we are marking this issue as stale.

Based on my understanding, the current issue is related to importing faiss-node and HNSWlib on AWS Lambda, resulting in an error message about a missing shared object file. There have been ongoing discussions and potential workarounds in the comments, including creating a Lambda Layer with the missing shared libraries and adding it to the AWS Lambda function. Docker solutions and updates on the faiss-node side have also been mentioned.

Before we close this issue, we wanted to check with you if it is still relevant to the latest version of the langchainjs repository. If it is, please let us know by commenting on the issue. Otherwise, feel free to close the issue yourself or it will be automatically closed in 7 days.

Thank you for your understanding and contribution to the langchainjs community!

@dosubot dosubot bot added the stale Issue has not had recent activity or appears to be solved. Stale issues will be automatically closed label Nov 13, 2023
@dmb0058
Copy link
Author

dmb0058 commented Nov 13, 2023 via email

@dosubot dosubot bot removed the stale Issue has not had recent activity or appears to be solved. Stale issues will be automatically closed label Nov 13, 2023
@Julien-cpsn
Copy link

I am facing the same issue inside of Vue 3 + Vite

@ashishbamania
Copy link

I have been using Next.js 13 and was getting the same issue ("Could not import faiss-node. Please install faiss-node as a dependency with, e.g. npm install -S faiss-node.") with the following code:

  const embeddings = new OpenAIEmbeddings({
    openAIApiKey: process.env.OPENAI_API_KEY,
  });

  const vectorStore = await FaissStore.fromDocuments(
    docs,
    embeddings,
  );

I fixed it with making these changes to next.config.js file:

/** @type {import('next').NextConfig} */
const nextConfig = {
  webpack: (config, { buildId, dev, isServer, defaultLoaders, webpack }) => {
    config.resolve.alias.canvas = false;
    config.resolve.alias.encoding = false;
    config.externals = [...config.externals, "faiss-node"]; //added this line
    return config;
  },
};

module.exports = nextConfig;

@jacoblee93
Copy link
Collaborator

Are you using the edge runtime? faiss-node is Node only :(

@PierrickLozach PierrickLozach mentioned this issue Jan 7, 2024
Closed
@bitsnaps
Copy link

bitsnaps commented Feb 4, 2024

I'm getting the same issue with faiss-node@^0.5.1 (also tried ^0.2.1 which was working with an old example).

...
Error: Could not import faiss-node. Please install faiss-node as a dependency with, e.g. `npm install -S faiss-node`.
...

Running locally with Node v18.19.0 LTS on macOS. I used HuggingFaceInferenceEmbeddings instead of openai's embeddings.

@dosubot DosuBot
Copy link

dosubot bot commented Feb 4, 2024

🤖

Based on the error messages you've shared, it seems like the AWS Lambda environment is missing some shared libraries required by faiss-node and hnswlib. Specifically, the errors mention libgomp.so.1 and /lib64/libm.so.6: version GLIBC_2.27.

One possible workaround is to create a custom AWS Lambda layer that includes these missing shared libraries. Here's a general outline of the steps:

  1. Create a Dockerfile that uses the Amazon Linux image (which is the same environment that AWS Lambda runs on). In this Dockerfile, install the necessary dependencies and build the shared object files.

  2. Use the Dockerfile to build a Docker image and run a Docker container.

  3. Inside the Docker container, create a zip file that contains the shared object files and dependencies.

  4. Upload the zip file to AWS S3.

  5. Create a new AWS Lambda layer and point it to the zip file in S3.

  6. Add the new layer to your AWS Lambda function.

Please note that the exact steps and code will depend on the specific shared object files and dependencies that are missing.

In the context of LangChain, you would need to ensure that the AWS Lambda environment has the necessary dependencies to run Faiss or HNSWlib, and that the shared object files are included in the AWS Lambda layer.

Unfortunately, without more specific information about the shared object files and dependencies required by Faiss and HNSWlib, I can't provide more specific steps or code. You might need to consult the documentation or community for these libraries to get this information.

As for the issue with GLIBC_2.27, it seems like AWS Lambda's Amazon Linux 2 environment only includes GLIBC_2.26. This might be causing the unmet dependencies issue. One possible solution is to wait for AWS to update the version of glibc in their environment, or to find a version of faiss-node that is compatible with GLIBC_2.26.

I hope this helps! If you have any more questions or need further clarification, feel free to ask.

Sources

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.

@apotox
Copy link

apotox commented Feb 18, 2024

here is an example to how to use / install faiss node on aws lambda https://github.com/apotox/faiss-node-aws-lambda

@dosubot dosubot bot added the stale Issue has not had recent activity or appears to be solved. Stale issues will be automatically closed label May 19, 2024
@dosubot DosuBot
Copy link

dosubot bot commented May 19, 2024

Hi, @dmb0058,

I'm helping the langchainjs team manage their backlog and am marking this issue as stale. The issue #1930 highlighted the problem of Langchain being broken on AWS Lambda with both Faiss and HNSWlib, resulting in an error related to missing shared object files. There have been potential workarounds discussed in the comments, including creating a Lambda Layer with the missing shared libraries, Docker solutions, and updates on the faiss-node side. Users have also shared their experiences and attempts to address the issue, with some providing Dockerfile examples and AWS Lambda layer solutions.

Could you please let us know if this issue is still relevant to the latest version of the repository? If it is, please comment on the issue to let the langchainjs team know. Otherwise, feel free to close the issue yourself, or it will be automatically closed in 7 days. Thank you!

@dmb0058
Copy link
Author

dmb0058 commented May 19, 2024 via email

@dosubot dosubot bot removed the stale Issue has not had recent activity or appears to be solved. Stale issues will be automatically closed label May 19, 2024
@dosubot dosubot bot added the stale Issue has not had recent activity or appears to be solved. Stale issues will be automatically closed label Aug 18, 2024
@dosubot dosubot bot closed this as not planned Won't fix, can't repro, duplicate, stale Aug 25, 2024
@dosubot dosubot bot removed the stale Issue has not had recent activity or appears to be solved. Stale issues will be automatically closed label Aug 25, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
auto:bug Related to a bug, vulnerability, unexpected error with an existing feature
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests