Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug] fix requirements.txt for example chat-w/-pdf #284

Merged
merged 2 commits into from
Oct 2, 2024
Merged
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
143 changes: 137 additions & 6 deletions examples/panel/chat-with-pdf/requirements.txt
Original file line number Diff line number Diff line change
@@ -1,6 +1,137 @@
panel
openai
chromadb
pypdf
langchain
tiktoken
aiohappyeyeballs==2.4.3
aiohttp==3.9.3
aiosignal==1.3.1
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All the dependencies can go in a requirements.lock.txt file. In this file, you can add the specific package that was needed. You can see the format in the other examples

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

requirements.lock.txt this file isn't use by our deployment pipeline. So thus why the missing dependencies.

This is the package installation of a Panel Dockerfile from our platform:

WORKDIR /srv
# Caching Introduced here
COPY requirements.txt /srv/
RUN pip install -r requirements.txt --no-cache-dir

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes the lock file is for debugging purposes. We can just add the specific package required to resolve the issue langchain_community.document_loaders in this file. I'm not sure why all the installed packages need to be mentioned in the requirements.txt file

Copy link
Contributor

@neelasha23 neelasha23 Oct 2, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is mentioned in the error: You can install it using pip install -U langchain-community
So just add langchain-community in this file

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Isn't the command pip freeze > requirements.txt better? It captures all packages necessary for the app with their versions. This way, future users can build it with the same versions and be sure the app will work.

  • Langchain are often breaking api so you need the exact version
  • All those requirements are from installing the
panel
openai
chromadb
pypdf
langchain
tiktoken
langchain-community

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We generally freeze all requirements in the lock file and not the requirements.txt. But maybe @edublancas can suggest?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

there are pros and cons for each approach. in the past, we've created a .lock.txt file and a .txt file. the problem is that the platform only looks at the .txt file.

let's pin dependencies in a .txt file so the platform uses that, but let's only list requirements manually (instead of running pip freeze) because listing everything has the potential to break examples (as there might be packages that are incompatible with Linux or that aren't even needed)

annotated-types==0.6.0
anyio==4.3.0
asgiref==3.7.2
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

let's pin versions but only for the explicitly imported packages

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fix: 3a3a901

async-timeout==4.0.3
attrs==23.2.0
backoff==2.2.1
bcrypt==4.1.2
bleach==6.1.0
bokeh==3.3.4
build==1.1.1
cachetools==5.3.3
certifi==2024.2.2
charset-normalizer==3.3.2
chroma-hnswlib==0.7.3
chromadb==0.4.24
click==8.1.7
coloredlogs==15.0.1
contourpy==1.2.0
dataclasses-json==0.6.4
Deprecated==1.2.14
distro==1.9.0
durationpy==0.8
exceptiongroup==1.2.0
fastapi==0.110.0
filelock==3.13.1
flatbuffers==23.5.26
frozenlist==1.4.1
fsspec==2024.2.0
google-auth==2.28.1
googleapis-common-protos==1.62.0
greenlet==3.1.1
grpcio==1.62.0
h11==0.14.0
httpcore==1.0.4
httptools==0.6.1
httpx==0.27.0
huggingface-hub==0.21.3
humanfriendly==10.0
idna==3.6
importlib-metadata==6.11.0
importlib_resources==6.1.2
Jinja2==3.1.3
jiter==0.5.0
jsonpatch==1.33
jsonpointer==2.4
kubernetes==29.0.0
langchain==0.1.10
langchain-community==0.0.25
langchain-core==0.1.28
langchain-text-splitters==0.0.1
langsmith==0.1.14
linkify-it-py==2.0.3
Markdown==3.5.2
markdown-it-py==3.0.0
MarkupSafe==2.1.5
marshmallow==3.21.0
mdit-py-plugins==0.4.0
mdurl==0.1.2
mmh3==4.1.0
monotonic==1.6
mpmath==1.3.0
multidict==6.0.5
mypy-extensions==1.0.0
numpy==1.26.4
oauthlib==3.2.2
onnxruntime==1.17.1
openai==1.13.3
opentelemetry-api==1.23.0
opentelemetry-exporter-otlp-proto-common==1.23.0
opentelemetry-exporter-otlp-proto-grpc==1.23.0
opentelemetry-instrumentation==0.44b0
opentelemetry-instrumentation-asgi==0.44b0
opentelemetry-instrumentation-fastapi==0.44b0
opentelemetry-proto==1.23.0
opentelemetry-sdk==1.23.0
opentelemetry-semantic-conventions==0.44b0
opentelemetry-util-http==0.44b0
orjson==3.9.15
overrides==7.7.0
packaging==23.2
pandas==2.2.1
panel==1.3.8
param==2.0.2
pillow==10.2.0
posthog==3.4.2
protobuf==4.25.3
pulsar-client==3.4.0
pyasn1==0.5.1
pyasn1-modules==0.3.0
pydantic==2.6.3
pydantic_core==2.16.3
Pygments==2.18.0
pypdf==4.1.0
PyPika==0.48.9
pyproject_hooks==1.0.0
python-dateutil==2.9.0.post0
python-dotenv==1.0.1
pytz==2024.1
pyviz_comms==3.0.1
PyYAML==6.0.1
regex==2023.12.25
requests==2.31.0
requests-oauthlib==1.3.1
rich==13.9.1
rsa==4.9
setuptools==75.1.0
shellingham==1.5.4
six==1.16.0
sniffio==1.3.1
SQLAlchemy==2.0.27
starlette==0.36.3
sympy==1.12
tenacity==8.2.3
tiktoken==0.6.0
tokenizers==0.15.2
tomli==2.0.1
tornado==6.4
tqdm==4.66.2
typer==0.9.0
typing-inspect==0.9.0
typing_extensions==4.10.0
tzdata==2024.1
uc-micro-py==1.0.3
urllib3==2.2.1
uvicorn==0.27.1
uvloop==0.19.0
watchfiles==0.21.0
webencodings==0.5.1
websocket-client==1.7.0
websockets==12.0
wrapt==1.16.0
xyzservices==2023.10.1
yarl==1.9.4
zipp==3.17.0
Loading