Skip to content

drewskidang/parse

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

11 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

README for Sean

This guide is designed to help you set up and run the project because you don't dev enough using WSL (Windows Subsystem for Linux). Please follow the steps below carefully. Utlizing -Llama-index -VLLM -Langchain -Pinecone cuz its easier -Chainlit for UI

Prerequisites

Steps to Run

1. Ingestion Service Setup

Thanks to https://github.com/nlmatics/nlm-ingestor for open-sourcing this paraser. Best parser I used.

First, we need to set up the ingestion service using Docker:

docker pull ghcr.io/nlmatics/nlm-ingestor:latest
docker run -p 5010:5001 ghcr.io/nlmatics/nlm-ingestor:latest

This will pull the latest version of the nlm-ingestor and run it, mapping the container's port 5001 to your local port 5010.

2. Running ingest.py

Make a directory for your data

Before running ingest.py, ensure you have created the necessary Pinecone index as required by the script.

Open a new terminal and navigate to the project directory. Run the following command:

python ingest.py

OR Try classifer.py

Super experimental having a model classify chunks maybe facts that could be relevant to the case rule of law etc, it takes alot longer. Ideally train a llm to better classify on what you want to potential improving embeddings.

python classifer.py

3. Model Loading

In a separate terminal, you need to export your AI keys and run the model server. Replace YOUR_OPENAI_API_KEY with your actual OpenAI API key.

export OPENAI_API_KEY="YOUR_OPENAI_API_KEY"
python -m vllm.entrypoints.openai.api_server --model "Drewskidang/AWQ_MERGE" --quantization awq --enforce-eager --chat-template=chatml --gpu-memory-utilization .7

4. Running the Chatbot

Finally, to run the chatbot application, use the following command:

chainlit run app.py --port 8081

This will start the Chainlit application on port 8081.

Conclusion

By following these steps, you should have the project up and running. If you encounter any issues, please review the steps to ensure all commands were executed correctly. Also there's no memory, i dont how to config chainlit with llama-index and memory.

About

No description, website, or topics provided.

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages