🚀 Feature: Installation Optimization

[pabik]

🔖 Feature description

The current installation process for DocsGPT is taking too long, and we need to optimize it. The goal is to streamline dependency management, improve Dockerfile performance, and explore lazy loading techniques for specific components like vector DBs, LLMs, parsers, and retrievers.

Additionally, we want to investigate transitioning away from LangChain in the future to reduce dependency load and improve flexibility.

🎤 Why is this feature needed ?

Long installation times and heavy dependencies impact the user experience, especially for contributors and developers working with DocsGPT. Optimizing the setup will lead to faster installation, fewer dependency issues, and better resource management.

✌️ How do you aim to achieve this?

1. Thorough Dependency Checks:

   • Review and audit all Python dependencies to identify unnecessary or outdated packages.
   • Perform a similar review for NPM dependencies to ensure no excessive or redundant libraries are being used.

2. Dockerfile Optimization:

   • Check the current Dockerfile setup for inefficiencies, reduce build layers.

3. Explore UV (https://github.com/astral-sh/uv):

   • Investigate the possibility of implementing UV. Test the difference in performance and installation times.

4. Lazy Loading:

   • Implement lazy loading for components that are not immediately needed during the initial setup, such as vector databases, LLMs, parsers, and retrievers, reducing the initial installation footprint.

5. Move Away from LangChain:

   • This will improve DocsGPT’s flexibility and reduce unnecessary dependencies in the future.

This task involves multiple areas of optimization, so collaboration is highly encouraged. Feel free to contribute directly to this issue or join the discussion in our Discord (collaborative-issues).

If necessary, this issue can be split into smaller, focused tasks. Please ensure all related issues are linked back to this original for tracking purposes.

🔄️ Additional Information

No response

👀 Have you spent some time to check if this feature request has been raised before?

• I checked and didn't find similar issue

Are you willing to submit PR?

None

[sanjay7178]

@pabik I have experience with uv and ruff , upon maintain pyproject.yml and other config files with it, place assign me for this issue

[Charlesnorris509]

@pabik I have some experience with these type of Fullstack Application with Node.js and Flask, It seems like there are some redundant dependencies and your requirements.txt and Package-lock.json Files. Please assign me the issue and I'll make a pull request