docs: Streamline all existing docs (#792)

* Update all docs

* remove

* new file

* add tool auth docs

* lint

* remove langchain docs

* remove calc

.env-template

@@ -51,7 +51,6 @@ BEDROCK_REGION_NAME=<REGION NAME>
 
 # Experimental features
 
-USE_EXPERIMENTAL_LANGCHAIN=False
 USE_AGENTS_VIEW=False
 
 # Community features

CONTRIBUTING.md

@@ -1,32 +1,41 @@
 # OSS Toolkit Contributing Guide
 
-Thank you for your interest in contributing to Cohere's OSS toolkit. This guide will help explain the contribution workflow from opening an issue, creating a PR, to reviewing and merging your added changes. For development setup, please refer to the [README](README.md).
+Thank you for your interest in contributing to Cohere's OSS toolkit. This guide covers the contribution workflow, from opening an issue to reviewing and merging changes. For development setup, please refer to the [README](README.md).
 
-# Getting Started
+## Getting Started
 
-Remember that there are many ways to contribute other than writing code: writing tutorials or blog posts, improving [the documentation](https://docs.cohere.com), and submitting bug reports.
+There are many ways to contribute beyond writing code. You can help by:
+- Writing tutorials or blog posts
+- Improving [the documentation](https://docs.cohere.com)
+- Submitting bug reports
 
 ## Table of Contents
 
 - [Assumptions](#assumptions)
 - [How to Contribute](#how-to-contribute)
+- [How to Add Features](#how-to-add-features)
 
 ## Assumptions
 
-1. **You're familiar with [GitHub](https://github.com) and the [Pull Requests (PR)](https://help.github.com/en/github/collaborating-with-issues-and-pull-requests/about-pull-requests) workflow.**
-2. **You've read the Cohere's [documentation](https://docs.cohere.com).**
-3. **You know about the [Cohere community on Discord](https://discord.com/invite/co-mmunity).**
+Before contributing, please ensure:
+1. You're familiar with [GitHub](https://github.com) and the [Pull Request (PR)](https://help.github.com/en/github/collaborating-with-issues-and-pull-requests/about-pull-requests) workflow.
+2. You've read [Cohere's documentation](https://docs.cohere.com).
+3. You know about the [Cohere community on Discord](https://discord.com/invite/co-mmunity).
 
 ## How to Contribute
 
-1. Ensure your change has an issue! Find an existing issue or open a new issue.
-   - This is where you can get a feel if the change will be accepted or not.
-2. Once approved, fork this repository repository in your own GitHub account.
-3. [Fork this repository](https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/fork-a-repo)
-4. Make your changes on your fork and make sure all your [commits are signed](https://docs.github.com/en/authentication/managing-commit-signature-verification/about-commit-signature-verification)!
-5. [Submit the fork as a Pull Request](https://help.github.com/en/github/collaborating-with-issues-and-pull-requests/creating-a-pull-request-from-a-fork) pointing to the `main` branch this repository. A maintainer should comment and/or review your Pull Request within a few days. Although depending on the circumstances, it may take longer.
+1. **Open or Find an Issue**  
+   Ensure your contribution addresses an open issue. This helps gauge if the change will be accepted.
+
+2. **Fork the Repository**  
+   [Fork this repository](https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/fork-a-repo) to your GitHub account.
 
+3. **Make Your Changes**  
+   Work on your fork and ensure [your commits are signed](https://docs.github.com/en/authentication/managing-commit-signature-verification/about-commit-signature-verification).
 
-## How to add features
+4. **Submit a Pull Request (PR)**  
+   [Submit your fork as a PR](https://help.github.com/en/github/collaborating-with-issues-and-pull-requests/creating-a-pull-request-from-a-fork) pointing to the `main` branch. A maintainer will review your PR, but response times may vary.
 
-All features must be added to the `community` folder. That's where you'll find tools and deployments that are not managed by Codeowners.
+## How to Add Features
+
+Add new features to the `community` folder. This is where tools and deployments not managed by Codeowners are located.

README.md

@@ -2,7 +2,7 @@
 
 # Cohere Toolkit
 
-Toolkit is a collection of prebuilt components enabling users to quickly build and deploy RAG applications.
+Toolkit is a deployable all-in-one RAG application that enables users to quickly build their LLM-based product.
 
 - [Try Toolkit](#try-now)
 - [About Toolkit](#about-toolkit)
@@ -11,6 +11,7 @@ Toolkit is a collection of prebuilt components enabling users to quickly build a
 - [How to guides](/docs/how_to_guides.md)
   - [How to set up command model providers](/docs/command_model_providers.md)
   - [How to add tools](/docs/custom_tool_guides/tool_guide.md)
+  - [How to add auth to your tools](/docs/custom_tool_guides/tool_auth_guide.md)
   - [How to setup Google Drive](/docs/custom_tool_guides/google_drive.md)
   - [How to add authentication](/docs/auth_guide.md)
   - [How to deploy toolkit services](/docs/service_deployments.md)
@@ -23,15 +24,11 @@ Toolkit is a collection of prebuilt components enabling users to quickly build a
 
 ## Try Now:
 
-Try the default Toolkit application yourself by deploying it in a container locally. Either with `docker run`, using the pre-built Docker image provided (note: this does not contain community tools):
+*You will need to have [Docker](https://www.docker.com/products/docker-desktop/) and [Docker-compose >= 2.22](https://docs.docker.com/compose/install/) installed. [Go here for a more detailed setup.](/docs/setup.md)*
 
-```bash
-
-docker run -e COHERE_API_KEY='>>YOUR_API_KEY<<' -p 8000:8000 -p 4000:4000 ghcr.io/cohere-ai/cohere-toolkit:latest
+There are two main ways of running Toolkit:
 
-```
-
-or cloning and running locally:
+1. **(Recommended) Docker Compose setup**
 
 Note: to include community tools when building locally, set the `INSTALL_COMMUNITY_DEPS` build arg in the `docker-compose.yml` to `true`.
 
@@ -41,30 +38,36 @@ cd cohere-toolkit
 make first-run
 ```
 
-Go to [localhost:4000](http://localhost:4000/) in your browser and start chatting with the model. 
+2. **Docker Image setup**
 
-*For the above you will need to have [Docker](https://www.docker.com/products/docker-desktop/) and [Docker-compose >= 2.22](https://docs.docker.com/compose/install/) installed. [Go here for a more detailed setup.](/docs/setup.md)*
+We offer pre-built Docker images as well that you can run, but note that they might contain out-of-date code, so the local compose setup is preferred.
+
+```bash
+docker run -e COHERE_API_KEY='>>YOUR_API_KEY<<' -p 8000:8000 -p 4000:4000 ghcr.io/cohere-ai/cohere-toolkit:latest
+```
+
+Go to [localhost:4000](http://localhost:4000/) in your browser and start chatting with Cohere's Command model!
 
 ## GitHub Codespaces
 
-To run this project using GitHub Codespaces, please refer to [our Codespaces Setup Guide](/docs/github_codespaces.md).
+To run this project using GitHub Codespaces, please refer to our [Codespaces Setup Guide](/docs/github_codespaces.md).
 
 ## About Toolkit
 
 ![](/docs/assets/toolkit_graphic.png)
 
-- **Interfaces** - these can be any frontend, application, bot or integration. You can customize any type of interface for your use case. By default included is: 
-  - Cohere's Web UI at `src/interfaces/coral_web` - A web app built in Next.js. Includes a simple SQL database out of the box to store conversation history in the app.
-- **Backend API** - `src/backend` this follows a similar structure to the [Cohere Chat API](https://docs.cohere.com/reference/chat) but also include customizable elements: 
+- **Interfaces** - any client-side UI, currently contains a web app and Slack bot.
+  - Defaults to Cohere's Web UI at `src/interfaces/coral_web` - A web app built in Next.js. Includes a simple SQL database out of the box to store conversation history in the app.
+  - You can turn on the Assistants view that uses `src/interfaces/assistants_web` instead.
+- **Backend API** - in `src/backend` this follows a similar structure to the [Cohere Chat API](https://docs.cohere.com/reference/chat) but also include customizable elements: 
   - **Model** - you can customize with which provider you access Cohere's Command models. By default included in the toolkit is Cohere's Platform, Sagemaker, Azure, Bedrock, HuggingFace, local models. [More details here.](/docs/command_model_providers.md)
-  - **Retrieval**- you can customize tools and data sources that the application is run with. By default, we have configured a Langchain data retriever to test RAG on Wikipedia and your own uploaded documents. It is possible to add any tool including any tools or retrievers from LangChain or LlamaIndex. You can also use a connector you have created.
+  - **Retrieval**- you can customize tools and data sources that the application is run with.
 - **Service Deployment Guides** - we also include guides for how to deploy the toolkit services in production including with AWS, GCP and Azure. [More details here.](/docs/service_deployments.md)
 
 ## Contributing
 
 Contributions are what drive an open source community, any contributions made are greatly appreciated. To get started, check out our [documentation.](CONTRIBUTING.md)
 
-
 ## Contributors
 
 <a href="https://github.com/cohere-ai/cohere-toolkit/graphs/contributors">

docs/auth_guide.md

@@ -1,32 +1,45 @@
 # Authentication
 
-Prior to setting up any authentication, we recommend setting up the base project using the `make first-run` command. If you've already done so, you can skip ahead.
+Prior to setting up any authentication, you will need to set up the base project using the `make first-run` command. If you've already done so, and both `configuration.yaml` and `secrets.yaml` files have been generated, you can skip ahead. 
 
 ## Adding Auth strategies
 
-By default, the Toolkit does not enforce any authentication strategies, but they can be enabled from `src/backend/config/auth.py`.
+By default, Toolkit does not enforce any authentication strategies. These can be enabled in your `configuration.yaml` by using the `auth.enabled_auth` list parameter and adding `basic`, `google_oauth` or `oidc`.
 
-This is the current list of implemented Auth strategies:
+For example:
+
+```bash
+auth:
+    enabled_auth:
+        - basic
+        - google_oauth
+        - oidc
+```
+
+Will enabled all 3 auth strategies. A `secret_key` must then be added to the `secrets.yaml` file for generating secure JWT tokens.
+
+To generate an appropriate production `auth.secret_key` value, you can use the following python code:
+
+```python
+import secrets
+print(secrets.token_hex(32))
+```
+
+Individual strategies might still require additional configuration, see below for more details.
+
+## Available strategies
 
 - BasicAuthentication: for email and password auth, no setup required.
 
 - GoogleOAuth: for authentication with a Gmail account.
     - Requires a GCP project.
     - Requires setting up [Google OAuth 2.0](https://support.google.com/cloud/answer/6158849?hl=en).
     - In your `secrets.yaml` file, update the `auth.google_oauth.client_id` and `auth.google_oauth.client_secret` variables.
+
 - OpenIDConnect: generic auth class for OpenIDConnectOAuth providers.
     - Requires an OIDC app for your provider (Google, Microsoft, Amazon, etc.)
     - In your `secrets.yaml` file, update the `auth.oidc.client_id`, `auth.oidc.client_secret`, and `auth.oidc.well_known_endpoint` variables.
 
-To enable one or more of these strategies, add them to the `ENABLED_AUTH_STRATEGIES` list in the `backend/config/auth.py` file, then add any required environment variables in your `secrets.yaml` file, then generate a secret key to be used as the `auth.secret_key` environment variable. This is used to encode and decode your access tokens for both login OAuth flows and Tool auth.
-
-To generate an appropriate production `auth.secret_key` value, you can use the following python code:
-
-```python
-import secrets
-print(secrets.token_hex(32))
-```
-
 ## Configuring your OAuth app's Redirect URI
 
 When configuring your OAuth apps, make sure to whitelist the Redirect URI to the frontend endpoint, it should look like 

docs/command_model_providers.md

@@ -1,19 +1,27 @@
 # Command Model Provider Guides
 
-A model deployment is a running version of one of the Cohere command models. The Toolkit currently supports the model deployments:
+A **model deployment** refers to an active instance of one of the Cohere command models. The Toolkit currently supports the following model deployment options:
 
 ## Command Model Provider Options
 
-- Cohere Platform (model_deployments/cohere_platform.py)
-  - This model deployment option call the Cohere Platform with the [Cohere python SDK](https://github.com/cohere-ai/cohere-python). You will need a Cohere API key. When you create an account with Cohere, we automatically create a trial API key for you. You can find it [here](https://dashboard.cohere.com/api-keys).
-- Azure (model_deployments/azure.py)
-  - This model deployment calls into your Azure deployment. To get an Azure deployment [follow these steps](https://learn.microsoft.com/en-us/azure/ai-studio/how-to/deploy-models-cohere-command). Once you have a model deployed you will need to get the endpoint URL and API key from the azure AI studio https://ai.azure.com/build/ -> Project -> Deployments -> Click your deployment -> You will see your URL and API Key. Note to use the Cohere SDK you need to add `/v1` to the end of the url.
-- SageMaker (model_deployments/sagemaker.py)
-  - This deployment option calls into your SageMaker deployment. To create a SageMaker endpoint [follow the steps here](https://docs.cohere.com/docs/amazon-sagemaker-setup-guide), alternatively [follow a command notebook here](https://github.com/cohere-ai/cohere-aws/tree/main/notebooks/sagemaker). Note your region and endpoint name when executing the notebook as these will be needed in the environment variables.
-- Local models with LlamaCPP (community/model_deployments/local_model.py)
-  - This deployment option calls into a local model. To use this deployment you will need to download a model. You can use Cohere command models or choose between a range of other models that you can see [here](https://github.com/ggerganov/llama.cpp). You will need to enable community features to use this deployment by setting `USE_COMMUNITY_FEATURES=True` in your .env file.
-- To add your own deployment:
-  1. Create a deployment file, add it to [/community/model_deployments](https://github.com/cohere-ai/cohere-toolkit/tree/main/src/community/model_deployments) folder, implement the function calls from `BaseDeployment` similar to the other deployments.
-  2. Add the deployment to [src/community/config/deployments.py](https://github.com/cohere-ai/cohere-toolkit/blob/main/src/community/config/deployments.py)
-  3. Add the environment variables required to the env template.
-- To add a Cohere private deployment, use the steps above copying the cohere platform implementation changing the base_url for your private deployment and add in custom auth steps.
+- **Cohere Platform** (`model_deployments/cohere_platform.py`)
+  - This deployment option utilizes the [Cohere Python SDK](https://github.com/cohere-ai/cohere-python) to interface with the Cohere Platform. You will need a Cohere API key, which is automatically generated when you create an account. You can find your API key [here](https://dashboard.cohere.com/api-keys).
+
+- **Azure** (`model_deployments/azure.py`)
+  - This option connects to your Azure deployment. To set up an Azure deployment, follow the instructions [here](https://learn.microsoft.com/en-us/azure/ai-studio/how-to/deploy-models-cohere-command). After deploying a model, retrieve the endpoint URL and API key from Azure AI Studio: navigate to [https://ai.azure.com/build/](https://ai.azure.com/build/) -> Project -> Deployments -> Click your deployment to access the URL and API Key. Remember to append `/v1` to the endpoint URL when using the Cohere SDK.
+
+- **SageMaker** (`model_deployments/sagemaker.py`)
+  - This deployment option connects to your SageMaker endpoint. For setup instructions, refer to the [SageMaker setup guide](https://docs.cohere.com/docs/amazon-sagemaker-setup-guide) or use the command notebook available [here](https://github.com/cohere-ai/cohere-aws/tree/main/notebooks/sagemaker). Note your AWS region and endpoint name when executing the notebook, as these will be required as environment variables.
+
+- **Local Models with LlamaCPP** (`community/model_deployments/local_model.py`)
+  - This option enables the use of a local model. To implement this deployment, download a model of your choice. You can use Cohere command models or explore other available models [here](https://github.com/ggerganov/llama.cpp). Enable community features by setting `USE_COMMUNITY_FEATURES=True` in your `.env` file.
+
+## Adding Your Own Deployment
+
+1. Create a deployment file and place it in the [/community/model_deployments](https://github.com/cohere-ai/cohere-toolkit/tree/main/src/community/model_deployments) folder. Implement the function calls from `BaseDeployment`, mirroring the existing deployment structures.
+2. Register your deployment in [src/community/config/deployments.py](https://github.com/cohere-ai/cohere-toolkit/blob/main/src/community/config/deployments.py).
+3. Add any required environment variables to the `.env` template.
+
+## Adding a Cohere Private Deployment
+
+To add a Cohere private deployment, replicate the steps above for the Cohere Platform implementation, adjusting the `base_url` for your private deployment and incorporating any custom authentication steps.

docs/config_details/config_description.md

@@ -28,7 +28,6 @@
      - python_interpreter - Python interpreter configurations
        - url - URL of the python interpreter tool
   - feature_flags - Feature flags configurations
-       - use_experimental_langchain - Use tools based on the langchain
        - use_agents_view - Use agents view - if set to true, the frontend agents view will be available. 
          Please note that this setting is available only for the Coral web frontend. To change which frontend is used, set the context in the docker-compose file. 
        - use_community_features - Use community features, if set to true, the community tools and deployments will be available