Skip to content

Commit

Permalink
Merge branch 'main' into add-file-content-viewer
Browse files Browse the repository at this point in the history
  • Loading branch information
danylo-boiko authored Nov 1, 2024
2 parents e556faf + 404f286 commit faf33ff
Show file tree
Hide file tree
Showing 26 changed files with 733 additions and 52 deletions.
26 changes: 19 additions & 7 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@ Toolkit is a deployable all-in-one RAG application that enables users to quickly
- [How to add tools](/docs/custom_tool_guides/tool_guide.md)
- [How to add auth to your tools](/docs/custom_tool_guides/tool_auth_guide.md)
- [How to setup Google Drive](/docs/custom_tool_guides/google_drive.md)
- [How to setup Slack Tool](/docs/custom_tool_guides/slack.md)
- [How to setup Google Text-to-Speech](/docs/text_to_speech.md)
- [How to add authentication](/docs/auth_guide.md)
- [How to deploy toolkit services](/docs/service_deployments.md)
Expand All @@ -24,22 +25,33 @@ Toolkit is a deployable all-in-one RAG application that enables users to quickly
![](/docs/assets/toolkit.gif)

## Try Now:
There are two main ways for quickly running Toolkit: local and cloud. See the specific instructions given below.
### Local
*You will need to have [Docker](https://www.docker.com/products/docker-desktop/) and [Docker-compose >= 2.22](https://docs.docker.com/compose/install/) installed. [Go here for a more detailed setup.](/docs/setup.md)*
Note: to include community tools when building locally, set the `INSTALL_COMMUNITY_DEPS` build arg in the `docker-compose.yml` to `true`.

*You will need to have [Docker](https://www.docker.com/products/docker-desktop/) and [Docker-compose >= 2.22](https://docs.docker.com/compose/install/) installed. [Go here for a more detailed setup.](/docs/setup.md)*
Both options will make the frontend available at http://localhost:4000.

There are two main ways of running Toolkit:
#### Using `make`
Use the provided Makefile to simplify and automate your development workflow with Cohere Toolkit, including Docker Compose management, testing, linting, and environment setup.
```bash
git clone https://github.com/cohere-ai/cohere-toolkit.git
cd cohere-toolkit
make first-run
```

**Docker Compose setup**

Note: to include community tools when building locally, set the `INSTALL_COMMUNITY_DEPS` build arg in the `docker-compose.yml` to `true`.

#### Docker Compose only
Use Docker Compose directly if you want to quickly spin up and manage your container environment without the additional automation provided by the Makefile.
```bash
git clone https://github.com/cohere-ai/cohere-toolkit.git
cd cohere-toolkit
make first-run
docker compose up
docker compose run --build backend alembic -c src/backend/alembic.ini upgrade head
```

## GitHub Codespaces
### Cloud
#### GitHub Codespaces

To run this project using GitHub Codespaces, please refer to our [Codespaces Setup Guide](/docs/github_codespaces.md).

Expand Down
131 changes: 131 additions & 0 deletions docs/custom_tool_guides/slack.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,131 @@
# Slack Tool Setup

To set up the Slack tool you will need a Slack application. Follow the steps below to set it up:

## 1. Create a Slack App

Head to the [Slack API](https://api.slack.com/apps) and create a new app.
After creating the app, you will see the `App Credentials` section. Copy the `Client ID` and `Client Secret` values.
That will be used for the environment variables specified above.

## 2. Set up OAuth & Permissions
OAuth flow is required to authenticate users with Slack.
To enable it please set the following redirect URL to your app's settings:
```bash
https://<your_backend_url>/v1/tool/auth
```
Please note that for the local development you will need to enable HTTPS.
See the [Setup HTTPS for Local Development](#5-setup-https-for-local-development) section for more details.
If you are using a local https setup, redirect url should be
```
https://localhost:8000/v1/tool/auth
```
Also, you can set up a proxy, such as [ngrok](https://ngrok.com/docs/getting-started/), to expose your local server to the internet.

The Slack tool uses User Token Scopes to access the user's Slack workspace.
The required and the default permission scope is `search:read`.
Set it in the `OAuth & Permissions` section of your Slack app settings.

To work with the Slack Tool Advanced token security via token rotation is required.
To enable it, go to the `OAuth & Permissions` section of your Slack app settings and click 'Opt in' button in the 'Advanced token security via token rotation' section.

More information about the OAuth flow can be found [here](https://api.slack.com/authentication/oauth-v2).

## 3. Set Up Environment Variables

Then set the following environment variables. You can either set the below values in your `secrets.yaml` file:
```bash
slack:
client_id: <your_client_id from step 1>
client_secret: <your_client_secret from step 1>
```
or update your `.env` configuration to contain:
```bash
SLACK_CLIENT_ID=<your_client_id from step 1>
SLACK_CLIENT_SECRET=<your_client_secret from step 1>
```

## 4. Enable the Slack Tool in the Frontend

To enable the Slack tool in the frontend, you will need to modify the `src/community/config/tools.py` file. Add the `TOOL_SLACK_ID` to the `AGENT_SETTINGS_TOOLS` list.

```typescript
export const AGENT_SETTINGS_TOOLS = [
TOOL_HYBRID_WEB_SEARCH_ID,
TOOL_PYTHON_INTERPRETER_ID,
TOOL_WEB_SCRAPE_ID,
TOOL_SLACK_ID,
];
```

To enable the Slack tool in the frontend for Base Agent, you will need to modify the `src/community/config/tools.py` file. Remove the `TOOL_SLACK_ID` from the `BASE_AGENT_EXCLUDED_TOOLS` list.
By default, the Slack Tool is disabled for the Base Agent. Also if you need to exclude some tool from the Base Agent just add it to the `BASE_AGENT_EXCLUDED_TOOLS` list.
```typescript
export const BASE_AGENT_EXCLUDED_TOOLS = [];
```

## 5. Setup HTTPS for Local Development

To enable HTTPS for local development, the self-signed certificate needs to be generated.
Run the following command in the project root directory to generate the certificate and key:

```bash
openssl req -x509 -newkey rsa:4096 -nodes -out cert.pem -keyout key.pem -days 365
```

Then, update the backend Docker configuration(src/backend/Dockerfile) to use the generated certificate.
Just change next lines in the Dockerfile:
```Dockerfile
COPY pyproject.toml poetry.lock cert.pem key.pem ./
```
and
```Dockerfile
CMD uvicorn backend.main:app --reload --host 0.0.0.0 --port ${PORT} --timeout-keep-alive 300 --ssl-keyfile /workspace/key.pem --ssl-certfile /workspace/cert.pem
```
Change NEXT_PUBLIC_API_HOSTNAME environment variable in the .env `https` protocol:
```bash
NEXT_PUBLIC_API_HOSTNAME=https://localhost:8000
```

or in the configurations.yaml file:

```yaml
auth:
backend_hostname: https://localhost:8000
```
To run the Frontend with HTTPS, update the `start` script in the `package.json` file:

```json
"scripts": {
"dev": "next dev --port 4000 --experimental-https",
..........
}
```

Add the following line to the 'docker-compose.yml' file to the frontend environment variables:

```yaml
NEXT_PUBLIC_API_HOSTNAME=https://localhost:8000
```

and change the API_HOSTNAME to

```yaml
API_HOSTNAME: https://localhost:8000
```
also change the src/interfaces/assistants_web/.env.development file env variables to use https.

## 6. Run the Backend and Frontend

run next command to start the backend and frontend:

```bash
make dev
```

## 7. Troubleshooting

If you encounter any issues with OAuth, please check the following [link](https://api.slack.com/authentication/oauth-v2#errors)
For example, if you see the invalid_team_for_non_distributed_app error,
please ensure the app is distributed or try logging in with the workspace owner's account.
37 changes: 21 additions & 16 deletions poetry.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 1 addition & 0 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -57,6 +57,7 @@ llama-index = "^0.11.10"
llama-index-llms-cohere = "^0.3.0"
llama-index-embeddings-cohere = "^0.2.1"
google-cloud-texttospeech = "^2.18.0"
slack-sdk = "^3.33.1"

[tool.poetry.group.dev]
optional = true
Expand Down
3 changes: 3 additions & 0 deletions src/backend/config/configuration.template.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,9 @@ tools:
- tavily_web_search
python_interpreter:
url: http://terrarium:8080
slack:
user_scopes:
- search:read
feature_flags:
# Experimental features
use_agents_view: false
Expand Down
3 changes: 3 additions & 0 deletions src/backend/config/secrets.template.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,9 @@ tools:
google_web_search:
api_key:
cse_id:
slack:
client_id:
client_secret:
auth:
secret_key:
google_oauth:
Expand Down
23 changes: 22 additions & 1 deletion src/backend/config/settings.py
Original file line number Diff line number Diff line change
Expand Up @@ -146,6 +146,24 @@ class GDriveSettings(BaseSettings, BaseModel):
)


class SlackSettings(BaseSettings, BaseModel):
model_config = SETTINGS_CONFIG
client_id: Optional[str] = Field(
default=None,
validation_alias=AliasChoices("SLACK_CLIENT_ID", "client_id"),
)
client_secret: Optional[str] = Field(
default=None,
validation_alias=AliasChoices("SLACK_CLIENT_SECRET", "client_secret"),
)
user_scopes: Optional[str] = Field(
default=None,
validation_alias=AliasChoices(
"SLACK_USER_SCOPES", "scopes"
),
)


class TavilyWebSearchSettings(BaseSettings, BaseModel):
model_config = SETTINGS_CONFIG
api_key: Optional[str] = Field(
Expand Down Expand Up @@ -195,10 +213,13 @@ class ToolSettings(BaseSettings, BaseModel):
brave_web_search: Optional[BraveWebSearchSettings] = Field(
default=BraveWebSearchSettings()
)

hybrid_web_search: Optional[HybridWebSearchSettings] = Field(
default=HybridWebSearchSettings()
)
slack: Optional[SlackSettings] = Field(
default=SlackSettings()
)



class DatabaseSettings(BaseSettings, BaseModel):
Expand Down
Loading

0 comments on commit faf33ff

Please sign in to comment.