-
Notifications
You must be signed in to change notification settings - Fork 1.3k
feat: Let the AI hold the line (New Wait Tool with Music & Interrupts) #572
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
arnofrxdd
wants to merge
5
commits into
dograh-hq:main
Choose a base branch
from
arnofrxdd:wait-tool
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
5 commits
Select commit
Hold shift + click to select a range
1fe20e0
feat: complete wait_for_user tool implementation
arnofrxdd 6a9f827
fix: address PR review feedback on wait tool
arnofrxdd 69a7c36
fix: update alembic down revision to current head
arnofrxdd b685ee9
fix: add 5 minute maximum limit to wait tool seconds parameter
arnofrxdd ddb9eda
fixes
arnofrxdd File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,87 @@ | ||
| # Future Tools Roadmap | ||
|
|
||
| Tools planned for future implementation in Dograh. | ||
|
|
||
| --- | ||
|
|
||
| ## 1. `get_current_time` / `convert_time` | ||
|
|
||
| **Status:** Backend fully implemented in `api/services/workflow/tools/timezone.py` — just needs wiring up. | ||
|
|
||
| **What it does:** | ||
| - `get_current_time(timezone)` — Agent tells the caller the current time in any timezone | ||
| - `convert_time(source_timezone, time, target_timezone)` — Agent converts a time between two timezones | ||
|
|
||
| **Use case:** | ||
| Scheduling agents, appointment bots, international call centers. | ||
| > *"What time is it in New York right now?"* → Agent calls `get_current_time("America/New_York")` and responds accurately. | ||
|
|
||
| **What needs to be done:** | ||
| - Add `TIME = "time"` to `ToolCategory` in `api/enums.py` | ||
| - Add a new Alembic migration for the enum | ||
| - Register `get_current_time` and `convert_time` handlers in `pipecat_engine_custom_tools.py` | ||
| - Add the tool category config in `ui/src/app/tools/config.tsx` | ||
| - Add `WaitToolDefinition` equivalent for time tool in `ui/src/client/types.gen.ts` | ||
|
|
||
| --- | ||
|
|
||
| ## 2. `play_audio_clip` | ||
|
|
||
| **Status:** Not implemented. Internal `play_audio` utility exists in `api/services/pipecat/audio_playback.py` but is not exposed as an LLM-callable tool. | ||
|
|
||
| **What it does:** | ||
| - LLM dynamically decides mid-call to play a specific pre-recorded audio clip | ||
| - Different from current usage where audio is always pre-configured and fixed (e.g. hold music during wait, goodbye on end call) | ||
|
|
||
| **Use case:** | ||
| Regulated industries — finance, insurance, healthcare — where specific legal disclaimers must be read **verbatim** from a pre-recorded clip, triggered based on conversation context. | ||
| > *User asks about refund policy* → LLM calls `play_audio_clip(clip_id="refund_disclaimer")` → exact approved recording plays. | ||
|
|
||
| Also useful for: brand voice recordings, non-verbal audio (beeps/tones), multilingual clips mid-call. | ||
|
|
||
| **What needs to be done:** | ||
| - Add `AUDIO_CLIP = "audio_clip"` to `ToolCategory` in `api/enums.py` | ||
| - Add a new Alembic migration for the enum | ||
| - Implement the tool handler in `pipecat_engine_custom_tools.py` using existing `play_audio()` | ||
| - UI config in `config.tsx` with a clip picker (select from existing recordings) | ||
| - The clip library would use the existing recordings/uploads system in Dograh | ||
|
|
||
| --- | ||
|
|
||
| ## 3. `send_dtmf` | ||
|
|
||
| **Status:** Not implemented. Dograh has inbound DTMF (receive user keypresses) but NOT outbound DTMF (agent sends keypresses). | ||
|
|
||
| **What it does:** | ||
| Agent programmatically sends DTMF tones through an outbound call. | ||
|
|
||
| **Use case:** | ||
| Agent makes an outbound call and the other end is an IVR system. | ||
| > *Clinic IVR: "Press 1 for appointments, press 2 for billing"* | ||
| > Agent calls `send_dtmf(digit="1")` → tone is sent → IVR proceeds to appointments menu. | ||
|
|
||
| **What needs to be done:** | ||
| - Implement per-provider: Twilio (`calls.update(dtmf=...)`) , Plivo (`send_digits`), Telnyx, etc. | ||
| - Each telephony provider has its own API for sending DTMF tones mid-call | ||
| - Register handler in `pipecat_engine_custom_tools.py` | ||
| - Expose in UI as a built-in tool | ||
|
|
||
| --- | ||
|
|
||
| ## 4. `send_sms` | ||
|
|
||
| **Status:** Not implemented. Telephony integrations for Twilio/Plivo exist in Dograh, but there is no SMS sending capability exposed. | ||
|
|
||
| **What it does:** | ||
| Agent programmatically sends a text message to the caller (or another number) during or after the call. | ||
|
|
||
| **Use case:** | ||
| Agent finishes an action and sends confirmation details. | ||
| > *Agent: "I've booked your appointment. I'll text you the confirmation details right now."* | ||
| > Agent calls `send_sms(message="Your appointment is confirmed for tomorrow at 3pm.")` → SMS is delivered via the telephony provider. | ||
|
|
||
| **What needs to be done:** | ||
| - Implement SMS sending for each provider (Twilio Programmable SMS, Plivo SMS API). | ||
| - Register `send_sms` handler in `pipecat_engine_custom_tools.py`. | ||
| - Expose in UI as a built-in tool. (Note: May require handling regulatory constraints like DLT in India). | ||
|
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,37 @@ | ||
| # Dograh Git Workflow (Fork-and-Pull) | ||
|
|
||
| This document outlines the gold standard Git workflow for contributing to the Dograh repository, using the Fork-and-Pull model. | ||
|
|
||
| ## Why this is the best approach: | ||
| 1. **Pristine Local `main` Branch**: By never committing directly to your local `main` branch, you guarantee it will never drift or get into messy conflict states with `upstream/main`. Your local `main` essentially just acts as a clean mirror of the official Dograh repository. *(Note: Your local `main` is also what is deployed to your server, so keeping it clean and stable is critical!)* | ||
| 2. **Feature Isolation**: By creating a new branch (like `wait-tool`) off your clean `main` and pushing it to your personal fork (`arnofrxdd`), you isolate your work. If you make a mistake, you can always just delete the branch and start over from your clean `main` without losing any official repo code. | ||
| 3. **Painless Updates**: When you pull updates from `upstream/main` every 1-3 days, you never have to deal with massive merge conflicts. Your local `main` fast-forwards instantly, and any new branches you create are built on top of the latest, most stable code. | ||
|
|
||
| ## The Ideal Development Cycle: | ||
|
|
||
| 1. **Go to your clean local mirror:** | ||
| ```bash | ||
| git checkout main | ||
| ``` | ||
|
|
||
| 2. **Sync your local main with Dograh's latest code:** | ||
| ```bash | ||
| git pull upstream main | ||
| ``` | ||
|
|
||
| 3. **Create an isolated space for your new feature/fix:** | ||
| ```bash | ||
| git checkout -b new-feature | ||
| ``` | ||
|
|
||
| 4. **Write code...** (Make your commits here) | ||
|
|
||
| 5. **Push to your fork and open a PR:** | ||
| ```bash | ||
| git push arnofrxdd new-feature | ||
| ``` | ||
|
|
||
| 6. **Go back to safety and repeat!** | ||
| ```bash | ||
| git checkout main | ||
| ``` |
64 changes: 64 additions & 0 deletions
64
api/alembic/versions/ceeaf3c37b2f_add_wait_to_tool_category.py
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,64 @@ | ||
| """add wait to tool category | ||
|
|
||
| Revision ID: ceeaf3c37b2f | ||
| Revises: gg11dd223344 | ||
| Create Date: 2026-07-22 12:42:18.521312 | ||
|
|
||
| """ | ||
| from typing import Sequence, Union | ||
|
|
||
| from alembic import op | ||
| import sqlalchemy as sa | ||
|
|
||
|
|
||
| from alembic_postgresql_enum import TableReference | ||
|
|
||
| # revision identifiers, used by Alembic. | ||
| revision: str = 'ceeaf3c37b2f' | ||
| down_revision: Union[str, None] = '00b0201ad918' | ||
| branch_labels: Union[str, Sequence[str], None] = None | ||
| depends_on: Union[str, Sequence[str], None] = None | ||
|
|
||
|
|
||
| def upgrade() -> None: | ||
| op.sync_enum_values( | ||
| enum_schema="public", | ||
| enum_name="tool_category", | ||
| new_values=[ | ||
| "http_api", | ||
| "end_call", | ||
| "transfer_call", | ||
| "calculator", | ||
| "native", | ||
| "integration", | ||
| "mcp", | ||
| "wait", | ||
| ], | ||
| affected_columns=[ | ||
| TableReference( | ||
| table_schema="public", table_name="tools", column_name="category" | ||
| ) | ||
| ], | ||
| enum_values_to_rename=[], | ||
| ) | ||
|
|
||
| def downgrade() -> None: | ||
| op.sync_enum_values( | ||
| enum_schema="public", | ||
| enum_name="tool_category", | ||
| new_values=[ | ||
| "http_api", | ||
| "end_call", | ||
| "transfer_call", | ||
| "calculator", | ||
| "native", | ||
| "integration", | ||
| "mcp", | ||
| ], | ||
| affected_columns=[ | ||
| TableReference( | ||
| table_schema="public", table_name="tools", column_name="category" | ||
| ) | ||
| ], | ||
| enum_values_to_rename=[], | ||
| ) |
Binary file not shown.
Binary file not shown.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
P0: Cannot create WaitToolDefinition through the API — ToolCategory enum in api/enums.py is missing a WAIT = "wait" member. The validate_category validator, DB CheckConstraint, and route-level validation all check category against ToolCategory values, so any request with category="wait" will be rejected before reaching the DB. Add
WAIT = "wait"to the ToolCategory enum to match the new ToolCategoryValue literal and WaitToolDefinition.Prompt for AI agents