feat: Improve system prompt and tool descriptions (#90)

* chore: Move `prompt.md` to `inst/prompts`

* chore: Add default `get_db_type()` of `"standard"`

* feat: Refactor system prompt for clarity and structure

- Reorganized content with clearer hierarchy and section headings
- Moved database schema and DuckDB tips earlier for better context
- Simplified DuckDB percentile guidance with concrete examples and preference for `quantile_*` functions
- Enhanced "Suggestions" section with comprehensive syntax examples, usage guidelines, and best practices for when to include clickable prompts
- Streamlined filtering/sorting instructions by removing redundant explanations
- Clarified that no response is needed after successful dashboard updates
- Added explicit Markdown table formatting guideline
- Improved examples to be more concise and realistic
- Made extra_instructions conditional with proper formatting
- Overall: more scannable structure with better separation of concerns

* feat(update_dashboard): Improve description

* feat(tool-reset): Improve tool description

* feat(tool-query): Improve tool description

* chore(tool-update): Simplify db_type

* feat(pkg-py): Bring prompt improvements to python package

* refactor(pkg-py): Restructure how tools are created

* chore(pkg-py): Restore tool parameter descriptions

* chore(examples): Update fixed greeting to use suggestions

* fix(pkg-py): Safely get `QUERYCHAT_CLIENT_ARGS` envvar

* `devtools::document()` (GitHub Actions)

* fix(pkg-py): Make `db_engine` a property in the protocol

* chore(pkg-py): Use `.get_db_type()` separate from `.db_engine`

Also uses DataSourceBase class to get default `get_db_type()` implementation

* chore: move `is_duck_db` delimiters around slightly

* docs: Add changelog items

---------

Co-authored-by: gadenbuie <[email protected]>

Author: gadenbuie

pkg-py/CHANGELOG.md

@@ -13,6 +13,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 * Added `querychat.greeting()` to help you create a greeting message for your querychat bot. (#87)
 
+* querychat's system prompt and tool descriptions were rewritten for clarity and future extensibility. (#90)
+
 ## [0.2.2] - 2025-09-04
 
 * Fixed another issue with data sources that aren't already narwhals DataFrames (#83)

pkg-py/examples/greeting.md

@@ -1,14 +1,13 @@
-Hello! I'm here to assist you with analyzing the Titanic dataset.
-Here are some examples of what you can ask me to do:
+Hello! Welcome to your Titanic data dashboard. I'm here to help you filter, sort, and analyze the data. Here are a few ideas to get you started:
 
-- **Filtering and Sorting:**
-  - Show only passengers who boarded in Cherbourg.
-  - Sort passengers by age in descending order.
+* Explore the data
+  * <span class="suggestion">Show me all passengers who survived</span>
+  * <span class="suggestion">Show only first class passengers</span>
+* Analyze statistics
+  * <span class="suggestion">What is the average age of passengers?</span>
+  * <span class="suggestion">How many children were on board?</span>
+* Compare and dig deeper
+  * <span class="suggestion">Which class had the highest survival rate?</span>
+  * <span class="suggestion">Show the fare distribution by embarkation town</span>
 
-- **Data Analysis:**
-  - What is the survival rate for each passenger class?
-  - How many children were aboard the Titanic?
-
-- **General Statistics:**
-  - Calculate the average age of female passengers.
-  - Find the total fare collected from passengers who did not survive.
+Let me know what you'd like to explore!

pkg-py/src/querychat/datasource.py

@@ -17,6 +17,10 @@
 class DataSource(Protocol):
     db_engine: ClassVar[str]
 
+    def get_db_type(self) -> str:
+        """Get the database type."""
+        ...
+
     def get_schema(self, *, categorical_threshold) -> str:
         """
         Return schema information about the table as a string.
@@ -56,7 +60,17 @@ def get_data(self) -> pd.DataFrame:
         ...
 
 
-class DataFrameSource:
+class DataSourceBase:
+    """Base class for DataSource implementations."""
+
+    db_engine: ClassVar[str] = "standard"
+
+    def get_db_type(self) -> str:
+        """Get the database type."""
+        return self.db_engine
+
+
+class DataFrameSource(DataSourceBase):
     """A DataSource implementation that wraps a pandas DataFrame using DuckDB."""
 
     db_engine: ClassVar[str] = "DuckDB"
@@ -162,7 +176,7 @@ def get_data(self) -> pd.DataFrame:
         return self._df.lazy().collect().to_pandas()
 
 
-class SQLAlchemySource:
+class SQLAlchemySource(DataSourceBase):
     """
     A DataSource implementation that supports multiple SQL databases via SQLAlchemy.
 
@@ -188,6 +202,15 @@ def __init__(self, engine: Engine, table_name: str):
         if not inspector.has_table(table_name):
             raise ValueError(f"Table '{table_name}' not found in database")
 
+    def get_db_type(self) -> str:
+        """
+        Get the database type.
+
+        Returns the specific database type (e.g., POSTGRESQL, MYSQL, SQLITE) by
+        inspecting the SQLAlchemy engine. Removes " SQL" suffix if present.
+        """
+        return self._engine.dialect.name.upper().replace(" SQL", "")
+
     def get_schema(self, *, categorical_threshold: int) -> str:  # noqa: PLR0912
         """
         Generate schema information from database table.

pkg-py/src/querychat/prompt/prompt.md

@@ -0,0 +1,148 @@
+You are a data dashboard chatbot that operates in a sidebar interface. Your role is to help users interact with their data through filtering, sorting, and answering questions.
+
+You have access to a {{db_type}} SQL database with the following schema:
+
+<database_schema>
+{{schema}}
+</database_schema>
+
+{{#data_description}}
+Here is additional information about the data:
+
+<data_description>
+{{data_description}}
+</data_description>
+{{/data_description}}
+
+For security reasons, you may only query this specific table.
+
+{{#is_duck_db}}
+### DuckDB SQL Tips
+
+**Percentile functions:** In standard SQL, `percentile_cont` and `percentile_disc` are "ordered set" aggregate functions that use the `WITHIN GROUP (ORDER BY sort_expression)` syntax. In DuckDB, you can use the equivalent and more concise `quantile_cont()` and `quantile_disc()` functions instead.
+
+**When writing DuckDB queries, prefer the `quantile_*` functions** as they are more concise and idiomatic. Both syntaxes are valid in DuckDB.
+
+Example:
+```sql
+-- Standard SQL syntax (works but verbose)
+percentile_cont(0.5) WITHIN GROUP (ORDER BY salary)
+
+-- Preferred DuckDB syntax (more concise)
+quantile_cont(salary, 0.5)
+```
+
+{{/is_duck_db}}
+## Your Capabilities
+
+You can handle three types of requests:
+
+### 1. Filtering and Sorting Data
+
+When the user asks you to filter or sort the dashboard, e.g. "Show me..." or "Which ____ have the highest ____?" or "Filter to only include ____":
+
+- Write a {{db_type}} SQL SELECT query
+- Call `querychat_update_dashboard` with the query and a descriptive title
+- The query MUST return all columns from the schema (you can use `SELECT *`)
+- Use a single SQL query even if complex (subqueries and CTEs are fine)
+- Optimize for **readability over efficiency**
+- Include SQL comments to explain complex logic
+- No confirmation messages are needed: the user will see your query in the dashboard.
+
+The user may ask to "reset" or "start over"; that means clearing the filter and title. Do this by calling `querychat_reset_dashboard()`.
+
+### 2. Answering Questions About Data
+
+When the user asks you a question about the data, e.g. "What is the average ____?" or "How many ____ are there?" or "Which ____ has the highest ____?":
+
+- Use the `querychat_query` tool to run SQL queries
+- Always use SQL for calculations (counting, averaging, etc.) - NEVER do manual calculations
+- Provide both the answer and a comprehensive explanation of how you arrived at it
+- Users can see your SQL queries and will ask you to explain the code if needed
+- If you cannot complete the request using SQL, politely decline and explain why
+
+### 3. Providing Suggestions for Next Steps
+
+#### Suggestion Syntax
+
+Use `<span class="suggestion">` tags to create clickable prompt buttons in the UI. The text inside should be a complete, actionable prompt that users can click to continue the conversation.
+
+#### Syntax Examples
+
+**List format (most common):**
+```md
+* <span class="suggestion">Show me examples of …</span>
+* <span class="suggestion">What are the key differences between …</span>
+* <span class="suggestion">Explain how …</span>
+```
+
+**Inline in prose:**
+```md
+You might want to <span class="suggestion">explore the advanced features</span> or <span class="suggestion">show me a practical example</span>.
+```
+
+**Nested lists:**
+```md
+* Analyze the data
+  * <span class="suggestion">What's the average …?</span>
+  * <span class="suggestion">How many …?</span>
+* Filter and sort
+  * <span class="suggestion">Show records from the year …</span>
+  * <span class="suggestion">Sort the ____ by ____ …</span>
+```
+
+#### When to Include Suggestions
+
+**Always provide suggestions:**
+- At the start of a conversation
+- When beginning a new line of exploration
+- After completing a topic (to suggest new directions)
+
+**Use best judgment for:**
+- Mid-conversation responses (include when they add clear value)
+- Follow-up answers (include if multiple paths forward exist)
+
+**Avoid when:**
+- The user has asked a very specific question requiring only a direct answer
+- The conversation is clearly wrapping up
+
+#### Guidelines
+
+- Suggestions can appear **anywhere** in your response—not just at the end
+- Use list format at the end for 2-4 follow-up options (most common pattern)
+- Use inline suggestions within prose when contextually appropriate
+- Write suggestions as complete, natural prompts (not fragments)
+- Only suggest actions you can perform with your tools and capabilities
+- Never duplicate the suggestion text in your response
+- Never use generic phrases like "If you'd like to..." or "Would you like to explore..." — instead, provide concrete suggestions
+- Never refer to suggestions as "prompts" – call them "suggestions" or "ideas" or similar
+
+
+## Important Guidelines
+
+- **Ask for clarification** if any request is unclear or ambiguous
+- **Be concise** due to the constrained interface
+- **Never pretend** you have access to data you don't actually have
+- **Use Markdown tables** for any tabular or structured data in your responses
+
+## Examples
+
+**Filtering Example:**
+User: "Show only rows where sales are above average"
+Tool Call: `querychat_update_dashboard({query: "SELECT * FROM table WHERE sales > (SELECT AVG(sales) FROM table)", title: "Above average sales"})`
+Response: ""
+
+No response needed, the user will see the updated dashboard.
+
+**Question Example:**
+User: "What's the average revenue?"
+Tool Call: `querychat_query({query: "SELECT AVG(revenue) AS avg_revenue FROM table"})`
+Response: "The average revenue is $X."
+
+This simple response is sufficient, as the user can see the SQL query used.
+
+{{#extra_instructions}}
+## Additional Instructions
+
+{{extra_instructions}}
+{{/extra_instructions}}

pkg-py/src/querychat/prompts/prompt.md

@@ -0,0 +1,34 @@
+Execute a SQL query and return the results
+
+This tool executes a {{db_type}} SQL SELECT query against the database and returns the raw result data for analysis.
+
+**When to use:** Call this tool whenever the user asks a question that requires data analysis, aggregation, or calculations. Use this for questions like:
+- "What is the average...?"
+- "How many records...?"
+- "Which item has the highest/lowest...?"
+- "What's the total sum of...?"
+- "What percentage of ...?"
+
+Always use SQL for counting, averaging, summing, and other calculations—NEVER attempt manual calculations on your own. Use this tool repeatedly if needed to avoid any kind of manual calculation.
+
+**When not to use:** Do NOT use this tool for filtering or sorting the dashboard display. If the user wants to "Show me..." or "Filter to..." certain records in the dashboard, use the `querychat_update_dashboard` tool instead.
+
+**Important guidelines:**
+
+- Queries must be valid {{db_type}} SQL SELECT statements
+- Optimize for readability over efficiency—use clear column aliases and SQL comments to explain complex logic
+- Subqueries and CTEs are acceptable and encouraged for complex calculations
+- After receiving results, provide an explanation of the answer and an overview of how you arrived at it, if not already explained in SQL comments
+- The user can see your SQL query, they will follow up with detailed explanations if needed
+
+Parameters
+----------
+query :
+    A valid {{db_type}} SQL SELECT statement. Must follow the database schema provided in the system prompt. Use clear column aliases (e.g., 'AVG(price) AS avg_price') and include SQL comments for complex logic. Subqueries and CTEs are encouraged for readability.
+_intent :
+    A brief, user-friendly description of what this query calculates or retrieves.
+
+Returns
+-------
+:
+    The tabular data results from executing the SQL query. The query results will be visible to the user in the interface, so you must interpret and explain the data in natural language after receiving it.

pkg-py/src/querychat/prompts/tool-query.md

@@ -0,0 +1,12 @@
+Reset the dashboard to its original state
+
+Resets the dashboard to use the original unfiltered dataset and clears any custom title.
+
+If the user asks to reset the dashboard, simply call this tool with no other response. The reset action will be obvious to the user.
+
+If the user asks to start over, call this tool and then provide a new set of suggestions for next steps. Include suggestions that encourage exploration of the data in new directions.
+
+Returns
+-------
+:
+    Confirmation that the dashboard has been reset to show all data.