Skip to content

Improve tool descriptions with usage examples to reduce model misinterpretation #33

@aborruso

Description

@aborruso

Problem

MCP tool descriptions are the primary interface through which LLMs decide when and how to call a tool. Vague or incomplete descriptions can lead to:

  • wrong tool selection
  • incorrect parameter construction
  • unnecessary retries / wasted tokens

Proposed Change

Audit and improve the description field of every registered MCP tool in src/tools/ to include:

  • a one-line purpose statement
  • 1–2 concrete usage examples inline (query patterns, parameter combos)
  • explicit notes on when NOT to use the tool (e.g. "use ckan_datastore_search for tabular data, not ckan_package_search")

Scope

Files to review:

  • src/tools/package.tsckan_package_search, ckan_package_show, ckan_find_relevant_datasets, ckan_analyze_datasets
  • src/tools/organization.tsckan_organization_list, ckan_organization_show, ckan_organization_search, ckan_group_*
  • src/tools/datastore.tsckan_datastore_search, ckan_datastore_search_sql
  • src/tools/status.tsckan_status_show, ckan_catalog_stats
  • src/tools/sparql.tssparql_query

Motivation

An article analyzing MCP reliability issues identified "models misinterpreting tools" as a primary failure mode, forcing developers to add fallback logic and guardrails. Better descriptions are the cheapest mitigation — no code changes to logic, just better prompting at the schema level.

Acceptance Criteria

  • Each tool description explicitly states its purpose and at least one example
  • Tools with overlapping functionality include a disambiguation note
  • Existing tests still pass (npm test)
  • No changes to tool input/output schemas

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions