Skip to content

Capture config registry recommendation outcomes - #187

Merged
masnwilliams merged 4 commits into
mainfrom
hypeship/bot-detection-feedback
Sep 14, 2026
Merged

masnwilliams merged 4 commits into
mainfrom
hypeship/bot-detection-feedback

Conversation

@masnwilliams

@masnwilliams masnwilliams commented Sep 14, 2026

Copy link
Copy Markdown
Collaborator

summary

  • extend submit_feedback with structured bot-detection and config-registry outcome routes
  • require config-registry feedback to include the browser session ID, recommendation metadata and evidence, and exact browser and proxy settings applied unchanged
  • emit a deterministic, privacy-safe applied-config key plus flattened PostHog properties so failed and successful outcomes can be grouped by configuration
  • validate public registrable domains with the public suffix list and reject URLs, subdomains, private hosts, and sensitive proxy details
  • preserve every existing feedback contract

why

A site-level block identifies missing coverage, but it does not show whether an existing config-registry recommendation failed after an agent used it. The config-registry route connects the observed outcome to the browser session, recommendation evidence, and applied configuration. If an agent changes the recommendation before testing, it uses the general bot-detection route instead so the result is not incorrectly attributed to the original config.

testing

  • bun test — 453 passed
  • bunx tsc --noEmit
  • KERNEL_CLI_PROD_CLIENT_ID=test-prod KERNEL_CLI_STAGING_CLIENT_ID=test-staging KERNEL_CLI_DEV_CLIENT_ID=test-dev bun run build
  • bunx prettier --check README.md src/lib/mcp/tools/feedback.ts src/lib/mcp/tools/feedback.test.ts src/lib/mcp/analytics.ts src/lib/mcp/analytics.test.ts package.json

bun run format:check still reports the pre-existing AGENTS.md formatting warning; all changed files pass Prettier.


Note

Medium Risk
Changes the public MCP tool input schema and analytics payload shape agents rely on; validation and redaction reduce privacy risk but mis-routed feedback could affect config-registry prioritization signals.

Overview
Extends submit_feedback with two new routes: bot_detection for site pass/challenge/block/degraded outcomes not tied to an unchanged registry recommendation, and config_registry when a recommendation was applied unchanged and the result should be attributed to that configuration.

Agents must supply structured bot_detection (public registrable domain via tldts, outcome, reproducibility, optional session/browser/proxy metadata) and, for config-registry reports, config_registry (request method, recommendation evidence, exact applied browser and proxy settings). The handler enforces cross-field rules (e.g. browser_session_id required for config-registry) and rejects URLs, subdomains, private hosts, and bot_detection on non-site feedback types.

PostHog capture adds feedback_destination (config_registry_prioritization vs config_registry_quality), a deterministic applied_config_key, and allow-listed flattened properties. README documents the new routing; existing product/mcp/docs feedback behavior is unchanged.

Reviewed by Cursor Bugbot for commit 58c5b15. Bugbot is set up for automated code reviews on this repo. Configure here.

@vercel

vercel Bot commented Sep 14, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
mcp Ready Ready Preview Sep 14, 2026 11:14pm UTC

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using default effort and found 2 potential issues.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 450b408. Configure here.

Comment thread src/lib/mcp/tools/feedback.ts Outdated
Comment thread src/lib/mcp/tools/feedback.ts
@socket-security

socket-security Bot commented Sep 14, 2026

Copy link
Copy Markdown

Review the following changes in direct dependencies. Learn more about Socket for GitHub.

Diff Package Supply Chain
Security
Vulnerability Quality Maintenance License
Addedtldts@​7.4.1310010010095100

View full report

@masnwilliams masnwilliams changed the title Capture structured bot detection feedback Capture config registry recommendation outcomes Sep 14, 2026
Comment thread src/lib/mcp/analytics.ts
Comment thread package.json

@ulziibay-kernel ulziibay-kernel left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@masnwilliams
masnwilliams removed the request for review from rgarcia September 14, 2026 21:06
@masnwilliams
masnwilliams merged commit e3789a1 into main Sep 14, 2026
10 checks passed
@masnwilliams
masnwilliams deleted the hypeship/bot-detection-feedback branch September 14, 2026 23:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants