Skip to content

feat: add --feedback flag for search result quality feedback loop (#604) - #611

Closed
lb1192176991-lab wants to merge 1 commit into
Ikalus1988:mainfrom
lb1192176991-lab:feat/search-feedback-604
Closed

feat: add --feedback flag for search result quality feedback loop (#604)#611
lb1192176991-lab wants to merge 1 commit into
Ikalus1988:mainfrom
lb1192176991-lab:feat/search-feedback-604

Conversation

@lb1192176991-lab

Copy link
Copy Markdown
Contributor

What

Adds a --feedback flag to search_knowledge.py so users/agents can report whether search results were useful.

  • After displaying results, prompts: Was this helpful? (y/n/c=comment/Enter=skip)
  • Logs feedback to data/search-feedback.jsonl with: query, found_results, feedback, timestamp
  • Supports: helpful, not_helpful, comment modes
  • Gracefully handles non-interactive environments (EOFError/KeyboardInterrupt)

Why

This feeds into #574 (feedback hub) and #591 (demand board) — enabling systematic tracking of search result quality and identification of knowledge gaps.

Testing

# With feedback prompt
python3 search_knowledge.py "pip install timeout" --feedback

# Without feedback (unchanged behavior)
python3 search_knowledge.py "pip install timeout"

# No results case
python3 search_knowledge.py "nonexistent_xyz_123" --feedback
  • Verified data/search-feedback.jsonl contains valid JSONL entries
  • No PII in logs (only query + result status + feedback)
  • Existing behavior unchanged when --feedback not used

@Ikalus1988 Ikalus1988 added needs-dco Missing Signed-off-by in commit risk:high High risk change, needs maintainer review shape-safe labels Jul 27, 2026
@Ikalus1988

Copy link
Copy Markdown
Owner

This PR competes with #607 and #622 for the same feature (#604). Only one will be merged — first clean + scoped + tested + DCO-passing wins.

@lb1192176991-lab

Copy link
Copy Markdown
Contributor Author

👋 Still actively working on this PR!

Acknowledging the competition with #607 and #622 for the same feature (#604). I've reviewed all three implementations and here's where we stand:

Current blockers on #611 (shared with competitors):

  • DCO check — the DCO workflow itself is failing with Node.js 20 deprecation errors on the runner. I'll fix this by adding Signed-off-by to the commit. Could you confirm whether the DCO infra issue is also being addressed on the repo side?
  • Audit check — also failing due to Node.js 20 deprecation in actions.

What makes #611 competitive:

  • ✅ Clean single commit with no merge history
  • ✅ More flexible feedback UX: y/n/c=comment/Enter=skip with structured logging
  • ✅ Graceful handling of non-interactive environments (EOFError/KeyboardInterrupt)
  • ✅ Proper --feedback flag parsing integrated into the existing arg loop
  • audit-shape and isolate checks passing

Next steps I'm planning:

  1. Add Signed-off-by to the commit to resolve DCO
  2. Consider incorporating result_ids into feedback logs for richer tracking
  3. Keep the scope tight — only search_knowledge.py modified, no external deps

Is there any specific area you'd like me to focus on to make this the winning PR? Happy to iterate.

cc @Ikalus1988

@Ikalus1988

Copy link
Copy Markdown
Owner

Thanks for the detailed update. To answer your questions:

  1. DCO infra — the DCO check itself works fine; the Node.js deprecation warnings are cosmetic and don't block the check. The real blocker is missing Signed-off-by: in commits. Adding it will pass DCO.

  2. What to focus on — this PR is competing with feat: add --feedback flag + search-feedback.jsonl (#604) #607 and feat(search): add --feedback flag for search result quality feedback loop #622 for [Search][Bounty] Add search result quality feedback loop to search_knowledge.py #604. Current ranking:

    To win: fix DCO first, then keep scope tight. Your y/n/c=comment/Enter=skip UX is a nice touch — keep it.

Fix DCO:

git commit --amend --signoff
git push --force-with-lease

- Add --feedback flag to search_knowledge.py
- After displaying results, prompt: 'Was this helpful? (y/n/c=comment)'
- Log feedback to data/search-feedback.jsonl
- No PII in logs (only query + result info + feedback)
- No external dependencies
- Existing search behavior unchanged when --feedback not used

Closes Ikalus1988#604

Signed-off-by: lb1192176991-lab <lb1192176991-lab@users.noreply.github.com>
@lb1192176991-lab

Copy link
Copy Markdown
Contributor Author

@Ikalus1988 DCO fixed! Rebased onto upstream/main, resolved conflict in search_knowledge.py (preserved both _collect_feedback and _prompt_feedback), and added Signed-off-by to the commit. Please re-check the DCO check when available. Thanks!

@codecov

codecov Bot commented Jul 29, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

@Ikalus1988

Copy link
Copy Markdown
Owner

Closing — this work has been superseded by main branch implementation (v2.13.0/v2.14.0). The intake endpoint, feedback flag, and triage pipeline are already in place. Thank you for the contribution.

@Ikalus1988 Ikalus1988 closed this Jul 29, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

needs-dco Missing Signed-off-by in commit risk:high High risk change, needs maintainer review shape-safe

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants