Promote candidate lote2 changes to main (AUD-007/008/009/010)

[nsalvacao]

Summary

Promote lote2 validated changes from candidate to main.

Included work:

• AUD-007: canonicalize legacy command surface and raw flag compatibility behavior
• AUD-008: reconcile spec/task drift for release-tracking reality
• AUD-009: add reproducible packaging release gates in CI (build + artifact smoke)
• AUD-010: define canonical docs routing for first-time users

Hygiene and safety

• All worker branches were isolated with non-overlapping file scopes.
• Worker branches were merged into candidate and then cleaned up (local + remote).
• Candidate branch verified against main before promotion.

Validation highlights

• Targeted tests for compatibility behavior
• CI workflow updates for packaging gate checks
• Docs routing consistency and link sanity in touched sections

[chatgpt-codex-connector]

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.
To continue using code reviews, you can upgrade your account or add credits to your account and enable them for code reviews in your settings.

[github-actions]

🔍 AI Code Review

🔵 CI Workflow (.github/workflows/ci.yml)

• Added candidate/** branches to trigger CI on push and PR events.
  • Good for testing feature branches before merging.
• Added package-release-gates job to build sdist and wheel, validate artifacts, and run smoke tests by importing modules and invoking CLI help commands.
  • Good practice to catch packaging and runtime issues early.
• Uses fixed SOURCE_DATE_EPOCH for reproducible builds.
• Uses Python 3.12, latest actions versions.
• Uses set -euo pipefail and shopt -s nullglob for robust shell scripting.
• Smoke tests install wheel in isolated venv and run import and CLI help commands.

🟡 Suggestions:

• Consider caching pip packages to speed up CI.
• Validate the candidate/** branch pattern matches your branch naming conventions.
• Add explicit error messages in shell scripts for easier debugging if tests fail.

---

🔵 cli_crawler.py

• The entire original CLI crawler script is replaced with a minimal legacy compatibility wrapper that imports and calls main() from crawler.cli_crawler.
• The old code is removed and replaced with a docstring recommending to use the canonical command cli-crawler <cli_name> [options].
• This reduces code duplication and centralizes CLI logic in crawler.cli_crawler.
• The wrapper still supports if __name__ == "__main__": main() for backward compatibility.

🟡 Suggestions:

• Ensure crawler.cli_crawler.main() is well tested and fully replaces the removed functionality.
• Update documentation and user guides to reflect the new recommended usage.
• Remove any unused imports or files related to the old CLI script to avoid confusion.

---

🔴 Critical

• None found. No security issues or critical bugs detected.

🟡 Warnings

• None critical, but consider adding error handling and caching in CI scripts for robustness and speed.

🔵 Info

• Good CI practices with build validation and smoke tests.
• Clean refactor of CLI wrapper to reduce duplication.

---

🤖 gpt-4.1-mini · 2120 tokens · GitHub Models free tier · 0 premium requests

[gemini-code-assist]

Summary of Changes

Hello, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request promotes a set of validated changes from the lote2 candidate branch to the main branch. The primary focus is on enhancing the project's usability and maintainability by streamlining documentation access, standardizing CLI argument handling, and ensuring development tasks are accurately tracked. These updates contribute to a more robust and user-friendly experience, both for developers and end-users.

Highlights

• Documentation Routing (AUD-010): Established a canonical documentation path for first-time users, starting with the root README.md and guiding them to deeper documentation.
• CLI Compatibility (AUD-007): Canonicalized the legacy cli_crawler.py entrypoint, ensuring consistent behavior for raw flag compatibility and providing warnings for deprecated usage.
• Specification Alignment (AUD-008): Updated the project's task list (tasks.md) to reflect the current state of development, including completed tasks and dependencies, aligning with release-tracking realities.
• CI Enhancements (AUD-009): Added reproducible packaging release gates in CI, enhancing the build and artifact smoke testing processes.
• Validation Process: Confirmed the changes through targeted compatibility tests, CI workflow updates for packaging, and consistency checks for documentation routing.

Changelog

• README.md
  • Added a "Start Here" section to guide new users through the project's documentation.
• cli_crawler.py
  • Transformed the script into a compatibility wrapper, delegating its core functionality to crawler.cli_crawler.main.
  • Implemented logic to normalize legacy --include-raw flag to --raw and issue deprecation warnings.
• docs/README.md
  • Replaced the detailed project overview with a concise documentation routing guide, directing users to the main README.md.
• specs/001-cli-plugins-base/tasks.md
  • Added a comprehensive task list document, outlining project phases, user stories, and individual development tasks with their completion status.
• src/crawler/cli_crawler.py
  • Updated the module to include functions for normalizing legacy command-line arguments and providing warnings for deprecated usage, ensuring backward compatibility.
• tests/unit/test_cli_crawler_compat.py
  • Introduced new unit tests to validate the argument normalization and warning mechanisms within the cli_crawler compatibility layer.

Ignored Files

• Ignored by pattern: .github/workflows/** (1)
  • .github/workflows/ci.yml

Activity

• The lote2 candidate branch, containing these validated changes, was promoted to main.
• Worker branches were isolated with non-overlapping file scopes, merged into candidate, and then cleaned up.
• The candidate branch was verified against the main branch prior to this promotion.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[gemini-code-assist]

Code Review

The pull request effectively promotes the lote2 changes to main, addressing the specified AUD tasks. The documentation updates in README.md and docs/README.md significantly improve the onboarding experience and documentation routing for new users. The refactoring of cli_crawler.py into a compatibility wrapper and moving the core logic to src/crawler/cli_crawler.py is well-executed, including robust handling of legacy arguments and sys.argv restoration. The new specs/001-cli-plugins-base/tasks.md provides a comprehensive and well-structured task list. All changes are clear, well-implemented, and align with the pull request's objectives. I found no issues requiring specific review comments at the medium, high, or critical severity levels.

[Copilot]

Pull request overview

Promotes the validated “candidate/lote2” changes into main, focusing on legacy CLI compatibility, aligning specs/tasks with current reality, adding CI packaging release gates, and clarifying documentation routing for first-time users.

Changes:

• Added legacy cli_crawler.py → cli-crawler compatibility behavior (including --include-raw → --raw) plus unit tests.
• Introduced CI “package release gates” job to build artifacts and smoke-check installed entrypoints/imports.
• Simplified docs/README.md into a routing/index page and added a canonical docs path section to the root README.

Reviewed changes

Copilot reviewed 7 out of 7 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
tests/unit/test_cli_crawler_compat.py	Adds unit tests covering legacy flag normalization and warning output.
src/crawler/cli_crawler.py	Implements legacy-entrypoint warning + legacy arg normalization and delegates to crawler.pipeline.main.
specs/001-cli-plugins-base/tasks.md	Adds a detailed task breakdown document for the base specs.
docs/README.md	Replaces long-form doc with a short “documentation routing” index.
cli_crawler.py	Converts the legacy root script into a thin wrapper around crawler.cli_crawler.main.
README.md	Adds a “Start Here” canonical docs routing section.
.github/workflows/ci.yml	Expands branch triggers and adds a packaging build + artifact smoke test job.

---

You can also share your feedback on Copilot code review. Take the survey.

[github-actions]

🔍 AI Code Review

🔵 CI Workflow (.github/workflows/ci.yml)

• Added candidate/** branch pattern to trigger CI on push and PR events.

  • Good for testing feature branches.
  • Ensure this does not cause excessive CI runs or leak sensitive info.

• New job package-release-gates:

  • Builds sdist and wheel with fixed SOURCE_DATE_EPOCH for reproducibility.
  • Validates exactly one sdist and one wheel artifact exist.
  • Smoke tests installation and basic CLI commands.
  • Best practice: Good to test artifacts before release.
  • Suggestion: Consider caching pip packages to speed up CI.
  • Security: No secrets exposed; safe.

🟡 cli_crawler.py

• The entire CLI crawler logic is removed and replaced with a legacy stub that imports main from crawler.cli_crawler.
• This is a breaking change for users relying on this script directly.
• Warning: If users/scripts depend on the old CLI interface, they will break.
• Suggestion: Document this change clearly in release notes.
• Security: No direct security issues, but ensure the new crawler.cli_crawler.main is secure and properly handles inputs.

🔵 General

• No obvious security vulnerabilities or bugs introduced.
• Code cleanup and CI improvements are positive.
• Ensure all dependencies (e.g., ruff, build) are pinned or audited regularly.

---

Summary:
The PR improves CI by adding release artifact validation and smoke tests, which is a strong best practice. The CLI crawler script is deprecated in favor of a new canonical command, which may break backward compatibility—this should be clearly communicated. No critical security issues detected.

---

🤖 gpt-4.1-mini · 2028 tokens · GitHub Models free tier · 0 premium requests