Skip to content

Refactor URL sanitization: shared helper, space-only encoding, robust test - #401

Merged
bpulluta merged 4 commits into
feature/geothermal-extraction-prfrom
copilot/sub-pr-400
Mar 21, 2026
Merged

Refactor URL sanitization: shared helper, space-only encoding, robust test#401
bpulluta merged 4 commits into
feature/geothermal-extraction-prfrom
copilot/sub-pr-400

Conversation

Copilot AI commented Mar 21, 2026

Copy link
Copy Markdown
Contributor

Three reviewer feedback items from the geothermal extraction PR addressed:

  • Shared _sanitize_url — extracted to compass/web/url_utils.py; both compass/scripts/download.py and compass/web/website_crawl.py import from there, eliminating duplicated logic that could drift.

  • Simplified encoding — replaced quote(unquote(path), safe="/") + query re-encoding with a targeted path.replace(" ", "%20"). The original approach could corrupt presigned/SAS URLs or alter semantically significant query parameters. Query string and existing percent-encoding are now left untouched.

# before — full unquote/requote round-trip, query string re-encoded
safe_path = quote(unquote(parsed.path), safe="/")
safe_query = urlencode(parse_qsl(parsed.query, keep_blank_values=True), doseq=True)

# after — only fixes literal spaces in path, everything else preserved
safe_path = path.replace(" ", "%20") if " " in path else path
# parsed.query passed through unchanged
  • Test robustness — added assert len(links) == 1 before next(iter(links)) in test_extract_links_from_html_sets_text_from_anchor to avoid silent false-passes if _extract_links_from_html returns an unexpected number of results.

✨ Let Copilot coding agent set things up for you — coding agent works faster and does higher quality work when set up for your repo.

…encoding, fix test robustness

Co-authored-by: bpulluta <115118857+bpulluta@users.noreply.github.com>
Agent-Logs-Url: https://github.com/NatLabRockies/COMPASS/sessions/ceb782b4-c312-41d1-b4eb-eccbbef67097
Copilot AI changed the title [WIP] Add geothermal electricity extraction support Refactor URL sanitization: shared helper, space-only encoding, robust test Mar 21, 2026
Copilot AI requested a review from bpulluta March 21, 2026 04:11
@bpulluta
bpulluta self-requested a review March 21, 2026 04:38
@codecov-commenter

codecov-commenter commented Mar 21, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 55.53%. Comparing base (bac918c) to head (6c6e950).
⚠️ Report is 1 commits behind head on feature/geothermal-extraction-pr.

Additional details and impacted files
@@                         Coverage Diff                          @@
##           feature/geothermal-extraction-pr     #401      +/-   ##
====================================================================
+ Coverage                             55.44%   55.53%   +0.09%     
====================================================================
  Files                                    61       62       +1     
  Lines                                  5699     5695       -4     
  Branches                                534      534              
====================================================================
+ Hits                                   3160     3163       +3     
+ Misses                                 2491     2485       -6     
+ Partials                                 48       47       -1     
Flag Coverage Δ
unittests 55.53% <100.00%> (+0.09%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@bpulluta
bpulluta marked this pull request as ready for review March 21, 2026 05:19
@bpulluta
bpulluta merged commit c6eb335 into feature/geothermal-extraction-pr Mar 21, 2026
14 checks passed
@bpulluta
bpulluta deleted the copilot/sub-pr-400 branch March 21, 2026 05:19
ppinchuk added a commit that referenced this pull request May 21, 2026
* Add geothermal extraction integration artifacts

* Add COMPASS workflow skills

* Added one-shot skills

* update one-shot SKILL.md structure and trigger contracts

* Initial plan (#398)

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>

* Fix skills documentation: correct paths, caching behavior, and tab formatting (#399)

* Initial plan

* Fix all review comments in skills documentation

Co-authored-by: bpulluta <115118857+bpulluta@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: bpulluta <115118857+bpulluta@users.noreply.github.com>

* renamed skills and fixed minor comments

* fix URL space encoding and anchor text scoring in website crawl retrieval

- percent-encode raw spaces in crawl4ai PDF source URLs before downstream use
- populate link text field from anchor text so ELMLinkScorer can score link labels
- add two regression tests covering both fixes

* remove testing skills

* fixed schema

* Refactor URL sanitization: shared helper, space-only encoding, robust test (#401)

* Initial plan

* Extract shared _sanitize_url to url_utils.py, simplify to space-only encoding, fix test robustness

Co-authored-by: bpulluta <115118857+bpulluta@users.noreply.github.com>
Agent-Logs-Url: https://github.com/NatLabRockies/COMPASS/sessions/ceb782b4-c312-41d1-b4eb-eccbbef67097

* fix failing test

* ruff error fix

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: bpulluta <115118857+bpulluta@users.noreply.github.com>

* Bump release-drafter/release-drafter from 7.0.0 to 7.1.1 (#402)

Bumps [release-drafter/release-drafter](https://github.com/release-drafter/release-drafter) from 7.0.0 to 7.1.1.
- [Release notes](https://github.com/release-drafter/release-drafter/releases)
- [Commits](release-drafter/release-drafter@3a7fb5c...139054a)

---
updated-dependencies:
- dependency-name: release-drafter/release-drafter
  dependency-version: 7.1.1
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Add One-Shot Skills Reliability and Guardrails (#397)

* Add COMPASS workflow skills

* Added one-shot skills

* update one-shot SKILL.md structure and trigger contracts

* Initial plan (#398)

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>

* Fix skills documentation: correct paths, caching behavior, and tab formatting (#399)

* Initial plan

* Fix all review comments in skills documentation

Co-authored-by: bpulluta <115118857+bpulluta@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: bpulluta <115118857+bpulluta@users.noreply.github.com>

* renamed skills and fixed minor comments

* udpated skills Paul review march 26

---------

Co-authored-by: Copilot <198982749+Copilot@users.noreply.github.com>

* updated schema, added cli overwrite option, and added optional post process rules

* fix ruff error in components.py

* ruff check on components.py

* fix ruff errors in process.py

* ruff check on updated files

* Bump codecov/codecov-action from 5 to 6 (#403)

Bumps [codecov/codecov-action](https://github.com/codecov/codecov-action) from 5 to 6.
- [Release notes](https://github.com/codecov/codecov-action/releases)
- [Changelog](https://github.com/codecov/codecov-action/blob/main/CHANGELOG.md)
- [Commits](codecov/codecov-action@v5...v6)

---
updated-dependencies:
- dependency-name: codecov/codecov-action
  dependency-version: '6'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Update tox file

* Update tox file

* resolved pending comments in PR400. Cleaned up logic.

* fix: Wrapped long docstring lines for ruff

* updated YAML

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: Copilot <198982749+Copilot@users.noreply.github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: ppinchuk <pinchukpaul@gmail.com>
rajeee pushed a commit that referenced this pull request May 27, 2026
* Add geothermal extraction integration artifacts

* Add COMPASS workflow skills

* Added one-shot skills

* update one-shot SKILL.md structure and trigger contracts

* Initial plan (#398)

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>

* Fix skills documentation: correct paths, caching behavior, and tab formatting (#399)

* Initial plan

* Fix all review comments in skills documentation

Co-authored-by: bpulluta <115118857+bpulluta@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: bpulluta <115118857+bpulluta@users.noreply.github.com>

* renamed skills and fixed minor comments

* fix URL space encoding and anchor text scoring in website crawl retrieval

- percent-encode raw spaces in crawl4ai PDF source URLs before downstream use
- populate link text field from anchor text so ELMLinkScorer can score link labels
- add two regression tests covering both fixes

* remove testing skills

* fixed schema

* Refactor URL sanitization: shared helper, space-only encoding, robust test (#401)

* Initial plan

* Extract shared _sanitize_url to url_utils.py, simplify to space-only encoding, fix test robustness

Co-authored-by: bpulluta <115118857+bpulluta@users.noreply.github.com>
Agent-Logs-Url: https://github.com/NatLabRockies/COMPASS/sessions/ceb782b4-c312-41d1-b4eb-eccbbef67097

* fix failing test

* ruff error fix

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: bpulluta <115118857+bpulluta@users.noreply.github.com>

* Bump release-drafter/release-drafter from 7.0.0 to 7.1.1 (#402)

Bumps [release-drafter/release-drafter](https://github.com/release-drafter/release-drafter) from 7.0.0 to 7.1.1.
- [Release notes](https://github.com/release-drafter/release-drafter/releases)
- [Commits](release-drafter/release-drafter@3a7fb5c...139054a)

---
updated-dependencies:
- dependency-name: release-drafter/release-drafter
  dependency-version: 7.1.1
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Add One-Shot Skills Reliability and Guardrails (#397)

* Add COMPASS workflow skills

* Added one-shot skills

* update one-shot SKILL.md structure and trigger contracts

* Initial plan (#398)

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>

* Fix skills documentation: correct paths, caching behavior, and tab formatting (#399)

* Initial plan

* Fix all review comments in skills documentation

Co-authored-by: bpulluta <115118857+bpulluta@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: bpulluta <115118857+bpulluta@users.noreply.github.com>

* renamed skills and fixed minor comments

* udpated skills Paul review march 26

---------

Co-authored-by: Copilot <198982749+Copilot@users.noreply.github.com>

* updated schema, added cli overwrite option, and added optional post process rules

* fix ruff error in components.py

* ruff check on components.py

* fix ruff errors in process.py

* ruff check on updated files

* Bump codecov/codecov-action from 5 to 6 (#403)

Bumps [codecov/codecov-action](https://github.com/codecov/codecov-action) from 5 to 6.
- [Release notes](https://github.com/codecov/codecov-action/releases)
- [Changelog](https://github.com/codecov/codecov-action/blob/main/CHANGELOG.md)
- [Commits](codecov/codecov-action@v5...v6)

---
updated-dependencies:
- dependency-name: codecov/codecov-action
  dependency-version: '6'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Update tox file

* Update tox file

* resolved pending comments in PR400. Cleaned up logic.

* fix: Wrapped long docstring lines for ruff

* updated YAML

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: Copilot <198982749+Copilot@users.noreply.github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: ppinchuk <pinchukpaul@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants