Rework natural gas pipelines schema - #546
Conversation
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #546 +/- ##
=======================================
Coverage 65.11% 65.12%
=======================================
Files 78 78
Lines 7490 7492 +2
Branches 754 754
=======================================
+ Hits 4877 4879 +2
Misses 2453 2453
Partials 160 160
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
bpulluta
changed the title
There was a problem hiding this comment.
Pull request overview
This PR reworks the natural gas pipelines one-shot extraction plugin to use a decomposed, multi-axis “one row per requirement” schema, expanding the output model to capture richer requirement context while keeping compatibility with COMPASS’s schema-driven one-shot pipeline.
Changes:
- Updated
plugin_config.yamldiscovery queries/keywords and rewrote the extraction system prompt to teach the decomposition model and stricter scope rules. - Replaced the prior flat outputs schema with a decomposed row schema (WHAT / HOW MUCH / HOW BINDING / WHEN-WHERE + evidence), expanding features and adding richer evidence fields.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.
| File | Description |
|---|---|
| compass/extraction/natural_gas_pipelines/plugin_config.yaml | Broadens discovery inputs and updates the extraction system prompt to align LLM behavior with the decomposed row model and scope rules. |
| compass/extraction/natural_gas_pipelines/natural_gas_pipelines_schema.json | Introduces the decomposed multi-axis output schema, expanded feature enum, and richer evidence/normalization guidance. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
| # Many jurisdictions regulate natural gas compressor stations inside broader | ||
| # "oil and gas" or gas-well chapters. Cast a wider net at discovery; the | ||
| # schema scope gate and applies_to field filter out well-only content later. |
Summary
Reworks the natural gas pipelines one-shot extraction plugin so its schema captures decomposed, multi-axis requirement rows instead of the previous flat 5-field
outputs.This is an initial cut to continue developing and track changes.
Schema (
natural_gas_pipelines_schema.json)outputsrow is now decomposed on four axes:feature,specific_subject,applies_to,requirement_descriptionvalue,value_category,value_interpretation,units,range_low/range_highobligationconditionsource_verbatim,summary,section,notes,explanationpermit_approval; folded day/night into a singlenoisefeature +condition, and rear/side yard intoproperty lines distance+specific_subject.$qualitative_features(zoning districts,permit_approval) — previously missing, so every feature routed as quantitative.$scope,$core_principles, grouped$definitions,$instructions,$examples, and$qa_checklistfollowing the freshest sibling (oil_gas_wells) house style.Config (
plugin_config.yaml)heuristic_keywordskeys (house style).extraction_system_promptto teach the decomposition model (axes, one-row-per-requirement, most-restrictive rule, siting-focus exclusions).Validation
gpt-5.4, full search→download→collect→extract→CSV): produces the 25-column decomposed output; correctly extractednoise55 dBA (fully decomposed, withnoteson the absence of separate day/night limits) and twopermit_approvalrows (special permit, site plan approval), routed to the quant/qual CSVs correctly. Compared to the old flat output it dropped both false positives (a site-plan filing radius mislabeled as a setback; an exhaust muffler mislabeled as an enclosure).