feat(intelligence): add regulatory action cross-source signal

[lspassos1]

Summary

This adds CROSS_SOURCE_SIGNAL_TYPE_REGULATORY_ACTION to seed-cross-source-signals.mjs, wiring the new regulatory:actions:v1 feed into cross-source detection with policy categorization, severity scoring, and composite escalation support in Global Markets.

Root cause

The regulatory seeder from #2567 can now publish classified actions, but the cross-source aggregator still ignored that key entirely, so market-moving regulatory actions could not participate in signal ranking or composite escalation.

Changes

• add regulatory:actions:v1 to SOURCE_KEYS
• map CROSS_SOURCE_SIGNAL_TYPE_REGULATORY_ACTION to the policy category and a base weight of 2.0
• add extractRegulatoryAction() to emit recent high/medium-tier signals from the regulatory payload
• keep the signal theater fixed to Global Markets and compute severity as 3.0 -> HIGH for high tier and 2.0 -> MEDIUM for medium
• add tests/cross-source-signals-regulatory.test.mjs covering source registration, extractor behavior, and composite escalation for policy + financial + economic

Validation

• node --test tests/cross-source-signals-regulatory.test.mjs

Risk

Low risk. This is an additive signal source plus a focused unit test; existing extractors and severity logic stay unchanged.

Note

This branch is currently stacked on #2567, which itself depends on #2564. Until those PRs merge, this PR includes the parent commits in the diff.

Depends on #2567
Closes #2494
Refs #2495

[vercel]

@lspassos1 is attempting to deploy a commit to the Elie Team on Vercel.

A member of the Team first needs to authorize it.

[greptile-apps]

Greptile Summary

This PR wires the new regulatory:actions:v1 feed into the cross-source signal aggregator by adding a source key, a policy category mapping, a 2.0 base weight, and an extractRegulatoryAction extractor that emits up to three recent non-low-tier actions as HIGH or MEDIUM severity signals in the Global Markets theater. The changes are purely additive; no existing extractor or scoring logic is touched.

Key observations:

• The extractor follows the established pattern exactly and integrates cleanly with detectCompositeEscalation (as demonstrated by the policy + financial + economic test case).
• runSeed in _seed-utils.mjs already calls writeFreshnessMetadata automatically, so no explicit afterPublish is needed for health-monitoring compliance — this is correct.
• regulatory:actions:v1 is an intermediate aggregation feed (analogous to the other SOURCE_KEYS entries), not a panel data source, so omitting it from api/bootstrap.js follows the existing pattern correctly.
• Severity thresholds are correct: high tier yields score 3.0 (HIGH) and medium yields 2.0 (MEDIUM), both verified by tests.
• Test coverage is extensive: all pure helpers in seed-regulatory-actions.mjs are unit-tested, and extractor + composite escalation behavior is verified via vm sandboxing.

Issues found:

• The FINRA feed URL uses http:// while all four other feeds use https://, creating a potential MITM injection path for a security-relevant signal source.
• buildActionId uses only the calendar date from publishedAt (not the time), so two same-agency same-title articles published on the same day but at different times would get identical IDs, which could cause silent deduplication or overwrites in downstream consumers.
• extractRegulatoryAction implicitly relies on the payload already being sorted newest-first — this invariant holds today but is undocumented and fragile against future payload changes.

Confidence Score: 5/5

• Safe to merge — purely additive change with no existing logic altered and comprehensive test coverage; remaining findings are P2 style/hardening suggestions.
• All three findings are P2: the FINRA HTTP URL is a hardening suggestion (real MITM risk requires network-level compromise of the Railway container), the ID collision requires a very specific same-agency same-title same-date scenario that dedup-by-URL already handles in normal operation, and the sort-order reliance is an implicit invariant that currently holds. No P0 or P1 issues were found. Existing extractors and severity logic are untouched, and test coverage is thorough for both new files.
• scripts/seed-regulatory-actions.mjs — FINRA HTTP URL and buildActionId date-only precision warrant a second look before deploying to production.

Important Files Changed

Filename	Overview
scripts/seed-cross-source-signals.mjs	Adds regulatory:actions:v1 to SOURCE_KEYS, maps the new type to the policy category and 2.0 base weight, and wires in extractRegulatoryAction — all following the established extractor pattern exactly. scoreTier correctly maps high (3.0) → HIGH and medium (2.0) → MEDIUM. One style suggestion: add an explicit sort inside the extractor to document the reliance on payload order.
scripts/seed-regulatory-actions.mjs	New seed script that fetches, parses, deduplicates, and classifies RSS/Atom items from five regulatory agencies. Two issues: (1) FINRA feed uses http:// while all other feeds use https://, creating a MITM injection vector for security signals; (2) buildActionId drops the time component, so same-agency same-title same-date items from two different feed versions would generate identical IDs. Core logic (RSS/Atom parsing, keyword classification, dedup) is sound and well-tested.
tests/cross-source-signals-regulatory.test.mjs	Thorough vm-sandboxed unit tests covering source registration, extractor behavior (high/medium inclusion, low exclusion, 48h staleness cutoff, 3-item cap), and composite escalation for the policy + financial + economic triplet. Assertions are precise and match the implementation exactly.
tests/regulatory-seed-unit.test.mjs	Comprehensive unit tests for every pure function in the seed script: entity decoding, HTML stripping, RSS and Atom parsing, link canonicalization, deduplication, keyword classification, payload building, and the main wiring. fetchAllFeeds partial-failure behavior is also covered. No gaps found.

Sequence Diagram

sequenceDiagram
    participant Cron as Railway Cron
    participant RegSeed as seed-regulatory-actions.mjs
    participant Feeds as Regulatory Feeds<br/>(SEC, CFTC, Fed, FDIC, FINRA)
    participant Redis as Upstash Redis
    participant XSeed as seed-cross-source-signals.mjs
    participant Client as API / Bootstrap Client

    Cron->>RegSeed: trigger (every ~2h)
    RegSeed->>Feeds: fetch RSS/Atom (parallel, Promise.allSettled)
    Feeds-->>RegSeed: XML responses
    RegSeed->>RegSeed: parseFeed → normalizeFeedItems → dedupeAndSortActions
    RegSeed->>RegSeed: classifyAction (high / medium / low)
    RegSeed->>Redis: SET regulatory:actions:v1 (TTL 7200s)
    RegSeed->>Redis: SET seed-meta:regulatory:actions (via runSeed)

    Cron->>XSeed: trigger (every ~15min)
    XSeed->>Redis: pipeline GET SOURCE_KEYS (incl. regulatory:actions:v1)
    Redis-->>XSeed: all source payloads
    XSeed->>XSeed: extractRegulatoryAction → filter(48h, tier≠low) → slice(0,3)
    XSeed->>XSeed: score: high→3.0 HIGH, medium→2.0 MEDIUM
    XSeed->>XSeed: detectCompositeEscalation (policy + financial + economic → COMPOSITE)
    XSeed->>Redis: SET intelligence:cross-source-signals:v1 (TTL 1800s)

    Client->>Redis: GET intelligence:cross-source-signals:v1 (via bootstrap)
    Redis-->>Client: signals incl. REGULATORY_ACTION + potential COMPOSITE_ESCALATION

Loading

_{Reviews (1): Last reviewed commit: "feat(intelligence): add regulatory actio..." | Re-trigger Greptile}