Skip to content

feat(climate): add WMO normals seeding and CO2 monitoring#2531

Merged
koala73 merged 4 commits intomainfrom
feat/greenhouse-gas-monitoring
Apr 2, 2026
Merged

feat(climate): add WMO normals seeding and CO2 monitoring#2531
koala73 merged 4 commits intomainfrom
feat/greenhouse-gas-monitoring

Conversation

@FayezBast
Copy link
Copy Markdown
Collaborator

Summary

Fixes climate anomaly seeding to use WMO-style 1991-2020 monthly normals instead of a rolling 30-day self-baseline. Adds a new seed-climate-zone-normals.mjs pipeline, wires required climate-specific zones and Redis cache/health registration, and updates climate anomaly fetching to read climate:zone-normals:v1 as the baseline.

Also adds climate CO2 monitoring seed/API support and regenerates the related climate service contracts and OpenAPI outputs.

Fixes #2467
Fixes #2468

Type of change

  • Bug fix
  • New feature
  • New data source / feed

Affected areas

  • Map / Globe
  • API endpoints (/api/*)
  • Other: climate seed scripts, Redis cache wiring, climate frontend service, generated climate API contracts

Checklist

  • No API keys or secrets committed

Additional verification:

  • node --test tests/climate-seeds.test.mjs
  • node --test tests/bootstrap.test.mjs

@mintlify
Copy link
Copy Markdown

mintlify bot commented Mar 30, 2026

Preview deployment for your docs. Learn more about Mintlify Previews.

Project Status Preview Updated (UTC)
WorldMonitor 🟢 Ready View Preview Mar 30, 2026, 12:11 AM

@vercel
Copy link
Copy Markdown

vercel bot commented Mar 30, 2026

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
worldmonitor Ready Ready Preview, Comment Apr 1, 2026 5:16pm

Request Review

@greptile-apps
Copy link
Copy Markdown

greptile-apps bot commented Mar 30, 2026

Greptile Summary

This PR replaces the rolling 30-day self-baseline anomaly algorithm with WMO-style 1991-2020 monthly normals, introduces a new seed-climate-zone-normals.mjs pipeline that pre-computes per-zone monthly temperature/precipitation means from Open-Meteo ERA5 archive data, and adds a full CO2/CH4/N2O monitoring feed sourced from NOAA GML. The overall architecture is solid — batched Open-Meteo calls with exponential backoff, upfront normals validation before the anomaly seed runs, a well-tested NOAA parsing layer, and proper circuit-breaker wiring on the client side.

Key issues:

  • climateZoneNormals is registered in both BOOTSTRAP_KEYS and STANDALONE_KEYS in health.js, and also in BOOTSTRAP_CACHE_KEYS / BOOTSTRAP_TIERS in bootstrap.js and cache-keys.ts. This causes the internal normals baseline to be included in every browser bootstrap response despite having no frontend consumer, and double-health-checked. Zone normals are a backend-only seed input and should live exclusively in STANDALONE_KEYS.
  • buildClimateAnomalyFromResponse throws on a missing monthly normal, silently discarding all other zones in the same batch (up to 8) rather than just the missing one.
  • The final throw after the retry loop in _open-meteo-archive.mjs is dead code.
  • CO2_MONITORING_KEY is not exported from server/_shared/cache-keys.ts even though CLIMATE_ZONE_NORMALS_KEY was added there; the handler hardcodes the string inline.

Confidence Score: 4/5

Safe to merge after resolving the climateZoneNormals double-registration — it currently sends unneeded normals data to every browser client on page load and double-counts the key in health checks.

One P1 issue (climateZoneNormals in both BOOTSTRAP_KEYS and STANDALONE_KEYS) means every bootstrap response carries the normals payload to clients that cannot use it, and health.js reports the key twice. All other findings are P2 (dead code, missing constant, batch failure granularity). The core logic is well-implemented and well-tested.

api/health.js and api/bootstrap.js (climateZoneNormals double-registration), server/_shared/cache-keys.ts (missing CO2_MONITORING_KEY constant)

Important Files Changed

Filename Overview
api/health.js climateZoneNormals registered in both BOOTSTRAP_KEYS and STANDALONE_KEYS — same Redis key health-checked twice, and bootstrap will serve the normals payload to every browser client despite no current frontend consumer.
api/bootstrap.js Adds climateZoneNormals and co2Monitoring to BOOTSTRAP_CACHE_KEYS and SLOW_KEYS; co2Monitoring wiring is clean, but climateZoneNormals in BOOTSTRAP_CACHE_KEYS sends internal baseline data to every client unnecessarily.
scripts/seed-climate-zone-normals.mjs New seed script computing WMO 1991-2020 monthly normals from Open-Meteo archive; solid batch retry logic, validation, and export surface. A throw-on-incomplete-months path will lose both zones in a batch rather than just the failing one.
scripts/seed-co2-monitoring.mjs New NOAA GML ingestion script; column indices are correct for daily (col 4), monthly (col 3), annual (col 1), and CH4/N2O (col 3) formats. Parsing, validation, trend-12m construction, and fallbacks are well-handled.
scripts/seed-climate-anomalies.mjs Refactored to use WMO normals baseline instead of rolling 30-day self-baseline; unit change from percentage to mm/day is reflected throughout; recalibrated PRECIP_* constants look appropriate.
scripts/_open-meteo-archive.mjs New shared Open-Meteo batch fetch helper with exponential backoff; retry loop always exits via return/throw inside the body, making the final throw after the loop unreachable dead code.
server/_shared/cache-keys.ts Adds CLIMATE_ZONE_NORMALS_KEY constant and climateZoneNormals/co2Monitoring to BOOTSTRAP_CACHE_KEYS and BOOTSTRAP_TIERS; CO2_MONITORING_KEY is not exported as a constant, minor inconsistency.
server/worldmonitor/climate/v1/get-co2-monitoring.ts New handler reads co2-monitoring from Redis cache; correctly returns empty object on miss or error, consistent with other cache-backed handlers.
src/services/climate/index.ts Adds fetchCo2Monitoring with circuit-breaker, hydration fallback, and measuredAt int64-string-to-Date conversion; Co2Monitoring interface is well-typed and consistent with the generated proto types.
tests/climate-seeds.test.mjs New comprehensive unit tests cover normals aggregation, batch mapping, anomaly classification thresholds, and full NOAA parsing pipeline; edge cases like missing values are implicitly tested.

Sequence Diagram

sequenceDiagram
    participant Cron as Railway Cron
    participant NormalsSeed as seed-climate-zone-normals.mjs
    participant AnomalySeed as seed-climate-anomalies.mjs
    participant CO2Seed as seed-co2-monitoring.mjs
    participant OpenMeteo as Open-Meteo Archive API
    participant NOAA as NOAA GML API
    participant Redis as Redis Cache

    Note over Cron,Redis: Monthly (1st of month)
    Cron->>NormalsSeed: Run (monthly cron)
    NormalsSeed->>OpenMeteo: Batch fetch 1991-2020 daily data (22 zones, batch=2)
    OpenMeteo-->>NormalsSeed: Daily temp+precip arrays
    NormalsSeed->>NormalsSeed: computeMonthlyNormals() per zone
    NormalsSeed->>Redis: SET climate:zone-normals:v1 (TTL 90d)

    Note over Cron,Redis: Every 2 hours
    Cron->>AnomalySeed: Run
    AnomalySeed->>Redis: GET climate:zone-normals:v1
    Redis-->>AnomalySeed: WMO 1991-2020 normals index
    AnomalySeed->>OpenMeteo: Batch fetch last 21 days (22 zones, batch=8)
    OpenMeteo-->>AnomalySeed: Daily temp+precip arrays
    AnomalySeed->>AnomalySeed: buildClimateAnomaly() vs monthly normal
    AnomalySeed->>Redis: SET climate:anomalies:v1 (TTL 3h)

    Note over Cron,Redis: Daily (06:00 UTC)
    Cron->>CO2Seed: Run
    CO2Seed->>NOAA: Parallel fetch daily/monthly/annual CO2 + CH4 + N2O
    NOAA-->>CO2Seed: NOAA GML text files
    CO2Seed->>CO2Seed: buildCo2MonitoringPayload()
    CO2Seed->>Redis: SET climate:co2-monitoring:v1 (TTL 24h)

    participant Client as Browser / MCP
    Client->>+Server: GET /api/climate/v1/get-co2-monitoring
    Server->>Redis: GET climate:co2-monitoring:v1
    Redis-->>Server: Co2Monitoring payload
    Server-->>-Client: GetCo2MonitoringResponse
Loading

Reviews (1): Last reviewed commit: "feat(climate): add WMO normals seeding a..." | Re-trigger Greptile

Comment on lines 86 to 87
bisPolicy: 'economic:bis:policy:v1',
bisExchange: 'economic:bis:eer:v1',
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 climateZoneNormals double-registered: BOOTSTRAP_KEYS + STANDALONE_KEYS

climateZoneNormals is being registered in both BOOTSTRAP_KEYS (line ~32) and STANDALONE_KEYS (line ~87). This causes two problems:

  1. Duplicate health check — the health module iterates both maps independently, so climate:zone-normals:v1 will be checked twice in every health report, potentially double-counting failures.

  2. Bootstrap response inflation — because the key is also in BOOTSTRAP_KEYS (and BOOTSTRAP_CACHE_KEYS in bootstrap.js), the full normals payload (22 zones × 12 monthly normals) is included in every bootstrap response sent to browser clients, even though there is no frontend consumer (climateZoneNormals is explicitly in PENDING_CONSUMERS in bootstrap.test.mjs).

Since zone normals are a backend-only baseline consumed directly by seed scripts via Redis, they have no business being in the bootstrap pipeline at all. The key should live only in STANDALONE_KEYS, and climateZoneNormals should be removed from BOOTSTRAP_KEYS in this file, from BOOTSTRAP_CACHE_KEYS in api/bootstrap.js, and from BOOTSTRAP_CACHE_KEYS / BOOTSTRAP_TIERS in server/_shared/cache-keys.ts.

Comment on lines +73 to +75

throw new Error(`Open-Meteo ${resp.status} for ${label}`);
}
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Unreachable dead code after retry loop

The throw on line 75 (Open-Meteo retries exhausted for ...) is never reached. Every path through the loop body terminates with either a return (on resp.ok) or a throw (on any non-OK, non-retried response), so the loop never exits naturally via its termination condition. The only way this line could be reached is if maxRetries < 0, which is not a supported input.

Consider removing it to avoid misleading future readers who might expect this to be the canonical exhausted-retries message.

Comment on lines +120 to 124
const normalsIndex = indexZoneNormals(normalsPayload);

const endDate = toIsoDate(new Date());
const startDate = toIsoDate(new Date(Date.now() - 21 * 24 * 60 * 60 * 1000));

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Missing monthly normal throws for entire batch, not just the offending zone

buildClimateAnomalyFromResponse throws (rather than returning null) when a zone's monthly normal is absent from the index. Because buildClimateAnomaliesFromBatch calls this inside .map() synchronously, the throw propagates to the try/catch in fetchClimateAnomalies, which counts batch.length (up to 8) failures instead of just 1. If a zone is ever added to CLIMATE_ZONES before the normals seed re-runs, every other zone in the same batch is silently discarded. Returning null and letting the existing .filter(anomaly => anomaly != null) handle the drop would be safer.

*/
export const SIMULATION_OUTCOME_LATEST_KEY = 'forecast:simulation-outcome:latest';
export const SIMULATION_PACKAGE_LATEST_KEY = 'forecast:simulation-package:latest';
export const CLIMATE_ZONE_NORMALS_KEY = 'climate:zone-normals:v1';
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Missing CO2_MONITORING_KEY constant — inconsistent with CLIMATE_ZONE_NORMALS_KEY

CLIMATE_ZONE_NORMALS_KEY was exported as a named constant here, but server/worldmonitor/climate/v1/get-co2-monitoring.ts hardcodes 'climate:co2-monitoring:v1' directly rather than importing a shared constant. If the key ever needs to be versioned, there is now one canonical source for zone-normals but not for CO2 monitoring. Consider adding an exported CO2_MONITORING_KEY constant alongside CLIMATE_ZONE_NORMALS_KEY and updating the handler to import it.

Copy link
Copy Markdown
Owner

@koala73 koala73 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the thorough work here — this is significantly more complete than the other open attempt at the same issue, and the architecture is noticeably better.

What's done well

  • Shared modules (_climate-zones.mjs, _open-meteo-archive.mjs) eliminate the duplication problem from the other PR
  • Batch API calls instead of one request per zone per year — far more efficient and friendlier to Open-Meteo
  • isMain guard on all three seeders — correct pattern, allows test imports without side effects
  • Full 4-file checklist covered: cache-keys.ts, api/bootstrap.js, api/health.js, api/seed-health.js
  • Test coverage in tests/climate-seeds.test.mjs — 222 lines, tests the core logic paths
  • No soft fallback to the old rolling baseline — hard fail when normals are missing is the right call (no silent wrong data)
  • PENDING_CONSUMERS set updated honestly in bootstrap.test.mjs for climateZoneNormals
  • CO2 monitoring (#2468) is a clean addition with proper proto/generated code chain

Issues to address

1. verifySeedKey may not exist in _seed-utils.mjs

seed-climate-anomalies.mjs imports verifySeedKey from _seed-utils.mjs, but that function doesn't appear in the diff touching that file. If it's not exported there, calling it throws a synchronous TypeError before the .catch(() => null) can catch it — meaning the seeder always fails with the wrong error message. Please confirm verifySeedKey is exported from _seed-utils.mjs (or add it if missing).

2. Magic string in get-co2-monitoring.ts

const SEED_CACHE_KEY = 'climate:co2-monitoring:v1';

cache-keys.ts defines CLIMATE_ZONE_NORMALS_KEY as a named export — the CO2 key should follow the same pattern. Add CLIMATE_CO2_MONITORING_KEY = 'climate:co2-monitoring:v1' to cache-keys.ts and import it here.

3. Threshold recalibration needs a comment

The old thresholds (absPrecip >= 80, / 20) were percentage-based. The new ones (PRECIP_EXTREME_THRESHOLD = 12, TEMP_TO_PRECIP_RATIO = 3) are daily mm values. The change is correct, but the specific values aren't explained. Add a brief comment (e.g., // mm/day thresholds calibrated against ERA5 daily precip distributions) so the next person doesn't wonder where 12 and 6 came from.


Minor nit

computeMonthlyNormals strips sampleCount via destructuring ({ sampleCount: _sampleCount, ...entry }) before returning — this is correct since it's internal metadata, but worth confirming the tests actually catch a case where sampleCount is 0 (currently they don't). Not a blocker.


Once verifySeedKey is confirmed and the Railway crон entries are added, this is ready to merge. The CO2 addition is a nice bonus — clean end-to-end implementation.

Copy link
Copy Markdown
Owner

@koala73 koala73 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Apologies for the partial review earlier — this one covers the full PR on its own merits.


Confirmed working

  • verifySeedKey, sleep, CHROME_UA are all exported from _seed-utils.mjs — imports are correct
  • computeMonthlyNormals algorithm is climatologically correct: year-as-sample averaging (per-year monthly means → 30-year mean of those means) is the proper WMO approach
  • NOAA column indices are correct for all five file formats (co2_daily_mlo.txt col[4], co2_mm_mlo.txt col[3], etc.)
  • measuredAt stored as epoch-ms string, converted via new Date(Number(...)) in the service layer — correct per project's int64→string pattern
  • isMain guard is correctly wired on all three seeders — test imports work without side effects
  • buildZoneNormalsFromBatch length validation, hasRequiredClimateZones guard, and MIN_CLIMATE_ZONE_COUNT threshold together give real protection against partial-write disasters
  • _open-meteo-archive.mjs batch API design is efficient and correct — Retry-After parsing handles both seconds and date-string formats

Bugs

1. climateZoneNormals is in BOTH BOOTSTRAP_KEYS and STANDALONE_KEYS in health.js

// BOOTSTRAP_KEYS (line ~76)
climateZoneNormals: 'climate:zone-normals:v1',

// STANDALONE_KEYS (line ~85)
climateZoneNormals: 'climate:zone-normals:v1',

The health endpoint processes both sets. allDataKeys concatenates them, so the STRLEN is queried twice for the same Redis key. Then the key is reported twice in the health response — once in the bootstrap section, once in standalone. Every other key in the codebase lives in one or the other, never both. Remove it from STANDALONE_KEYS.

2. climateZoneNormals should not be in BOOTSTRAP_CACHE_KEYS

Zone normals are seeder-internal data — they're only read by seed-climate-anomalies.mjs via verifySeedKey() (direct Redis call, not bootstrap). The frontend has no use for 30-year WMO monthly means. You've correctly marked it as PENDING_CONSUMERS in the bootstrap test, but the right fix is not bootstrapping it at all:

  • Remove climateZoneNormals from api/bootstrap.js BOOTSTRAP_CACHE_KEYS
  • Remove climateZoneNormals from SLOW_KEYS in api/bootstrap.js
  • Remove climateZoneNormals from BOOTSTRAP_TIERS in server/_shared/cache-keys.ts
  • Move it to STANDALONE_KEYS only in health.js
  • Remove the PENDING_CONSUMERS exemption from tests/bootstrap.test.mjs

This avoids shipping a ~20KB seeder-internal payload to every browser on page load.


Issues

3. Magic string in get-co2-monitoring.ts

const SEED_CACHE_KEY = 'climate:co2-monitoring:v1';

cache-keys.ts exports CLIMATE_ZONE_NORMALS_KEY as a named constant. The CO2 key should follow the same pattern — add CLIMATE_CO2_MONITORING_KEY to cache-keys.ts and import it here.

4. buildZoneNormalsFromBatch throws hard on incomplete months

if (months.length !== 12) {
  throw new Error(`Open-Meteo normals incomplete for ${zone.name}: expected 12 months, got ${months.length}`);
}

WestAntarctic (-78°S, -100°W) is deep Antarctica. ERA5/Open-Meteo coverage there may have missing months — particularly austral summer months where the reanalysis grid has known gaps. If this throws, the entire batch fails, increasing failure count by NORMALS_BATCH_SIZE (2) per throw. With a batch of 2 containing one polar zone, one throw = 2 failures. Worth either: (a) catching per-zone within the batch builder and logging a warning, or (b) testing the actual API response for those coordinates before merging.

5. Precipitation threshold recalibration is uncommented

The old thresholds (absPrecip >= 80, / 20) were percentage-based. The new ones (PRECIP_EXTREME_THRESHOLD = 12, TEMP_TO_PRECIP_RATIO = 3) are daily mm values — a completely different scale. The recalibration is physically reasonable, but the values aren't explained. Add a one-line comment: // mm/day thresholds; 6mm/day ≈ 42mm/week above WMO normal. Anyone reading this 6 months from now will wonder where 12 and 6 came from.

6. CO2 API is fully wired but has no UI panel

Proto, seeder, handler, gateway, service client, circuit breaker — all complete and correct. But fetchCo2Monitoring() is not called anywhere in the UI. If there's a follow-up issue for the panel, link it in the PR description. If the intent is to ship the data pipeline now and the panel later, that's fine — just make it explicit.


Nits

  • NORMALS_TTL = 90 days vs maxStaleMin: 89280 (62 days) — not a bug (62d is a 2x safety window for monthly cron) but worth a comment explaining the gap
  • co2Breaker uses cacheTtlMs: 6h on a daily-seeded key — if the seed misses a day, the circuit breaker will serve data that's already up to 24h old for an extra 6h before the next attempt. Acceptable but worth a comment

Summary

The core logic is sound and the architecture is solid. Fix the health.js duplication (bug 1) and the unnecessary bootstrap registration (bug 2) before merging — both are straightforward.

Copy link
Copy Markdown
Owner

@koala73 koala73 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why this PR?

Fixes a fundamental methodological flaw in climate anomaly seeding: replacing the meaningless 30-day self-baseline with WMO 1991-2020 30-year normals. Adds CO2/methane/N2O monitoring from NOAA GML. The architecture — batch Open-Meteo API, clean separation into _climate-zones.mjs + _open-meteo-archive.mjs, proper test coverage (296 lines), health/bootstrap/seed-health wiring — is well-designed and follows project conventions. This is solid foundational work.

Three issues block merge, two are very quick fixes.


🔴 P1 — Blocks Merge

1. precipDelta unit change without cache key version bump (todo #097)

The seeder now computes precipDelta in millimeters (WMO baseline delta) vs the old percentage (rolling self-baseline). The key stays at climate:anomalies:v1. A stale cached entry with precipDelta = 40 (old %) will trigger ANOMALY_SEVERITY_EXTREME after deploy (PRECIP_EXTREME_THRESHOLD = 12mm). Misclassification lasts until the next 2h seed run.

Fix: Bump to climate:anomalies:v2 in all 6 registration points (seed-climate-anomalies.mjs, cache-keys.ts, bootstrap.js, health.js, mcp.ts, seed-health.js). The v1 key expires naturally after 3h.

2. CO2 TTL = 1× daily interval (gold standard: 3×) (todo #096)

CACHE_TTL = 86400 (24h) for a daily cron. One missed Railway run = CO2 data goes dark for up to 24h. Project gold standard requires TTL ≥ 3× interval.

Fix (2 lines):

const CACHE_TTL = 259200; // 72h = 3× daily interval (gold standard)

Update api/health.js co2Monitoring.maxStaleMin to 4320 (72h).

3. Cold-start gap: anomaly seeder fails for up to 31 days on first deploy (todo #098)

seed-climate-anomalies.mjs now throws if climate:zone-normals:v1 is absent. The anomaly cron runs every 2h, the normals cron runs monthly. If both register simultaneously on a fresh environment, the anomaly seeder fails every 2h until the 1st of the following month. Health shows climateAnomalies: STALE immediately with no signal about the root cause.

Fix (minimum): Add a ## First Deploy comment near verifySeedKey documenting that node scripts/seed-climate-zone-normals.mjs must be run manually before the anomaly cron is enabled. Ideally also trigger a one-time normals seed on Railway deploy.


🟡 P2 — Should Fix

4. Zone normals TTL = 2.9× interval (under 3× gold standard) (todo #099)

NORMALS_TTL = 90 days, monthly interval ≈ 31 days → 2.9×. Change to 95 * 24 * 60 * 60.

5. MCP _seedMetaKey reports anomaly freshness for a tool that now includes CO2 (todo #100)

_seedMetaKey: 'seed-meta:climate:anomalies' + _maxStaleMin: 120. CO2 updates daily — the tool will report stale: false when CO2 is 23h stale. Minimum fix: switch to _seedMetaKey: 'seed-meta:climate:co2-monitoring' + _maxStaleMin: 2880. Also update the description to call out CO2/methane/N2O ppm explicitly for agent routing.

6. Railway cron not registered for new seeders (todo #101)

Health entries added (climateZoneNormals, co2Monitoring) but no Railway cron service is configured. From deploy minute zero, health shows two false failures. Either register the crons in this PR or defer the health entries to the Railway config PR.

7. measuredAt: new Date(0) when proto field is missing (todo #102)

src/services/climate/index.ts: new Date(Number(proto.measuredAt ?? 0)) returns 1970-01-01 when data is absent. measuredAt should be Date | undefined.

8. Co2DataPoint.anomaly has no unit documented anywhere (todo #103)

Proto, OpenAPI, TypeScript interface — all have anomaly: number with no description. It's ppm year-over-year delta, not a percentage. Add a proto comment (// Year-over-year delta vs same calendar month, in ppm) and regenerate.

9. Open-Meteo Retry-After unbounded + 503 not retried (todo #104)

parseRetryAfterMs has no upper bound cap — an upstream Retry-After: 86400 blocks the worker for 24h. Also, 503 is not retried (only 429 is), so Open-Meteo maintenance windows permanently drop zones from the monthly normals run.


🔵 P3 — Nice to Have

10. computeMonthlyNormals: O(n) bucket.find() + sampleCount compute-then-strip (todo #105)

Replace array bucket with Map<string, {temps, precips}> for O(1) day lookup. Eliminate the sampleCount intermediate property that's computed, filtered on, then stripped with { sampleCount: _sampleCount, ...entry }.

11. get_climate_data description should name CO2 ppm / methane / N2O (todo #106)

Agents querying CO2 levels may not route here. Add: (CO2 ppm, methane ppb, N2O ppb, Mauna Loa 12-month trend) to the description.


What's Well Done

  • isMain guard on both new seed scripts ✓
  • _open-meteo-archive.mjs batch helper: clean, reusable, tested ✓
  • _climate-zones.mjs as canonical zone registry ✓
  • computeMonthlyNormals() with Number.isFinite() guards ✓
  • 296 lines of unit tests covering all pure computation functions ✓
  • Health + bootstrap + seed-health all wired correctly ✓
  • verifySeedKey dependency enforces seeding order ✓
  • CO2 parser functions are well-validated with isValidMeasurement()

The P1 fixes are small (2 lines + a key rename + a comment). Happy to review again quickly once those are in.

Copy link
Copy Markdown
Owner

@koala73 koala73 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why this PR?

Fixes a fundamental methodological flaw in climate anomaly seeding: replacing the meaningless 30-day self-baseline with WMO 1991-2020 30-year normals. Adds CO2/methane/N2O monitoring from NOAA GML. The architecture -- batch Open-Meteo API, clean separation into _climate-zones.mjs + _open-meteo-archive.mjs, proper test coverage (296 lines), health/bootstrap/seed-health wiring -- is well-designed and follows project conventions. This is solid foundational work.

Three issues block merge, two are very quick fixes.


P1 -- Blocks Merge

1. precipDelta unit change without cache key version bump

The seeder now computes precipDelta in millimeters (WMO baseline delta) vs the old percentage (rolling self-baseline). The key stays at climate:anomalies:v1. A stale cached entry with precipDelta = 40 (old %) will trigger ANOMALY_SEVERITY_EXTREME after deploy (PRECIP_EXTREME_THRESHOLD = 12mm). Misclassification lasts until the next 2h seed run.

Fix: Bump to climate:anomalies:v2 in all 6 registration points (seed-climate-anomalies.mjs, cache-keys.ts, bootstrap.js, health.js, mcp.ts, seed-health.js). The v1 key expires naturally after 3h.

2. CO2 TTL = 1x daily interval (gold standard: 3x)

CACHE_TTL = 86400 (24h) for a daily cron. One missed Railway run = CO2 data goes dark for up to 24h. Gold standard requires TTL >= 3x interval.

Fix (2 lines):

const CACHE_TTL = 259200; // 72h = 3x daily interval (gold standard)

Update api/health.js co2Monitoring.maxStaleMin to 4320 (72h).

3. Cold-start gap: anomaly seeder fails for up to 31 days on first deploy

seed-climate-anomalies.mjs now throws if climate:zone-normals:v1 is absent. The anomaly cron runs every 2h, the normals cron runs monthly. If both register simultaneously on a fresh environment, the anomaly seeder fails every 2h until the 1st of the following month. Health shows climateAnomalies: STALE immediately with no signal about the root cause.

Fix (minimum): Add a comment near verifySeedKey documenting that seed-climate-zone-normals.mjs must be run manually before the anomaly cron is enabled on first deploy.


P2 -- Should Fix

4. Zone normals TTL = 2.9x interval (under 3x gold standard)
NORMALS_TTL = 90 days, monthly interval ~31 days = 2.9x. Change to 95 * 24 * 60 * 60.

5. MCP _seedMetaKey reports anomaly freshness for a tool that now includes CO2
_seedMetaKey: 'seed-meta:climate:anomalies' + _maxStaleMin: 120. CO2 updates daily -- the tool will report stale: false when CO2 is 23h stale. Switch to _seedMetaKey: 'seed-meta:climate:co2-monitoring' + _maxStaleMin: 2880. Also update description to mention CO2 ppm, methane ppb, N2O ppb explicitly for agent routing.

6. Railway cron not registered for new seeders
Health entries added (climateZoneNormals, co2Monitoring) but no Railway cron service configured. From deploy minute zero, health shows two false failures. Either register the crons in this PR or defer the health entries to the Railway config PR.

7. measuredAt: new Date(0) when proto field is missing
src/services/climate/index.ts: new Date(Number(proto.measuredAt ?? 0)) returns 1970-01-01 when data is absent. measuredAt should be Date | undefined.

8. Co2DataPoint.anomaly has no unit documented
Proto, OpenAPI, TypeScript interface -- all have anomaly: number with no description. Add proto comment: // Year-over-year delta vs same calendar month, in ppm. Regenerate.

9. Open-Meteo Retry-After unbounded + 503 not retried
parseRetryAfterMs has no upper bound -- upstream Retry-After: 86400 would block the worker for 24h. Also 503 is not retried (only 429 is), so maintenance windows permanently drop zones from the monthly normals run.


P3 -- Nice to Have

10. computeMonthlyNormals: O(n) bucket.find() + sampleCount compute-then-strip
Replace array bucket with Map<string, {temps, precips}> for O(1) day lookup. Eliminate the sampleCount intermediate property that is computed, filtered on, then stripped.

11. get_climate_data description should name CO2 ppm / methane / N2O explicitly for agent routing


What's Well Done

  • isMain guard on both new seed scripts
  • _open-meteo-archive.mjs batch helper: clean, reusable, tested
  • _climate-zones.mjs as canonical zone registry
  • computeMonthlyNormals() with Number.isFinite() guards (no null-as-NaN bug)
  • 296 lines of unit tests covering all pure computation functions
  • Health + bootstrap + seed-health all wired correctly
  • verifySeedKey dependency enforces correct seeding order
  • CO2 parser functions well-validated with isValidMeasurement()

The P1 fixes are small (2 lines + a key rename + a comment). Happy to re-review once those are in.

@FayezBast
Copy link
Copy Markdown
Collaborator Author

done @koala73

Copy link
Copy Markdown
Owner

@koala73 koala73 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Follow-up review — fix commit verified

All P1 and P2 issues from the first review have been addressed. Comprehensive fix, well done.

Resolved

P1 — All fixed:

  • CO2 TTL bumped to 259200s (72h = 3× daily interval) — gold standard met
  • climate:anomalies:v2 key bump propagated to cache-keys.ts, health.js, seed-health.js, get-risk-scores.ts, and mcp.ts
  • First Deploy comment added with verifySeedKey guard and clear ## First Deploy callout in anomaly seeder
  • Zone normals TTL set to 95 days (3× monthly interval)

P2 — All fixed:

  • 503 now retried alongside 429 via RETRYABLE_STATUSES = new Set([429, 503])
  • Retry-After capped at MAX_RETRY_AFTER_MS = 60_000 — unbounded worker sleep eliminated
  • measuredAt?: Date optional in Co2Monitoring display interface; returns undefined (not epoch 1970) when absent
  • Co2DataPoint.anomaly documented in proto with unit (ppm, year-over-year delta)
  • MCP tool description now explicitly mentions CO2 ppm, methane ppb, N2O ppb, Mauna Loa

Beyond what was asked:
The _freshnessChecks multi-key approach in mcp.ts is a genuine improvement over the single _seedMetaKey pattern. evaluateFreshness() correctly aggregates staleness across anomalies, CO2, and weather alerts — any stale key surfaces in the tool response. Clean design.

Still open (non-blocking, tracked as todo #101)

Railway cron not registered — health entries exist for both new seeders (climate:zone-normals, climate:co2-monitoring) but no Railway config was updated with corresponding cron service definitions. From deploy minute zero, the health dashboard will show these as stale until manually configured. Recommend a follow-up Railway config PR before or shortly after this merges to prod.

P3 (non-blocking, tracked as todo #105)

computeMonthlyNormals() still has the sampleCount compute-then-strip pattern and bucket.find() O(n) scan. Neither blocks correctness — optional cleanup.

Approving. Ship it.

@koala73 koala73 merged commit bb4f8dc into main Apr 2, 2026
9 checks passed
@koala73 koala73 deleted the feat/greenhouse-gas-monitoring branch April 2, 2026 04:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

2 participants