Skip to content

feat(climate): add WMO normals seeding and CO2 monitoring #2531

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
koala73 merged 4 commits into from
Apr 2, 2026
Merged

feat(climate): add WMO normals seeding and CO2 monitoring #2531

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
3602084
feat(climate): add WMO normals seeding and CO2 monitoring
FayezBast Mar 30, 2026
3390944
fix(climate): skip missing normals per-zone and align anomaly tooltip…
FayezBast Mar 30, 2026
7bfcb33
fix(climate): remove normals from bootstrap and harden health/cache k…
FayezBast Mar 30, 2026
5f92077
feat(climate): version anomaly cache to v2, harden seed freshness, an…
FayezBast Apr 1, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 3 additions & 2 deletions api/bootstrap.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,8 @@ const BOOTSTRAP_CACHE_KEYS = {
chokepointTransits: 'supply_chain:chokepoint_transits:v1',
minerals: 'supply_chain:minerals:v2',
giving: 'giving:summary:v1',
climateAnomalies: 'climate:anomalies:v1',
climateAnomalies: 'climate:anomalies:v2',
co2Monitoring: 'climate:co2-monitoring:v1',
radiationWatch: 'radiation:observations:v1',
thermalEscalation: 'thermal:escalation:v1',
crossSourceSignals: 'intelligence:cross-source-signals:v1',
Expand Down Expand Up @@ -85,7 +86,7 @@ const BOOTSTRAP_CACHE_KEYS = {

const SLOW_KEYS = new Set([
'bisPolicy', 'bisExchange', 'bisCredit', 'minerals', 'giving',
'sectors', 'etfFlows', 'wildfires', 'climateAnomalies',
'sectors', 'etfFlows', 'wildfires', 'climateAnomalies', 'co2Monitoring',
'radiationWatch', 'thermalEscalation', 'crossSourceSignals',
'cyberThreats', 'techReadiness', 'progressData', 'renewableEnergy',
'naturalEvents',
Expand Down
6 changes: 5 additions & 1 deletion api/health.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,8 @@ const BOOTSTRAP_KEYS = {
outages: 'infra:outages:v1',
sectors: 'market:sectors:v1',
etfFlows: 'market:etf-flows:v1',
climateAnomalies: 'climate:anomalies:v1',
climateAnomalies: 'climate:anomalies:v2',
co2Monitoring: 'climate:co2-monitoring:v1',
wildfires: 'wildfire:fires:v1',
marketQuotes: 'market:stocks-bootstrap:v1',
commodityQuotes: 'market:commodities-bootstrap:v1',
Expand Down Expand Up @@ -84,6 +85,7 @@ const STANDALONE_KEYS = {
bisPolicy: 'economic:bis:policy:v1',
bisExchange: 'economic:bis:eer:v1',
Comment on lines 85 to 86
Copy link
Copy Markdown

@greptile-apps greptile-apps bot Mar 30, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 climateZoneNormals double-registered: BOOTSTRAP_KEYS + STANDALONE_KEYS

climateZoneNormals is being registered in both BOOTSTRAP_KEYS (line ~32) and STANDALONE_KEYS (line ~87). This causes two problems:

  1. Duplicate health check — the health module iterates both maps independently, so climate:zone-normals:v1 will be checked twice in every health report, potentially double-counting failures.

  2. Bootstrap response inflation — because the key is also in BOOTSTRAP_KEYS (and BOOTSTRAP_CACHE_KEYS in bootstrap.js), the full normals payload (22 zones × 12 monthly normals) is included in every bootstrap response sent to browser clients, even though there is no frontend consumer (climateZoneNormals is explicitly in PENDING_CONSUMERS in bootstrap.test.mjs).

Since zone normals are a backend-only baseline consumed directly by seed scripts via Redis, they have no business being in the bootstrap pipeline at all. The key should live only in STANDALONE_KEYS, and climateZoneNormals should be removed from BOOTSTRAP_KEYS in this file, from BOOTSTRAP_CACHE_KEYS in api/bootstrap.js, and from BOOTSTRAP_CACHE_KEYS / BOOTSTRAP_TIERS in server/_shared/cache-keys.ts.

bisCredit: 'economic:bis:credit:v1',
climateZoneNormals: 'climate:zone-normals:v1',
shippingRates: 'supply_chain:shipping:v2',
chokepoints: 'supply_chain:chokepoints:v4',
minerals: 'supply_chain:minerals:v2',
Expand Down Expand Up @@ -127,6 +129,8 @@ const SEED_META = {
wildfires: { key: 'seed-meta:wildfire:fires', maxStaleMin: 360 }, // FIRMS NRT resets at midnight UTC; new-day data takes 3-6h to accumulate
outages: { key: 'seed-meta:infra:outages', maxStaleMin: 30 },
climateAnomalies: { key: 'seed-meta:climate:anomalies', maxStaleMin: 120 }, // runs as independent Railway cron (0 */2 * * *)
climateZoneNormals: { key: 'seed-meta:climate:zone-normals', maxStaleMin: 89280 }, // monthly cron on the 1st; 62d = 2x 31-day cadence
co2Monitoring: { key: 'seed-meta:climate:co2-monitoring', maxStaleMin: 4320 }, // daily cron at 06:00 UTC; 72h tolerates two missed runs
unrestEvents: { key: 'seed-meta:unrest:events', maxStaleMin: 120 }, // 45min cron; 120 = 2h grace (was 75 = 30min buffer, too tight)
cyberThreats: { key: 'seed-meta:cyber:threats', maxStaleMin: 240 }, // 2h interval; 240min = 2x interval
cryptoQuotes: { key: 'seed-meta:market:crypto', maxStaleMin: 30 },
Expand Down
65 changes: 51 additions & 14 deletions api/mcp.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -48,11 +48,17 @@ interface BaseToolDef {
inputSchema: { type: string; properties: Record<string, unknown>; required: string[] };
}

interface FreshnessCheck {
key: string;
maxStaleMin: number;
}

// Cache-read tool: reads one or more Redis keys and returns them with staleness info.
interface CacheToolDef extends BaseToolDef {
_cacheKeys: string[];
_seedMetaKey: string;
_maxStaleMin: number;
_freshnessChecks?: FreshnessCheck[];
_execute?: never;
}

Expand All @@ -61,6 +67,7 @@ interface RpcToolDef extends BaseToolDef {
_cacheKeys?: never;
_seedMetaKey?: never;
_maxStaleMin?: never;
_freshnessChecks?: never;
_execute: (params: Record<string, unknown>, base: string, apiKey: string) => Promise<unknown>;
}

Expand Down Expand Up @@ -180,11 +187,16 @@ const TOOL_REGISTRY: ToolDef[] = [
},
{
name: 'get_climate_data',
description: 'Climate anomalies (Open-Meteo temperature/precipitation deviations), weather alerts, and natural environmental events from NASA EONET.',
description: 'Climate anomalies, NOAA atmospheric greenhouse gas monitoring (CO2 ppm, methane ppb, N2O ppb, Mauna Loa 12-month trend), weather alerts, and natural environmental events from WorldMonitor climate feeds.',
inputSchema: { type: 'object', properties: {}, required: [] },
_cacheKeys: ['climate:anomalies:v1', 'weather:alerts:v1'],
_seedMetaKey: 'seed-meta:climate:anomalies',
_maxStaleMin: 120,
_cacheKeys: ['climate:anomalies:v2', 'climate:co2-monitoring:v1', 'weather:alerts:v1'],
_seedMetaKey: 'seed-meta:climate:co2-monitoring',
_maxStaleMin: 2880,
_freshnessChecks: [
{ key: 'seed-meta:climate:anomalies', maxStaleMin: 120 },
{ key: 'seed-meta:climate:co2-monitoring', maxStaleMin: 2880 },
{ key: 'seed-meta:weather:alerts', maxStaleMin: 45 },
],
},
{
name: 'get_infrastructure_status',
Expand Down Expand Up @@ -673,21 +685,46 @@ function rpcError(id: unknown, code: number, message: string): Response {
return jsonResponse({ jsonrpc: '2.0', id: id ?? null, error: { code, message } }, 200);
}

export function evaluateFreshness(checks: FreshnessCheck[], metas: unknown[], now = Date.now()): { cached_at: string | null; stale: boolean } {
let stale = false;
let oldestFetchedAt = Number.POSITIVE_INFINITY;
let hasAnyValidMeta = false;
let hasAllValidMeta = true;

for (const [i, check] of checks.entries()) {
const meta = metas[i];
const fetchedAt = meta && typeof meta === 'object' && 'fetchedAt' in meta
? Number((meta as { fetchedAt: unknown }).fetchedAt)
: Number.NaN;

if (!Number.isFinite(fetchedAt) || fetchedAt <= 0) {
hasAllValidMeta = false;
stale = true;
continue;
}

hasAnyValidMeta = true;
oldestFetchedAt = Math.min(oldestFetchedAt, fetchedAt);
stale ||= (now - fetchedAt) / 60_000 > check.maxStaleMin;
}

return {
cached_at: hasAnyValidMeta && hasAllValidMeta ? new Date(oldestFetchedAt).toISOString() : null,
stale,
};
}

// ---------------------------------------------------------------------------
// Tool execution
// ---------------------------------------------------------------------------
async function executeTool(tool: CacheToolDef): Promise<{ cached_at: string | null; stale: boolean; data: Record<string, unknown> }> {
const reads = tool._cacheKeys.map(k => readJsonFromUpstash(k));
const metaRead = readJsonFromUpstash(tool._seedMetaKey);
const [results, meta] = await Promise.all([Promise.all(reads), metaRead]);

let cached_at: string | null = null;
let stale = true;
if (meta && typeof meta === 'object' && 'fetchedAt' in meta) {
const fetchedAt = (meta as { fetchedAt: number }).fetchedAt;
cached_at = new Date(fetchedAt).toISOString();
stale = (Date.now() - fetchedAt) / 60_000 > tool._maxStaleMin;
}
const freshnessChecks = tool._freshnessChecks?.length
? tool._freshnessChecks
: [{ key: tool._seedMetaKey, maxStaleMin: tool._maxStaleMin }];
const metaReads = freshnessChecks.map((check) => readJsonFromUpstash(check.key));
const [results, metas] = await Promise.all([Promise.all(reads), Promise.all(metaReads)]);
const { cached_at, stale } = evaluateFreshness(freshnessChecks, metas);

const data: Record<string, unknown> = {};
// Walk backward through ':'-delimited segments, skipping non-informative suffixes
Expand Down
2 changes: 2 additions & 0 deletions api/seed-health.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,8 @@ const SEED_DOMAINS = {
'wildfire:fires': { key: 'seed-meta:wildfire:fires', intervalMin: 60 },
'infra:outages': { key: 'seed-meta:infra:outages', intervalMin: 15 },
'climate:anomalies': { key: 'seed-meta:climate:anomalies', intervalMin: 60 },
'climate:zone-normals': { key: 'seed-meta:climate:zone-normals', intervalMin: 44640 },
'climate:co2-monitoring': { key: 'seed-meta:climate:co2-monitoring', intervalMin: 2160 },
// Phase 2 — Parameterized endpoints
'unrest:events': { key: 'seed-meta:unrest:events', intervalMin: 15 },
'cyber:threats': { key: 'seed-meta:cyber:threats', intervalMin: 240 },
Expand Down
Loading
Loading