fix(climate): replace 30-day rolling baseline with WMO 30-year normals#2504
fix(climate): replace 30-day rolling baseline with WMO 30-year normals#2504fuleinist wants to merge 4 commits intokoala73:mainfrom
Conversation
- Create seed-climate-zone-normals.mjs to fetch 1991-2020 historical monthly means from Open-Meteo archive API per zone - Update seed-climate-anomalies.mjs to use WMO normals as baseline instead of climatologically meaningless 30-day rolling window - Add 7 new climate-specific zones: Arctic, Greenland, WestAntarctic, TibetanPlateau, CongoBasin, CoralTriangle, NorthAtlantic - Register climateZoneNormals cache key in cache-keys.ts - Add fallback to rolling baseline if normals not yet cached Fixes: koala73#2467
|
Someone is attempting to deploy a commit to the Elie Team on Vercel. A member of the Team first needs to authorize it. |
Greptile SummaryThis PR replaces the 30-day rolling climate baseline with WMO 30-year normals (1991\u20132020) and adds 7 new climate-specific monitoring zones, plus an unrelated font-switching refactor. The climatological motivation is sound, but the implementation has two P1 bugs that will cause the climate anomaly seeder to fail on every run after deployment. Key changes:
Critical issues:
Font changes are clean and correctly fix the RTL specificity problem.
Confidence Score: 4/5Not safe to merge as-is — the climate anomaly seeder will abort on every run after deployment due to two interacting P1 bugs. Two P1 bugs interact to completely break the climate anomaly seeder: the normals seeder only covers 7 of 22 zones so 15 original zones always hit the rolling fallback, and the rolling fallback is broken because only 7 days of data is fetched. MIN_ZONES will never be reached, so the seeder throws and preserves increasingly stale Redis data forever. The font-switching and CSS changes are correct and clean. scripts/seed-climate-zone-normals.mjs must seed all 22 zones (not just 7 CLIMATE_ZONES). scripts/seed-climate-anomalies.mjs fallback path needs to fetch 30 days when normals are unavailable, or the fallback should be removed if the intention is WMO-only. Important Files Changed
Sequence DiagramsequenceDiagram
participant RailwayCron as Railway Cron
participant NormalsSeeder as seed-climate-zone-normals.mjs
participant AnomalySeeder as seed-climate-anomalies.mjs
participant OpenMeteo as Open-Meteo Archive API
participant Redis as Redis (Upstash)
Note over RailwayCron,Redis: Monthly run (1st of month)
RailwayCron->>NormalsSeeder: trigger
loop 7 CLIMATE_ZONES x 30 years
NormalsSeeder->>OpenMeteo: GET archive 1991-2020
OpenMeteo-->>NormalsSeeder: daily temps + precip
end
NormalsSeeder->>Redis: SET climate:zone-normals:v1 (TTL 30d)
Note over RailwayCron,Redis: Every 2h run
RailwayCron->>AnomalySeeder: trigger
AnomalySeeder->>Redis: GET climate:zone-normals:v1
Redis-->>AnomalySeeder: normals (7 CLIMATE_ZONES only)
loop ALL_ZONES (22 zones)
AnomalySeeder->>OpenMeteo: GET last 7 days
OpenMeteo-->>AnomalySeeder: 7 days data
alt WMO normal found (7 CLIMATE_ZONES)
AnomalySeeder->>AnomalySeeder: delta = current - WMO normal
else No normal (15 geopolitical zones)
AnomalySeeder->>AnomalySeeder: baselineTemps = slice(0,-7) empty
AnomalySeeder->>AnomalySeeder: return null - fallback broken
end
end
Note over AnomalySeeder: anomalies=7 less than MIN_ZONES=15 - throws, aborts write
Reviews (1): Last reviewed commit: "fix(climate): replace 30-day rolling bas..." | Re-trigger Greptile |
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
| if (!zoneNormal) { | ||
| // Fallback: compute from previous 30 days if normals not available | ||
| // (This is the old behavior for backwards compatibility during transition) | ||
| const baselineTemps = temps.slice(0, -7); | ||
| const baselinePrecips = precips.slice(0, -7); | ||
|
|
||
| if (baselineTemps.length < 7) return null; | ||
|
|
||
| const baselineTempMean = avg(baselineTemps); | ||
| const baselinePrecipMean = avg(baselinePrecips); | ||
|
|
||
| const tempDelta = Math.round((currentTempMean - baselineTempMean) * 10) / 10; | ||
| const precipDelta = Math.round((currentPrecipMean - baselinePrecipMean) * 10) / 10; | ||
|
|
||
| return { | ||
| zone: zone.name, | ||
| location: { latitude: zone.lat, longitude: zone.lon }, | ||
| tempDelta, | ||
| precipDelta, | ||
| severity: classifySeverity(tempDelta, precipDelta), | ||
| type: classifyType(tempDelta, precipDelta), | ||
| period: `${startDate} to ${endDate}`, | ||
| baselineSource: 'rolling-30d-fallback', | ||
| }; |
There was a problem hiding this comment.
The date range fetched was reduced from 30 days to 7 days (startDate = Date.now() - 7 days), but the rolling fallback still tries to split the data into a baseline window + a recent window:
const baselineTemps = temps.slice(0, -7); // Empty when only 7 days fetchedWith only 7 days of data available, baselineTemps is always [], so baselineTemps.length < 7 is always true, and the function always return null.
This is critical because the normals seeder (seed-climate-zone-normals.mjs) only seeds normals for 7 CLIMATE_ZONES — it does not seed the original 15 geopolitical zones (Ukraine, Middle East, Sahel, etc.). So every one of those 15 zones will hit this fallback and return null.
Net effect: At most 7 anomalies (from CLIMATE_ZONES) can be produced, but MIN_ZONES = ceil(22 × 2/3) = 15. The seeder will throw and abort on every single run after this deployment, silently preserving the old (increasingly stale) Redis data.
Two separate fixes are needed:
- Add all 22
ALL_ZONEStoseed-climate-zone-normals.mjs, not justCLIMATE_ZONES. - Fix the fallback to fetch 30 days when normals are unavailable, or explicitly return
nullwithout the broken baseline split (since the window is only 7 days).
| const CLIMATE_ZONES = [ | ||
| { name: 'Arctic', lat: 70.0, lon: 0.0 }, // sea ice proxy | ||
| { name: 'Greenland', lat: 72.0, lon: -42.0 }, // ice sheet melt | ||
| { name: 'WestAntarctic', lat: -78.0, lon: -100.0 }, // Antarctic Ice Sheet | ||
| { name: 'TibetanPlateau', lat: 31.0, lon: 91.0 }, // third pole | ||
| { name: 'CongoBasin', lat: -1.0, lon: 24.0 }, // largest tropical forest after Amazon | ||
| { name: 'CoralTriangle', lat: -5.0, lon: 128.0 }, // reef bleaching proxy | ||
| { name: 'NorthAtlantic', lat: 55.0, lon: -30.0 }, // AMOC slowdown signal | ||
| ]; |
There was a problem hiding this comment.
This seeder seeds normals for only the 7 new CLIMATE_ZONES, but seed-climate-anomalies.mjs uses ALL_ZONES (22 zones) and looks up normals for every one:
// In seed-climate-anomalies.mjs
const zoneNormal = normals?.find((n) => n.zone === zone.name);The 15 original geopolitical zones (Ukraine, Middle East, Sahel, Horn of Africa, etc.) will never match a cached normal and will always fall through to the broken rolling-baseline path (see companion comment on seed-climate-anomalies.mjs).
To make the WMO baseline work for ALL zones, CLIMATE_ZONES here should be replaced with — or merged with — the full ALL_ZONES list from seed-climate-anomalies.mjs. Since both files share the same zone definitions, consider extracting them into a shared constant (e.g., a shared/climate-zones.json) to prevent future drift.
server/_shared/cache-keys.ts
Outdated
| giving: 'giving:summary:v1', | ||
| climateAnomalies: 'climate:anomalies:v1', | ||
| climateAnomalies: 'climate:anomalies:v1', | ||
| climateZoneNormals: 'climate:zone-normals:v1', |
There was a problem hiding this comment.
climateZoneNormals registered in BOOTSTRAP_CACHE_KEYS but not wired in api/bootstrap.js or BOOTSTRAP_TIERS
climateZoneNormals was added to server/_shared/cache-keys.ts but is absent from both:
api/bootstrap.js(the actual Edge Function that serves bootstrap data) — which maintains its own hardcoded copy of the key registry and is the file that AGENTS.md requires be updated for new data sourcesBOOTSTRAP_TIERSin this same file (line ~82) — which governs cache tier assignment
If climateZoneNormals is intended as frontend-accessible bootstrap data, both api/bootstrap.js and BOOTSTRAP_TIERS need to be updated. If it's purely an internal seed-pipeline artifact (the anomaly seeder reads it, but the UI doesn't), it arguably shouldn't be in BOOTSTRAP_CACHE_KEYS at all, which is documented as being for the bootstrap endpoint.
- seed-climate-zone-normals.mjs: Now fetches normals for ALL 22 zones (15 original geopolitical + 7 new climate zones) instead of just the 7 new climate zones. The 15 original zones were falling through to the broken rolling fallback. - seed-climate-anomalies.mjs: Fixed rolling fallback to fetch 30 days of data when WMO normals are not yet cached. Previously fetched only 7 days, causing baselineTemps slice to be empty and returning null for all zones. Now properly falls back to 30-day rolling baseline (last 7 days vs. prior 23 days) when normals seeder hasn't run. - cache-keys.ts: Removed climateZoneNormals from BOOTSTRAP_CACHE_KEYS. This is an internal seed-pipeline artifact (used by the anomaly seeder to read cached normals) and is not meant for the bootstrap endpoint. Only climate:anomalies:v1 (the final computed output) should be exposed to clients. Fixes greptile-apps P1 comments on PR koala73#2504.
koala73
left a comment
koala73
left a comment
There was a problem hiding this comment.
Thanks for tackling this — replacing the rolling-30d baseline with proper WMO climatological normals is the right fix, and the fallback logic for the transition period is a nice touch.
A few things need to be addressed before this is ready to merge:
Blockers
1. cache-keys.ts registration is missing
The issue explicitly requires adding climateZoneNormals: 'climate:zone-normals:v1' to server/_shared/cache-keys.ts. The diff doesn't touch that file — ZONE_NORMALS_KEY ends up as a magic string duplicated across two seeders with no central registry.
2. No Railway cron registration
The issue calls for the normals seeder to run "monthly (1st of month, 03:00 UTC) via Railway cron." There is no cron config change in this PR. Without it, climate:zone-normals:v1 will never be populated automatically, and seed-climate-anomalies.mjs will silently fall back to the old rolling baseline forever — meaning the fix never actually activates in production.
Other Issues
3. Unrelated changes bundled in
src/services/font-settings.ts and src/styles/main.css have no relation to the climate fix. Please move those to a separate PR.
4. Raised MIN_ZONES threshold may cause write skips
MIN_ZONES now uses ALL_ZONES.length (22) instead of the original 15. The 7 new zones include polar regions (Arctic at 70°N, WestAntarctic at −78°S) that may have sparse Open-Meteo coverage. If 8+ zones return null, the entire Redis write is skipped. Worth validating actual API coverage for those coordinates before locking in the threshold.
5. No tests
Neither the normals-lookup path nor the fallback path has test coverage. Please add tests when adding new seeder logic.
Core logic looks correct — the normals fetch/parse/lookup chain, the Upstash REST API shape handling, and the zone definitions being kept in sync between both files are all solid. Just need the Railway cron wired up and the cache-keys registration added to make this actually work end-to-end.
|
Actually , another contributor @FayezBast did a more complete implementation #2531 I suggest you close this, with thanks |
- seed-climate-zone-normals.mjs: Now fetches normals for ALL 22 zones (15 original geopolitical + 7 new climate zones) instead of just the 7 new climate zones. The 15 original zones were falling through to the broken rolling fallback. - seed-climate-anomalies.mjs: Fixed rolling fallback to fetch 30 days of data when WMO normals are not yet cached. Previously fetched only 7 days, causing baselineTemps slice to be empty and returning null for all zones. Now properly falls back to 30-day rolling baseline (last 7 days vs. prior 23 days) when normals seeder hasn't run. - cache-keys.ts: Removed climateZoneNormals from BOOTSTRAP_CACHE_KEYS. This is an internal seed-pipeline artifact (used by the anomaly seeder to read cached normals) and is not meant for the bootstrap endpoint. Only climate:anomalies:v1 (the final computed output) should be exposed to clients. Fixes greptile-apps P1 comments on PR koala73#2504.
- seed-climate-zone-normals.mjs: Now fetches normals for ALL 22 zones (15 original geopolitical + 7 new climate zones) instead of just the 7 new climate zones. The 15 original zones were falling through to the broken rolling fallback. - seed-climate-anomalies.mjs: Fixed rolling fallback to fetch 30 days of data when WMO normals are not yet cached. Previously fetched only 7 days, causing baselineTemps slice to be empty and returning null for all zones. Now properly falls back to 30-day rolling baseline (last 7 days vs. prior 23 days) when normals seeder hasn't run. - cache-keys.ts: Removed climateZoneNormals from BOOTSTRAP_CACHE_KEYS. This is an internal seed-pipeline artifact (used by the anomaly seeder to read cached normals) and is not meant for the bootstrap endpoint. Only climate:anomalies:v1 (the final computed output) should be exposed to clients. Fixes greptile-apps P1 comments on PR koala73#2504.
2 participants
Summary
This PR replaces the climatologically meaningless 30-day rolling baseline with proper WMO 30-year climatological normals (1991-2020) for climate anomaly detection.
Problem
The current implementation in
seed-climate-anomalies.mjscompares the last 7 days against the previous 23 days of the same 30-day window. This is wrong because:Solution
Step 1: New
seed-climate-zone-normals.mjsclimate:zone-normals:v1(TTL 30 days)Step 2: Updated
seed-climate-anomalies.mjsclimate:zone-normals:v1from Redis as baselineclimate:anomalies:v1cache key — fix in placeStep 3: Added 7 new climate zones
New zones for climate-specific monitoring:
Step 4: Cache key registration
Added
climateZoneNormals: 'climate:zone-normals:v1'toserver/_shared/cache-keys.ts.Testing
Related Issue
Fixes #2467