Skip to content

Tier 16: WS2 burndown sub-tier 4 — fact_order_events DDL (final sub-tier) - #18

Merged
jamezycesar-collab merged 1 commit into
mainfrom
tier16-ws2-fact-order-events
Jun 18, 2026
Merged

Tier 16: WS2 burndown sub-tier 4 — fact_order_events DDL (final sub-tier)#18
jamezycesar-collab merged 1 commit into
mainfrom
tier16-ws2-fact-order-events

Conversation

@jamezycesar-collab

Copy link
Copy Markdown
Owner

Summary

Final sub-tier of WS2 phantom-table burndown. Builds fact_order_events across all four dialects, completing the 120-row F3.1 phantom-table closure that started in Tier 13.

Files changed

  • ddl/equity/02_equity_gold_delta.sql - add fact_order_events (Delta)
  • ddl/equity/04_equity_gold_hive.sql - add fact_order_events (Hive)
  • ddl/equity/05_equity_gold_fabric_lakehouse.sql - add fact_order_events (Fabric Lakehouse)
  • ddl/equity/06_equity_gold_fabric_warehouse.sql - add fact_order_events (T-SQL)
  • guardrails/known_field_mapping_gaps.csv - 151 → 92 rows
  • guardrails/validate_check_constraints.py - register event_type_code + action_type
  • CHANGELOG.md - Tier 16 entry

Schema

fact_order_events (72 columns — largest fact in the equity Gold layer):

  • order_event_sk BIGINT IDENTITY (PK)
  • event_dts / event_date
  • 5 dim FKs: date_sk, instrument_sk, party_sk, venue_sk (nullable), event_type_sk
  • action_type (CHECK ⊂ {NEW, FRC, RPR}) — Section 4.1 row 1
  • event_type_code (CHECK ⊂ 24 equity order codes) — widest CHECK in the repo
  • 57 other spec mapping columns (firm/error/CAT identifiers, order key/timestamps, side/price/quantity, manual-event flags, parent/child linkage, RFQ linkage, NBBO snapshot, BFMM short-sale flag, routing destination/session/ISO, paired order, quote-context refs)
  • 4 lineage cols

Spec coverage

CAT IM v4.1.0r15 sections:

  • §4.1 New Order (MENO), §4.2 New Order Supplement (MENOS)
  • §4.3 Order Route family: Order Route (MEOR), Route Supplement (MEORS), Route Modified (MEMR), Modified Supplement (MEMRS), Route Cancelled (MECR), Cancel Supplement (MECRS)
  • §4.4 Order Accepted (MEOA)
  • §4.5 Internal Route family: Internal Route Accepted (MEIR), Modified (MEIM), Cancelled (MEIC), Modification Request (MEIMR), Cancel Request (MEICR)
  • §4.6 Child Order family: Child Order (MECO), Modified (MECOM), Cancelled (MECOC)
  • §4.7 Modified family: Order Modified (MEOM), Modified Supplement (MEOMS), Modification Request (MEOMR)
  • §4.8 Order Adjusted (MEOJ)
  • §4.9 Cancel family: Order Cancelled (MEOC), Cancel Request (MEOCR)
  • §4.14 Order Effective (MEOE)

Trade events (MEOT/MEOTS, §4.11.1/§4.11.2) intentionally excluded — already on fact_execution_events.

Why Option B over retrofit

Legacy fact_cat_order_events compressed the 59-field order lifecycle into ~12 generic columns. Only 7 spec columns overlap. Building fresh keeps each spec field as a discrete column for spec-fidelity, query ergonomics, and CAT JSON submission file generation.

Coverage

All 8 guardrails:                  pass (exit 0)
known_field_mapping_gaps.csv:    92 (was 151; -59)
  F3.1 phantom-table rows:        0 (was 120; ALL CLEARED across Tiers 13-16)
  F3.2 missing-column rows:      92 (unchanged - Tier 17 target)
DDL files in parity scope:        22 (unchanged)
Tables in 2+ dialects:            42 (was 41)
New parity violations:             0
SQL CHECK constraints validated:  74 (was 71)

WS2 burndown — COMPLETE

Sub-tier Phantom Cols Allowlist Status
Tier 13 fact_execution_events 11 212 → 201
Tier 14 fact_allocations 19 201 → 182
Tier 15 fact_quotes 31 182 → 151 ✅ (shipped with PR #16)
Tier 16 (this) fact_order_events 59 151 → 92
Phantom-table total 120 212 → 92

Test plan

  • All 8 guardrails pass (exit 0)
  • No new parity violations
  • 59 fact_order_events mapping rows now resolve in DDL
  • CHECK constraint covers all 24 equity order event codes (no MEOT — that's fact_execution_events)
  • CHECK constraint on action_type (NEW/FRC/RPR) per Section 4.1 row 1
  • Recurrence-safe push script (precheck for unexpected uncommitted files)
  • Replaces fact_cat_order_events semantically (24-code coverage vs 1-table legacy)

Final sub-tier of WS2 phantom-table burndown. Builds
gold.fact_order_events across all four dialects, appended to
the ddl/equity/ files created in Tiers 13-15.

Largest fact table in the equity Gold layer: 72 columns (59
spec + 9 framework + 4 lineage). Covers 24 distinct equity
order event codes across CAT IM v4.1.0r15 sections 4.1, 4.2,
4.3 (route family), 4.4, 4.5 (internal route family),
4.6 (child order family), 4.7 (modified family), 4.8, 4.9
(cancel family), 4.14 (Order Effective). Trade events
(MEOT/MEOTS) intentionally excluded - they live on
fact_execution_events.

Two CHECK constraints:
  - event_type_code IN (MENO, MENOS, MEOR, MEORS, MEMR,
    MEMRS, MECR, MECRS, MEOA, MEIR, MEIM, MEIC, MEIMR, MEICR,
    MECO, MECOM, MECOC, MEOM, MEOMS, MEOMR, MEOJ, MEOC, MEOCR,
    MEOE) -- 24 codes, widest CHECK in the repo
  - action_type IN (NEW, FRC, RPR) -- section 4.1 row 1

Replaces legacy fact_cat_order_events (which compressed
59 spec fields into ~12 generic columns; only 7 overlapped).

Validator update (guardrails/validate_check_constraints.py):
  - event_type_code mapped to cat_im_event_types.csv (same as
    cat_event_code; equity order tables use the column name
    from the field mapping CSV).
  - action_type added to KNOWN_UNMAPPED_COLUMNS (fixed 3-value
    spec enum, no separate primary-source CSV).

Allowlist:
  known_field_mapping_gaps.csv: 151 -> 92 rows (-59)
  F3.1 phantom-table rows:      0 (was 120; all cleared
  across Tiers 13-16)
  F3.2 missing-column rows:     92 (unchanged - next target)

Tables in 2+ dialects: 42 (was 41)

All 8 guardrails pass.
@jamezycesar-collab
jamezycesar-collab merged commit 37ec7b8 into main Jun 18, 2026
1 check passed
@jamezycesar-collab
jamezycesar-collab deleted the tier16-ws2-fact-order-events branch June 18, 2026 02:20
jamezycesar-collab added a commit that referenced this pull request Jun 18, 2026
)

First sub-tier of WS2 F3.2 follow-on (the 92 missing-column rows
remaining after the WS2 phantom-table burndown completed in Tier 16).

Adds 3 spec-mapping columns to gold.fact_option_executions across
all four dialects (Delta, Hive, Fabric Lakehouse, Fabric Warehouse):

  - trade_key_date       TIMESTAMP  CAT IM v4.1.0r15 sec 5.1.11 row 6 (MOOT)
  - fill_key_date        TIMESTAMP  CAT IM v4.1.0r15 sec 5.1.12.1 row 6 (MOOF/MOOFS)
  - prior_fill_key_date  TIMESTAMP  CAT IM v4.1.0r15 sec 5.1.12.3 row 9 (MOFA)

All three are natural-key timestamps grouped with the existing
trade_id, fulfillment_id, prior_fulfillment_id identifiers.

Mapping CSV section-ref corrections (atomic with the DDL):
  - fill_key_date:        sec 5.1.13.1 -> sec 5.1.12.1
  - prior_fill_key_date:  sec 5.1.14   -> sec 5.1.12.3
  Both were PDF-verified via verify_cat_im_v3.py against the
  CAT IM v4.1.0r15 PDF.

Allowlist:
  known_field_mapping_gaps.csv: 92 -> 89 rows (-3)
  F3.2 remaining: 89 (was 92)
  F3.1 phantom-table: 0 (unchanged, cleared in Tier 13-16)

CHANGELOG: removed leftover stash-pop merge-conflict markers that
were committed inadvertently with PR #18; both pre-conflict sections
(Tier 15 banner + Tier 16 entry) preserved.

All 8 guardrails pass.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant