jamezycesar-collab · jamezycesar-collab · Jun 18, 2026 · Jun 18, 2026
@@ -2,6 +2,7 @@
 
 All notable changes to the data model are documented here. The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).
 
+<<<<<<< Updated upstream
 ## [Unreleased] - Tier 15 (post-hoc): fact_quotes shipped with PR #16
 
 > The `fact_quotes` DDL and 31 corresponding allowlist deletions were physically
@@ -11,6 +12,72 @@ All notable changes to the data model are documented here. The format is based o
 > branch was empty and has been deleted. The merge commit `b9f5d8c` (PR #16)
 > physically contains both `fact_allocations` (Tier 14) and `fact_quotes`
 > (Tier 15). This banner records the bundle for audit-trail completeness.
+=======
+## [Unreleased] - Tier 16: WS2 burndown sub-tier 4 — fact_order_events DDL (final WS2 sub-tier)
+
+### Added
+
+- `gold.fact_order_events` across all four dialects (appended to existing Tier-13/14/15 `ddl/equity/` files):
+  - `ddl/equity/02_equity_gold_delta.sql`
+  - `ddl/equity/04_equity_gold_hive.sql`
+  - `ddl/equity/05_equity_gold_fabric_lakehouse.sql`
+  - `ddl/equity/06_equity_gold_fabric_warehouse.sql`
+
+  Largest fact table in the equity Gold layer: 72 columns total (59 spec + 9 framework + 4 lineage). PK + FK + two CHECK constraints:
+  - `event_type_code IN (24 codes)` — CAT IM v4.1.0r15 sections 4.1 New Order, 4.2 New Order Supplement, 4.3 Order Route family (MEOR/MEORS/MEMR/MEMRS/MECR/MECRS), 4.4 Order Accepted, 4.5 Internal Route family (MEIR/MEIM/MEIC/MEIMR/MEICR), 4.6 Child Order family (MECO/MECOM/MECOC), 4.7 Order Modified family (MEOM/MEOMS/MEOMR), 4.8 Order Adjusted (MEOJ), 4.9 Order Cancelled family (MEOC/MEOCR), 4.14 Order Effective (MEOE). Trade events (MEOT/MEOTS) intentionally excluded — they live on `fact_execution_events`.
+  - `action_type IN ('NEW', 'FRC', 'RPR')` — Section 4.1 row 1 firm-initiated correction vs error repair.
+
+  Replaces the legacy `fact_cat_order_events` (which compressed 59 spec fields into ~12 generic columns, only 7 overlapping).
+
+### Changed
+
+- `guardrails/known_field_mapping_gaps.csv` - 151 → 92 rows. **All 120 F3.1 phantom-table rows are now cleared.** Only F3.2 missing-column rows (92) remain.
+- `guardrails/validate_check_constraints.py`:
+  - Added `event_type_code` → `primary-sources/cat_im_event_types.csv` mapping (same target as `cat_event_code`; equity order tables use the column name from the field mapping CSV).
+  - Added `action_type` to `KNOWN_UNMAPPED_COLUMNS` (CAT IM §4.1 row 1 fixed 3-value enum: NEW/FRC/RPR — no separate primary-source CSV).
+
+### Why
+
+Final sub-tier of WS2 phantom-table burndown. `fact_order_events` was the largest phantom (59 mapping cols). Same Option B pattern: build the new spec-faithful design rather than retrofitting the legacy `fact_cat_order_events`. The legacy table flattened the entire order lifecycle into compact generic columns; the new `fact_order_events` keeps each spec field as a discrete column for spec-fidelity, downstream queries, and CAT JSON submission file generation.
+
+### Schema
+
+`fact_order_events` (72 columns):
+- `order_event_sk` BIGINT IDENTITY (PK)
+- `event_dts` / `event_date` (timestamps; partition key in non-Delta dialects)
+- 5 dim FKs: `date_sk`, `instrument_sk`, `party_sk`, `venue_sk` (nullable), `event_type_sk`
+- `action_type` (CHECK ⊂ {`NEW`, `FRC`, `RPR`})
+- `event_type_code` (CHECK ⊂ 24 equity order codes)
+- 57 other spec mapping columns (firm/error/CAT identifiers, order key/timestamps, side/price/quantity, manual-event flags, parent/child linkage, RFQ linkage, NBBO snapshot, BFMM/short-sale flag, routing destination/session/ISO indicator, paired order, quote-context refs)
+- 4 lineage cols
+
+### Coverage
+
+```
+Validators:                        8/8 pass
+known_field_mapping_gaps.csv:     92  (was 151)
+  - F3.1 phantom-table rows:       0  (was 120; -120 across Tiers 13-16)
+  - F3.2 missing-column rows:     92  (unchanged - next workstream target)
+DDL files in parity scope:        22  (unchanged - fact_order_events appended to existing equity/ files)
+Tables in 2+ dialects:            42  (was 41)
+New parity violations:             0
+SQL CHECK constraints validated:  74  (was 71; +3 = action_type x 3 dialects with constraint syntax)
+```
+
+### WS2 burndown — COMPLETE
+
+| Sub-tier | Phantom | Cols | Allowlist | Status |
+|---|---|---|---|---|
+| Tier 13 | `fact_execution_events` | 11 | 212 → 201 | ✅ |
+| Tier 14 | `fact_allocations` | 19 | 201 → 182 | ✅ |
+| Tier 15 | `fact_quotes` | 31 | 182 → 151 | ✅ (shipped with PR #16) |
+| **Tier 16 (this)** | `fact_order_events` | 59 | 151 → 92 | ✅ |
+| **Phantom-table total** | — | **120** | **212 → 92 (only F3.2 left)** | ✅ |
+
+### Next workstream
+
+F3.2 missing-column closure (~92 rows): add the absent columns to their host tables (mostly `fact_cat_order_events`, `fact_cat_quotes`, etc.). Will be sequenced as Tier 17.
+>>>>>>> Stashed changes
 
 ## [Unreleased] - Tier 15: WS2 burndown sub-tier 3 — fact_quotes DDL
 

@@ -223,3 +223,120 @@ TBLPROPERTIES (
     'cat_submission_file_type' = 'OrderEvents',
     'grain' = 'one row per equity quote event'
 );
+
+
+-- ----------------------------------------------------------------------------
+-- 4. fact_order_events  -- equity order lifecycle events
+--    CAT IM v4.1.0r15 sections 4.1 - 4.9 + 4.14 (24 distinct event codes)
+--    Tier 16 (WS2 sub-tier 4 - final phantom-table burndown)
+--
+--    Covers New Order, Order Route, Route Modified, Route Cancelled,
+--    Order Accepted, Internal Route family, Child Order family,
+--    Order Modified, Order Adjusted, Order Cancelled, Order Effective.
+--    Trade events (MEOT/MEOTS) intentionally excluded - they live on
+--    fact_execution_events.
+-- ----------------------------------------------------------------------------
+
+CREATE TABLE IF NOT EXISTS gold.fact_order_events (
+    order_event_sk              BIGINT GENERATED ALWAYS AS IDENTITY,
+    event_dts                   TIMESTAMP NOT NULL,
+    event_date                  DATE      NOT NULL,
+    -- conformed dimension FKs
+    date_sk                     BIGINT NOT NULL,
+    instrument_sk               BIGINT NOT NULL,
+    party_sk                    BIGINT NOT NULL,
+    venue_sk                    BIGINT,
+    event_type_sk               BIGINT NOT NULL,
+    -- equity order event specifics (CAT IM v4.1.0r15 sections 4.1 - 4.9 + 4.14)
+    action_type                 STRING NOT NULL,          -- Section 4.1 row 1 (NEW/FRC/RPR)
+    error_roe_id                STRING,                   -- Section 4.1 row 2 (required when RPR)
+    firm_roe_id                 STRING NOT NULL,          -- Section 2.3.3
+    event_type_code             STRING NOT NULL,          -- Section 6.1.5: message type code (CHECK)
+    cat_reporter_imid           STRING,                   -- Section 2.4.1
+    order_key_date              TIMESTAMP NOT NULL,       -- Section 4.1 row 6
+    cat_order_id                STRING NOT NULL,          -- Section 4.1 row 7
+    symbol                      STRING NOT NULL,          -- Section 2.4.3
+    event_timestamp             TIMESTAMP NOT NULL,       -- Section 4.1 row 9
+    manual_flag                 BOOLEAN NOT NULL,         -- Section 4.1 row 10
+    electronic_dup_flag         BOOLEAN NOT NULL,         -- Section 4.1 row 11
+    electronic_timestamp        TIMESTAMP,                -- Section 4.1 row 12
+    manual_order_key_date       TIMESTAMP,                -- Section 4.1 row 13
+    manual_order_id             STRING,                   -- Section 4.1 row 14
+    dept_type                   STRING,                   -- Section 4.1 row 15 (MENO/MEOM)
+    solicitation_flag           BOOLEAN,                  -- Section 4.1 row 16 (MENO)
+    rfq_id                      STRING,                   -- Section 4.1 row 17 (MENO)
+    side                        STRING,                   -- Section 4.1 row 18
+    price                       DECIMAL(38, 18),          -- Section 4.1 row 19
+    quantity                    DECIMAL(38, 18),          -- Section 4.1 row 20
+    leaves_quantity             DECIMAL(38, 18),          -- Section 4.1 / 4.7 / 4.11.1
+    parent_order_id             STRING,                   -- Section 4.1 parent linkage
+    min_qty                     DECIMAL(38, 18),          -- Section 4.1 row 21
+    order_type                  STRING,                   -- Section 4.1 row 22
+    time_in_force               STRING,                   -- Section 4.1 row 23
+    trading_session             STRING,                   -- Section 4.1 row 24
+    handling_instructions       STRING,                   -- Section 4.1 row 25 / 4.3 row 28
+    cust_dsp_intr_flag          BOOLEAN,                  -- Section 4.1 row 26
+    firm_designated_id          STRING,                   -- Section 2.4.2 / Appendix G
+    account_holder_type         STRING,                   -- Section 4.1 row 28
+    affiliate_flag              BOOLEAN,                  -- Section 4.1 row 29 / 4.3 row 26
+    info_barrier_id             STRING,                   -- Section 4.1 row 30
+    negotiated_trade_flag       BOOLEAN,                  -- Section 4.1 row 32
+    representative_ind          STRING,                   -- Section 4.1 row 33
+    seq_num                     STRING,                   -- Section 4.1 row 34
+    ats_display_ind             STRING,                   -- Section 4.1 row 35
+    display_price               DECIMAL(38, 18),          -- Section 4.1 row 36
+    working_price               DECIMAL(38, 18),          -- Section 4.1 row 37
+    display_qty                 DECIMAL(38, 18),          -- Section 4.1 row 38
+    nbb_price                   DECIMAL(38, 18),          -- Section 4.1 row 40
+    nbb_qty                     DECIMAL(38, 18),          -- Section 4.1 row 41
+    nbo_price                   DECIMAL(38, 18),          -- Section 4.1 row 42
+    nbo_qty                     DECIMAL(38, 18),          -- Section 4.1 row 43
+    nbbo_source                 STRING,                   -- Section 4.1 row 44
+    nbbo_timestamp              TIMESTAMP,                -- Section 4.1 row 45
+    net_price                   DECIMAL(38, 18),          -- Section 4.1 row 46
+    bfmm_flag                   BOOLEAN,                  -- Section 4.1 row 47 (short-sale BFMM)
+    originating_imid            STRING,                   -- Section 4.3 row 9
+    sender_imid                 STRING,                   -- Section 4.3 row 14
+    destination                 STRING,                   -- Section 4.3 row 15
+    destination_type            STRING,                   -- Section 4.3 row 16
+    routed_order_id             STRING,                   -- Section 4.3 row 17
+    session                     STRING,                   -- Section 4.3 row 18
+    iso_ind                     STRING,                   -- Section 4.3 row 27 (ISO)
+    route_rejected_flag         BOOLEAN,                  -- Section 4.3 row 29
+    multi_leg_ind               BOOLEAN,                  -- Section 4.3 row 32 / 4.11.1 row 35
+    paired_order_id             STRING,                   -- Section 4.3 row 33
+    quote_key_date              TIMESTAMP,                -- Section 4.3 row 36 (RFQ response)
+    quote_id                    STRING,                   -- Section 4.3 row 37 (RFQ response)
+    -- lineage
+    source_file                 STRING NOT NULL,
+    source_batch_id             STRING NOT NULL,
+    dv2_source_hk               STRING NOT NULL,
+    quality_outcome             STRING,
+    CONSTRAINT pk_fact_order_event PRIMARY KEY (order_event_sk),
+    CONSTRAINT chk_fact_order_code CHECK (event_type_code IN (
+        'MENO','MENOS','MEOR','MEORS','MEMR','MEMRS','MECR','MECRS',
+        'MEOA','MEIR','MEIM','MEIC','MEIMR','MEICR',
+        'MECO','MECOM','MECOC','MEOM','MEOMS','MEOMR',
+        'MEOJ','MEOC','MEOCR','MEOE'
+    )),
+    CONSTRAINT chk_fact_order_action CHECK (action_type IN ('NEW','FRC','RPR')),
+    CONSTRAINT fk_fact_oe_date     FOREIGN KEY (date_sk)
+        REFERENCES gold.dim_date (date_sk),
+    CONSTRAINT fk_fact_oe_instr    FOREIGN KEY (instrument_sk)
+        REFERENCES gold.dim_instrument (instrument_sk),
+    CONSTRAINT fk_fact_oe_party    FOREIGN KEY (party_sk)
+        REFERENCES gold.dim_party (party_sk),
+    CONSTRAINT fk_fact_oe_evtype   FOREIGN KEY (event_type_sk)
+        REFERENCES gold.dim_event_type (event_type_sk)
+)
+USING DELTA
+PARTITIONED BY (event_date)
+COMMENT 'CAT equity order event fact - one row per equity order lifecycle event (24 distinct codes from sections 4.1-4.9 + 4.14, excluding trades MEOT/MEOTS)'
+TBLPROPERTIES (
+    'delta.enableChangeDataFeed' = 'true',
+    'delta.autoOptimize.optimizeWrite' = 'true',
+    'compression.codec' = 'zstd',
+    'subject_area' = 'gold_fact',
+    'cat_submission_file_type' = 'OrderEvents',
+    'grain' = 'one row per equity order lifecycle event'
+);
@@ -147,3 +147,88 @@ CREATE TABLE IF NOT EXISTS gold.fact_quotes (
 PARTITIONED BY (event_date DATE)
 STORED AS PARQUET
 TBLPROPERTIES ('comment' = 'Gold fact: one row per equity quote event (MENQ/MENQS/MERQ/MERQS/MEQR/MEQC/MEQM/MEQS).');
+
+
+-- ----------------------------------------------------------------------------
+-- 4. fact_order_events  -- equity order lifecycle events (24 codes)
+--    Tier 16 (WS2 sub-tier 4 - final phantom-table burndown)
+-- ----------------------------------------------------------------------------
+
+CREATE TABLE IF NOT EXISTS gold.fact_order_events (
+    order_event_sk              BIGINT,
+    event_dts                   TIMESTAMP,
+    -- conformed dimension FKs
+    date_sk                     BIGINT,
+    instrument_sk               BIGINT,
+    party_sk                    BIGINT,
+    venue_sk                    BIGINT,
+    event_type_sk               BIGINT,
+    -- equity order event specifics (CAT IM v4.1.0r15 sections 4.1 - 4.9 + 4.14)
+    action_type                 STRING COMMENT 'CHECK in (NEW, FRC, RPR)',
+    error_roe_id                STRING,
+    firm_roe_id                 STRING,
+    event_type_code             STRING COMMENT 'CHECK in (MENO, MENOS, MEOR, MEORS, MEMR, MEMRS, MECR, MECRS, MEOA, MEIR, MEIM, MEIC, MEIMR, MEICR, MECO, MECOM, MECOC, MEOM, MEOMS, MEOMR, MEOJ, MEOC, MEOCR, MEOE)',
+    cat_reporter_imid           STRING,
+    order_key_date              TIMESTAMP,
+    cat_order_id                STRING,
+    symbol                      STRING,
+    event_timestamp             TIMESTAMP,
+    manual_flag                 BOOLEAN,
+    electronic_dup_flag         BOOLEAN,
+    electronic_timestamp        TIMESTAMP,
+    manual_order_key_date       TIMESTAMP,
+    manual_order_id             STRING,
+    dept_type                   STRING,
+    solicitation_flag           BOOLEAN,
+    rfq_id                      STRING,
+    side                        STRING,
+    price                       DECIMAL(38, 18),
+    quantity                    DECIMAL(38, 18),
+    leaves_quantity             DECIMAL(38, 18),
+    parent_order_id             STRING,
+    min_qty                     DECIMAL(38, 18),
+    order_type                  STRING,
+    time_in_force               STRING,
+    trading_session             STRING,
+    handling_instructions       STRING,
+    cust_dsp_intr_flag          BOOLEAN,
+    firm_designated_id          STRING,
+    account_holder_type         STRING,
+    affiliate_flag              BOOLEAN,
+    info_barrier_id             STRING,
+    negotiated_trade_flag       BOOLEAN,
+    representative_ind          STRING,
+    seq_num                     STRING,
+    ats_display_ind             STRING,
+    display_price               DECIMAL(38, 18),
+    working_price               DECIMAL(38, 18),
+    display_qty                 DECIMAL(38, 18),
+    nbb_price                   DECIMAL(38, 18),
+    nbb_qty                     DECIMAL(38, 18),
+    nbo_price                   DECIMAL(38, 18),
+    nbo_qty                     DECIMAL(38, 18),
+    nbbo_source                 STRING,
+    nbbo_timestamp              TIMESTAMP,
+    net_price                   DECIMAL(38, 18),
+    bfmm_flag                   BOOLEAN,
+    originating_imid            STRING,
+    sender_imid                 STRING,
+    destination                 STRING,
+    destination_type            STRING,
+    routed_order_id             STRING,
+    session                     STRING,
+    iso_ind                     STRING,
+    route_rejected_flag         BOOLEAN,
+    multi_leg_ind               BOOLEAN,
+    paired_order_id             STRING,
+    quote_key_date              TIMESTAMP,
+    quote_id                    STRING,
+    -- lineage
+    source_file                 STRING,
+    source_batch_id             STRING,
+    dv2_source_hk               STRING,
+    quality_outcome             STRING
+)
+PARTITIONED BY (event_date DATE)
+STORED AS PARQUET
+TBLPROPERTIES ('comment' = 'Gold fact: one row per equity order lifecycle event (24 codes from sections 4.1-4.9 + 4.14, excluding trades MEOT/MEOTS).');