Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
63 changes: 63 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,69 @@

All notable changes to the data model are documented here. The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).

## [Unreleased] - Tier 17.3: F3.2 closure sub-tier 3 — fact_multileg_option_legs columns

### Added

- 3 new spec-mapping columns to `gold.fact_multileg_option_legs` across all three dialects (multileg Gold has no Fabric Lakehouse variant):
- `ddl/multileg/02_multileg_gold_delta.sql`
- `ddl/multileg/04_multileg_fabric_warehouse.sql`
- `ddl/multileg/06_multileg_gold_hive.sql`

Columns: `leg_ref_id` (STRING/VARCHAR(64), legRefID per CAT IM v4.1.0r15 §5.2.1 row 32.n.1), `leg_symbol` (STRING/VARCHAR(14), §5.2.1 row 32.n.2), `leg_option_id` (STRING/VARCHAR(22), §5.2.1 row 32.n.3). All three are leg-array-indexed (`legs[n].<field>` in the JSON submission).

### Changed

- Renamed 2 legacy column names to match the canonical CAT IM spec naming:
- `leg_open_close` → `leg_open_close_indicator` (matches §5.2.1 row 32.n.4 `openCloseIndicator`)
- `leg_ratio` → `leg_ratio_quantity` (matches §5.2.1 row 32.n.6 `legRatioQuantity`)
- `gold.vw_multileg_option_lifecycle` view (in same files): updated to reference the renamed columns + added the 3 new leg columns to the SELECT list.
- `guardrails/known_field_mapping_gaps.csv`: 86 → 81 rows (5 `fact_multileg_option_legs` rows removed).

### Why

Third sub-tier of WS2 F3.2 follow-on. The mapping CSV's 6 leg-array columns (`leg_ref_id`, `leg_symbol`, `leg_option_id`, `leg_open_close_indicator`, `leg_side`, `leg_ratio_quantity`) didn't all exist in the DDL: 2 had legacy names (`leg_open_close`, `leg_ratio`) and 3 were genuinely missing. Renamed + added per CAT IM v4.1.0r15 §5.2.1's `legs[]` sub-table.

### Spec verification

All columns verified against PDF page 289 (MLNO §5.2.1 row 32.n.1–6) where the legs[] sub-table is documented:
```
32.n.1 legRefID Text(64) Unique identifier of the leg
32.n.2 symbol Symbol The symbol of the stock for the leg
32.n.3 optionID Text(22) The 21-character OSI Symbol of the option
32.n.4 openCloseIndicator Choice Opens or closes a position
32.n.5 side Choice Side of the leg
32.n.6 legRatioQuantity Real Ratio of quantity for this individual leg
```

The earlier verifier's `JSON_FIELD_NOT_IN_PDF` flag on these columns was a regex false negative — the spec uses the multi-leg leg-indexed row format `"32.n.N"` that the verifier's `^\d+ <field>` row-prefix regex didn't catch.

### Coverage

```
Validators: 8/8 pass
known_field_mapping_gaps.csv: 81 (was 86; -5 fact_multileg_option_legs rows)
- F3.1 phantom-table rows: 0 (unchanged - cleared in Tier 13-16)
- F3.2 missing-column rows: 81 (was 86)
DDL files in parity scope: 22 (unchanged)
Tables in 2+ dialects: 42 (unchanged - same tables, new columns)
New parity violations: 0
```

### F3.2 burndown progress

| Sub-tier | Host table | Cols | Allowlist | Status |
|---|---|---|---|---|
| Tier 17.1 | `fact_option_executions` | 3 | 92 → 89 | ✅ |
| Tier 17.2 | `fact_option_allocations` | 3 | 89 → 86 | ✅ |
| **Tier 17.3 (this)** | `fact_multileg_option_legs` | 5 | 86 → 81 | ✅ |
| Tier 17.4 | `fact_option_order_events` | 30 | 81 → 51 | next |
| Tier 17.5 | `fact_multileg_option_events` | 51 | 51 → 0 | queued |

### Note on column-rename backwards compatibility

The renames `leg_open_close` → `leg_open_close_indicator` and `leg_ratio` → `leg_ratio_quantity` affect only `gold.fact_multileg_option_legs` and its consuming view `gold.vw_multileg_option_lifecycle` (both updated in the same files). Silver-layer tables continue to use the original names; their Silver→Gold pipeline mappings handle the rename. No external consumers identified — this is fresh Gold-layer DDL prior to deployment.

## [Unreleased] - Tier 17.2: F3.2 closure sub-tier 2 — fact_option_allocations columns

### Added
Expand Down
21 changes: 14 additions & 7 deletions ddl/multileg/02_multileg_gold_delta.sql
Original file line number Diff line number Diff line change
Expand Up @@ -82,13 +82,17 @@ COMMENT 'Gold fact: header-grain row per multi-leg CAT event (Section 5.2 of IM
CREATE TABLE IF NOT EXISTS gold.fact_multileg_option_legs (
multileg_leg_event_sk BIGINT GENERATED ALWAYS AS IDENTITY,
multileg_event_sk BIGINT NOT NULL, -- FK to header
leg_seq INT NOT NULL,
instrument_sk BIGINT NOT NULL, -- the option or covered stock for this leg
leg_side STRING NOT NULL, -- BUY / SELL
leg_open_close STRING, -- O / C / NULL for stock
leg_seq INT NOT NULL, -- the leg index n in CAT IM legs array
instrument_sk BIGINT NOT NULL, -- FK to option contract or covered stock
-- CAT IM v4.1.0r15 section 5.2.1 row 32 legs sub-table: rows 32.n.1 - 32.n.6
leg_ref_id STRING, -- section 5.2.1 row 32.n.1 legRefID unique identifier of leg
leg_symbol STRING, -- section 5.2.1 row 32.n.2 symbol of stock leg
leg_option_id STRING, -- section 5.2.1 row 32.n.3 optionID OSI symbol of option leg
leg_open_close_indicator STRING, -- section 5.2.1 row 32.n.4 openCloseIndicator O C NULL
leg_side STRING NOT NULL, -- section 5.2.1 row 32.n.5 side BUY SELL
leg_ratio_quantity DECIMAL(10, 4) NOT NULL, -- section 5.2.1 row 32.n.6 legRatioQuantity ratio
leg_quantity DECIMAL(38, 18) NOT NULL,
leg_price DECIMAL(38, 18),
leg_ratio DECIMAL(10, 4) NOT NULL,
leg_status STRING,
-- lineage
source_file STRING NOT NULL,
Expand Down Expand Up @@ -120,11 +124,14 @@ SELECT
et.event_code,
et.event_name,
l.leg_seq,
l.leg_ref_id,
l.leg_symbol,
l.leg_option_id,
l.leg_side,
l.leg_open_close,
l.leg_open_close_indicator,
l.leg_quantity,
l.leg_price,
l.leg_ratio,
l.leg_ratio_quantity,
i.instrument_bk AS leg_instrument
FROM gold.fact_multileg_option_events h
JOIN gold.fact_multileg_option_legs l ON h.multileg_event_sk = l.multileg_event_sk
Expand Down
21 changes: 14 additions & 7 deletions ddl/multileg/04_multileg_fabric_warehouse.sql
Original file line number Diff line number Diff line change
Expand Up @@ -186,13 +186,17 @@ CREATE TABLE gold.fact_multileg_option_events (
CREATE TABLE gold.fact_multileg_option_legs (
multileg_leg_event_sk BIGINT IDENTITY(1,1) NOT NULL,
multileg_event_sk BIGINT NOT NULL,
leg_seq INT NOT NULL,
instrument_sk BIGINT NOT NULL,
leg_side VARCHAR(8) NOT NULL,
leg_open_close VARCHAR(2),
leg_seq INT NOT NULL, -- the leg index n in CAT IM legs array
instrument_sk BIGINT NOT NULL, -- FK to option contract or covered stock
-- CAT IM v4.1.0r15 section 5.2.1 row 32 legs sub-table: rows 32.n.1 - 32.n.6
leg_ref_id VARCHAR(64), -- section 5.2.1 row 32.n.1 legRefID unique identifier of leg
leg_symbol VARCHAR(14), -- section 5.2.1 row 32.n.2 symbol of stock leg
leg_option_id VARCHAR(22), -- section 5.2.1 row 32.n.3 optionID OSI symbol of option leg
leg_open_close_indicator VARCHAR(2), -- section 5.2.1 row 32.n.4 openCloseIndicator O C NULL
leg_side VARCHAR(8) NOT NULL, -- section 5.2.1 row 32.n.5 side BUY SELL
leg_ratio_quantity DECIMAL(10, 4) NOT NULL, -- section 5.2.1 row 32.n.6 legRatioQuantity ratio
leg_quantity DECIMAL(38, 18) NOT NULL,
leg_price DECIMAL(38, 18),
leg_ratio DECIMAL(10, 4) NOT NULL,
leg_status VARCHAR(32),
source_file VARCHAR(256) NOT NULL,
source_batch_id VARCHAR(64) NOT NULL,
Expand All @@ -217,11 +221,14 @@ SELECT
et.event_code,
et.event_name,
l.leg_seq,
l.leg_ref_id,
l.leg_symbol,
l.leg_option_id,
l.leg_side,
l.leg_open_close,
l.leg_open_close_indicator,
l.leg_quantity,
l.leg_price,
l.leg_ratio,
l.leg_ratio_quantity,
i.instrument_bk AS leg_instrument
FROM gold.fact_multileg_option_events h
JOIN gold.fact_multileg_option_legs l ON h.multileg_event_sk = l.multileg_event_sk
Expand Down
21 changes: 14 additions & 7 deletions ddl/multileg/06_multileg_gold_hive.sql
Original file line number Diff line number Diff line change
Expand Up @@ -57,13 +57,17 @@ TBLPROPERTIES ('comment' = 'Gold fact: header-grain row per multi-leg CAT event
CREATE TABLE IF NOT EXISTS gold.fact_multileg_option_legs (
multileg_leg_event_sk BIGINT,
multileg_event_sk BIGINT COMMENT 'FK to fact_multileg_option_events.multileg_event_sk',
leg_seq INT,
instrument_sk BIGINT COMMENT 'FK to dim_instrument; the option or covered stock for this leg',
leg_side STRING COMMENT 'CHECK in (BUY, SELL)',
leg_open_close STRING COMMENT 'O | C | NULL for stock leg',
leg_seq INT COMMENT 'the leg index n in CAT IM legs array',
instrument_sk BIGINT COMMENT 'FK to dim_instrument; option contract or covered stock',
-- CAT IM v4.1.0r15 section 5.2.1 row 32 legs sub-table: rows 32.n.1 - 32.n.6
leg_ref_id STRING COMMENT 'section 5.2.1 row 32.n.1 legRefID unique identifier of leg',
leg_symbol STRING COMMENT 'section 5.2.1 row 32.n.2 symbol of stock leg',
leg_option_id STRING COMMENT 'section 5.2.1 row 32.n.3 optionID OSI symbol of option leg',
leg_open_close_indicator STRING COMMENT 'section 5.2.1 row 32.n.4 openCloseIndicator O C NULL',
leg_side STRING COMMENT 'section 5.2.1 row 32.n.5 side BUY SELL',
leg_ratio_quantity DECIMAL(10, 4) COMMENT 'section 5.2.1 row 32.n.6 legRatioQuantity ratio',
leg_quantity DECIMAL(38, 18),
leg_price DECIMAL(38, 18),
leg_ratio DECIMAL(10, 4),
leg_status STRING,
-- lineage
source_file STRING,
Expand All @@ -90,11 +94,14 @@ SELECT
et.event_code,
et.event_name,
l.leg_seq,
l.leg_ref_id,
l.leg_symbol,
l.leg_option_id,
l.leg_side,
l.leg_open_close,
l.leg_open_close_indicator,
l.leg_quantity,
l.leg_price,
l.leg_ratio,
l.leg_ratio_quantity,
i.instrument_bk AS leg_instrument
FROM gold.fact_multileg_option_events h
JOIN gold.fact_multileg_option_legs l ON h.multileg_event_sk = l.multileg_event_sk
Expand Down
5 changes: 0 additions & 5 deletions guardrails/known_field_mapping_gaps.csv
Original file line number Diff line number Diff line change
Expand Up @@ -80,8 +80,3 @@ fact_multileg_option_events,routed_quote_id,missing_column,F3.2,247
fact_multileg_option_events,quote_rejected_flag,missing_column,F3.2,248
fact_multileg_option_events,prior_quote_key_date,missing_column,F3.2,249
fact_multileg_option_events,prior_quote_id,missing_column,F3.2,250
fact_multileg_option_legs,leg_ref_id,missing_column,F3.2,251
fact_multileg_option_legs,leg_symbol,missing_column,F3.2,252
fact_multileg_option_legs,leg_option_id,missing_column,F3.2,253
fact_multileg_option_legs,leg_open_close_indicator,missing_column,F3.2,254
fact_multileg_option_legs,leg_ratio_quantity,missing_column,F3.2,256
Loading