Tier 17.4: F3.2 sub-tier 4 — fact_option_order_events columns (30 cols) - #22
Merged
Merged
Conversation
… (30 cols)
Fourth sub-tier of WS2 F3.2 follow-on. Largest single addition
in the F3.2 closure: 30 spec-mapping columns added to
gold.fact_option_order_events across all four dialects (Delta,
Hive, Fabric Lakehouse, Fabric Warehouse).
Column groups:
- Order key + timestamps (2): order_key_date, event_timestamp
- Manual event flags (5): manual_flag, manual_order_key_date,
manual_order_id, electronic_dup_flag, electronic_timestamp
- Order metadata (6): dept_type, min_qty, solicitation_flag,
rfq_id, representative_ind, exch_origin_code
- Party / account (3): firm_designated_id, account_holder_type,
affiliate_flag
- Routing (11): sender_imid, receiver_imid, sender_type,
originating_imid, destination, destination_type,
routed_order_id, session, route_rejected_flag, multi_leg_ind,
paired_order_id
- Modify / parent linkage (3): prior_order_key_date,
parent_order_key_date, request_timestamp
All 30 columns PDF-verified against CAT IM v4.1.0r15 sections
5.1.1, 5.1.3, 5.1.4, 5.1.6.1, and 5.1.7. No mapping CSV
section-ref corrections needed (all 30 passed verification
with OK status).
Mirrors the equity fact_order_events structure introduced in
Tier 16 (30 cols here is a subset of the 59 in equity equivalent).
Lesson-learned (tooling debt tracked separately): Hive variant
uses '-- section ...' line comments rather than per-column
COMMENT 'string' clauses, because the validator's CREATE TABLE
regex treats COMMENT as a body terminator and silently truncated
column discovery on first attempt.
Allowlist:
known_field_mapping_gaps.csv: 81 -> 51 rows (-30)
F3.2 remaining: 51 (was 81; only fact_multileg_option_events left)
F3.1 phantom-table: 0 (unchanged, cleared in Tier 13-16)
All 8 guardrails pass.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fourth sub-tier of WS2 F3.2 follow-on. Adds 30 spec-mapping columns to
fact_option_order_eventsacross all four dialects.Files changed
ddl/option/02_option_gold_delta.sql- 30 columns addedddl/option/04_option_gold_hive.sql- 30 columns added (Hive line comments to avoid validator COMMENT terminator)ddl/option/05_option_silver_fabric_lakehouse.sql- 30 columns addedddl/option/06_option_fabric_warehouse.sql- 30 columns addedguardrails/known_field_mapping_gaps.csv- 81 → 51 rowsCHANGELOG.md- Tier 17.4 entry30 columns added (PDF-verified)
order_key_date,event_timestampmanual_flag,manual_order_key_date,manual_order_id,electronic_dup_flag,electronic_timestampdept_type,min_qty,solicitation_flag,rfq_id,representative_ind,exch_origin_codefirm_designated_id,account_holder_type,affiliate_flagsender_imid,receiver_imid,sender_type,originating_imid,destination,destination_type,routed_order_id,session,route_rejected_flag,multi_leg_ind,paired_order_idprior_order_key_date,parent_order_key_date,request_timestampNo mapping CSV fixes
All 30 columns passed PDF verification with
OKstatus (verifier output inTIER_17_VERIFICATION.csv). The mapping CSV's section refs are already correct.Validator quirk found and worked around
The validator's
_CREATE_TABLE_REregex includesCOMMENTin its terminator alternation. When the Hive variant initially used per-columnCOMMENT 'section ...'documentation, the regex misinterpreted the first column-levelCOMMENTappearing after aDECIMAL(38, 18))close-paren as the table-level terminator — silently truncating column discovery and triggering falseparityerrors.Workaround in this PR: Hive uses
-- section ...line comments instead of inlineCOMMENT 'string'. Delta / Fabric Lakehouse / Fabric Warehouse variants use--end-of-line comments (their conventional style anyway).Followup (tooling debt): harden the validator's regex to require column-list close-paren before considering
COMMENTas a body terminator. Tracking outside this PR.Coverage
F3.2 burndown progress
fact_option_executionsfact_option_allocationsfact_multileg_option_legsfact_option_order_eventsfact_multileg_option_eventsTest plan
fact_option_order_eventsmapping rows now resolve in DDL