Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
63 changes: 56 additions & 7 deletions crates/omnigraph-cli/src/main.rs
Original file line number Diff line number Diff line change
Expand Up @@ -1035,14 +1035,48 @@ fn render_schema_plan_step(step: &SchemaMigrationStep) -> String {
type_name,
render_annotations(annotations)
),
SchemaMigrationStep::DropType {
type_kind,
name,
mode,
} => format!(
"drop {} type '{}' ({} mode)",
schema_type_kind_label(*type_kind),
name,
drop_mode_label(*mode),
),
SchemaMigrationStep::DropProperty {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2: According to linked Linear issue MR-694, schema plan output should surface code-tagged/classified diagnostics per step, but the new DropProperty output only shows (soft mode) and omits code/tier.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At crates/omnigraph-cli/src/main.rs, line 1048:

<comment>According to linked Linear issue MR-694, schema plan output should surface code-tagged/classified diagnostics per step, but the new `DropProperty` output only shows `(soft mode)` and omits code/tier.</comment>

<file context>
@@ -1035,14 +1035,48 @@ fn render_schema_plan_step(step: &SchemaMigrationStep) -> String {
+            name,
+            drop_mode_label(*mode),
+        ),
+        SchemaMigrationStep::DropProperty {
+            type_kind,
+            type_name,
</file context>

type_kind,
type_name,
property_name,
mode,
} => format!(
"drop property '{}.{}' of {} '{}' ({} mode)",
type_name,
property_name,
schema_type_kind_label(*type_kind),
type_name,
drop_mode_label(*mode),
),
SchemaMigrationStep::UnsupportedChange {
entity,
reason,
code,
} => match code {
Some(c) => format!("unsupported change on {} [{}]: {}", entity, c, reason),
None => format!("unsupported change on {}: {}", entity, reason),
},
entity, reason, ..
} => {
// When a schema-lint code is attached, render code + tier
// so operators see at-a-glance the kind of risk (destructive
// / validated / safe) — not just the rule identifier.
// Reach the diagnostic via the `diagnostic()` helper so the
// CLI doesn't need to know how the lookup works.
match step.diagnostic() {
Some(diag) => format!(
"unsupported change on {} [{}, {}]: {}",
entity,
diag.code,
schema_lint_tier_label(diag.tier),
reason,
),
None => format!("unsupported change on {}: {}", entity, reason),
}
}
}
}

Expand All @@ -1054,6 +1088,21 @@ fn schema_type_kind_label(kind: omnigraph_compiler::SchemaTypeKind) -> &'static
}
}

fn schema_lint_tier_label(tier: omnigraph_compiler::SafetyTier) -> &'static str {
match tier {
omnigraph_compiler::SafetyTier::Safe => "safe",
omnigraph_compiler::SafetyTier::Validated => "validated",
omnigraph_compiler::SafetyTier::Destructive => "destructive",
}
}

fn drop_mode_label(mode: omnigraph_compiler::DropMode) -> &'static str {
match mode {
omnigraph_compiler::DropMode::Soft => "soft",
omnigraph_compiler::DropMode::Hard => "hard",
}
}

fn render_prop_type(prop_type: &omnigraph_compiler::PropType) -> String {
let base = if let Some(values) = &prop_type.enum_values {
format!("Enum({})", values.join("|"))
Expand Down
204 changes: 192 additions & 12 deletions crates/omnigraph-compiler/src/catalog/schema_plan.rs
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,29 @@ pub enum SchemaTypeKind {
Edge,
}

/// How a drop step interacts with data.
///
/// - **`Soft`** — catalog tombstone only. The type / property is hidden
/// from queries but the underlying Lance column / dataset is retained
/// on disk. Reversible via `omnigraph schema unhide` (forthcoming).
/// Tier: `safe`.
/// - **`Hard`** — actual data removal. The Lance column is rewritten
/// without the property, or the Lance dataset is dropped. Irreversible
/// short of branch / snapshot restore. Tier: `destructive`; requires
/// `--allow-data-loss` to apply.
///
/// The planner emits `Soft` by default; `--allow-data-loss` on the apply
/// CLI promotes drops to `Hard`. This is the dimension orthogonal to
/// `SafetyTier` from the schema-lint chassis (`crate::lint`): tier
/// describes the rule's class; mode describes the operator's intent for
/// data treatment.
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]
#[serde(rename_all = "snake_case")]
pub enum DropMode {
Soft,
Hard,
}

#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)]
pub struct SchemaMigrationPlan {
pub supported: bool,
Expand Down Expand Up @@ -62,6 +85,28 @@ pub enum SchemaMigrationStep {
property_name: String,
annotations: Vec<Annotation>,
},
/// Remove a node or edge type. Soft mode tombstones in the catalog
/// and retains data on disk; Hard mode drops the Lance dataset and
/// requires `--allow-data-loss`.
///
/// Dormant in this commit — emitted by the planner in a later
/// commit (see `docs/schema-lint-v1-plan.md`).

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P3: Custom agent: Flag AI Slop and Fabricated Changes

Comments reference nonexistent docs path docs/schema-lint-v1-plan.md instead of docs/dev/schema-lint-v1-plan.md

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At crates/omnigraph-compiler/src/catalog/schema_plan.rs, line 93:

<comment>Comments reference nonexistent docs path `docs/schema-lint-v1-plan.md` instead of `docs/dev/schema-lint-v1-plan.md`</comment>

<file context>
@@ -62,6 +85,28 @@ pub enum SchemaMigrationStep {
+    /// requires `--allow-data-loss`.
+    ///
+    /// Dormant in this commit — emitted by the planner in a later
+    /// commit (see `docs/schema-lint-v1-plan.md`).
+    DropType {
+        type_kind: SchemaTypeKind,
</file context>
Suggested change
/// commit (see `docs/schema-lint-v1-plan.md`).
+ /// commit (see `docs/dev/schema-lint-v1-plan.md`).

DropType {
type_kind: SchemaTypeKind,
name: String,
mode: DropMode,
},
/// Remove a property from an existing type. Soft mode tombstones
/// the property in the catalog and retains the Lance column; Hard
/// mode rewrites the column out and requires `--allow-data-loss`.
///
/// Dormant in this commit.
DropProperty {
type_kind: SchemaTypeKind,
type_name: String,
property_name: String,
mode: DropMode,
},
UnsupportedChange {
entity: String,
reason: String,
Expand Down Expand Up @@ -93,6 +138,24 @@ impl SchemaMigrationStep {
_ => None,
}
}

/// If this step carries a schema-lint code, return the full
/// catalog entry — including family, safety tier, and default
/// severity. Used by renderers that want to display richer
/// context than just the code string (e.g. `omnigraph schema
/// plan` annotating each line with its tier).
///
/// Returns `None` for steps that carry no code (the 12 of 17
/// `UnsupportedChange` paths still untagged in v0, plus every
/// non-`UnsupportedChange` variant).
pub fn diagnostic(&self) -> Option<&'static crate::lint::DiagnosticCode> {
match self {
Self::UnsupportedChange {
code: Some(c), ..
} => crate::lint::lookup(c),
_ => None,

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2: diagnostic() does not map DropProperty to a stable code, so soft-drop steps lose code-tagged lint metadata (e.g., OG-DS-104) needed for MR-694-style severity/suppression and plan diagnostics.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At crates/omnigraph-compiler/src/catalog/schema_plan.rs, line 156:

<comment>`diagnostic()` does not map `DropProperty` to a stable code, so soft-drop steps lose code-tagged lint metadata (e.g., OG-DS-104) needed for MR-694-style severity/suppression and plan diagnostics.</comment>

<file context>
@@ -93,6 +138,24 @@ impl SchemaMigrationStep {
+            Self::UnsupportedChange {
+                code: Some(c), ..
+            } => crate::lint::lookup(c),
+            _ => None,
+        }
+    }
</file context>
Suggested change
_ => None,
Self::DropProperty { .. } => Some(&crate::lint::codes::OG_DS_104),
_ => None,

}
}
}

pub fn plan_schema_migration(
Expand Down Expand Up @@ -499,18 +562,22 @@ fn plan_properties(
.iter()
.filter(|property| !consumed.contains(&property.name))
{
steps.push(SchemaMigrationStep::UnsupportedChange {
entity: format!(
"{}:{}.{}",
schema_type_kind_key(type_kind),
type_name,
leftover.name
),
reason: format!(
"removing property '{}.{}' is not supported in schema migration v1",
type_name, leftover.name
),
code: Some(crate::lint::codes::OG_DS_104.code.to_string()),
// Property removed from the desired schema: emit
// DropProperty { Soft } per docs/schema-lint-v1-plan.md
// commit #3. The Soft mode reuses the existing
// stage_overwrite rewrite path — batch_for_schema_apply_rewrite
// iterates target_schema.fields(), so the dropped column is
// naturally projected away. The prior Lance version retains
// the column until cleanup_old_versions runs, matching the
// OG-DS-104 destructive-tier expectation that data remains
// recoverable via time travel until cleanup. Hard mode (with
// immediate compact_files + cleanup_old_versions) lands in
// commit #5, gated by --allow-data-loss.
steps.push(SchemaMigrationStep::DropProperty {
type_kind,
type_name: type_name.to_string(),
property_name: leftover.name.clone(),
mode: DropMode::Soft,
});
Comment on lines +576 to 581

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔴 DropProperty on interfaces crashes apply with 'missing source table' because interfaces have no Lance dataset

The plan_properties function is called for interfaces (schema_plan.rs:197), and the leftover-property branch now emits DropProperty { type_kind: Interface, ... } at schema_plan.rs:576-581. In the apply path at schema_apply.rs:162, this produces table key "interface:<Name>" and inserts it into rewritten_tables. However, interfaces are schema-level constructs with no Lance dataset or manifest entry. When the rewrite loop at schema_apply.rs:365 calls snapshot.entry("interface:..."), it returns None and the apply fails with a confusing "missing source table 'interface:...' for schema apply" error.

Under the old code, interface property removal emitted UnsupportedChange, so supported was false and the apply was rejected before reaching the rewrite loop. The new DropProperty { Soft } bypasses the supported check (schema_plan.rs:176-178 only checks for UnsupportedChange), allowing apply to proceed into the crash. Any schema migration that removes a property from an interface will hit this.

Prompt for agents
The plan_properties function is called for interfaces (SchemaTypeKind::Interface) at plan_interfaces (schema_plan.rs:197). When a property is removed from an interface, the leftover branch now emits DropProperty { type_kind: Interface, ... }. But the apply path in schema_apply.rs treats DropProperty by adding the table key to rewritten_tables — and interfaces have no Lance dataset or manifest entry, so the rewrite loop fails with 'missing source table'.

Two possible fixes:

1. In the leftover-property branch of plan_properties (around line 576), check if type_kind is Interface and emit UnsupportedChange instead of DropProperty for interface properties. This preserves the old behavior for interfaces while allowing node/edge property drops.

2. In the apply path's DropProperty handler in schema_apply.rs (around line 141), skip adding to rewritten_tables when type_kind is Interface, since interfaces don't have physical datasets. Interface property drops would be a catalog-only change.

Option 1 is safer for now since interface property semantics are inherited by implementing nodes, and dropping an interface property has cascading effects on all implementing node types that would also need handling.
Open in Devin Review

Was this helpful? React with 👍 or 👎 to provide feedback.

Comment on lines +576 to 581

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔴 User-facing schema-lint docs not updated after property-drop behavior change (AGENTS.md Rule 1 violation)

docs/user/schema-lint.md:10 says [OG-DS-104] removing property 'Person.age' is not supported … appears in error messages, and line 34 lists OG-DS-104 as an error that blocks apply. After this PR's change, property drops now succeed via DropProperty { Soft }OG-DS-104 is no longer emitted at all. AGENTS.md Rule 1 states: "New endpoint, query function, CLI flag, env var, constant, schema construct, or invariant: update both the source code and the doc in the same change. Never split documentation drift into a follow-up." This is a user-visible behavioral change (property drops went from rejected to allowed) without a corresponding doc update.

Prompt for agents
The behavioral change at schema_plan.rs:576-581 means property drops are now allowed (soft mode) instead of rejected with OG-DS-104. The following user-facing docs need updating in this same PR:

1. docs/user/schema-lint.md line 10: The example error message '[OG-DS-104] removing property ...' no longer appears because the operation now succeeds. Update to describe the new soft-drop behavior.

2. docs/user/schema-lint.md line 34: The OG-DS-104 row should note it is no longer emitted for property drops (or update to reflect that soft drops are now the default behavior).

3. docs/user/schema-language.md line 78: The list of SchemaMigrationStep variants only mentions UnsupportedChange. Add DropProperty (and DropType as dormant).

4. crates/omnigraph-compiler/src/lint/codes.rs line 119: EMITTED_IN_V0 includes 'OG-DS-104' which is no longer emitted as an UnsupportedChange code. Remove it from the list or rename the constant to reflect the current state.
Open in Devin Review

Was this helpful? React with 👍 or 👎 to provide feedback.

}

Expand Down Expand Up @@ -863,6 +930,67 @@ node Account @rename_from("User") {
}));
}

#[test]
fn plan_emits_soft_drop_for_removed_nullable_property() {
// Removing a property from the desired schema emits
// DropProperty { Soft } (schema-lint v1 chassis commit #3,
// MR-694). The plan is `supported = true` — the apply path
// handles soft drop via the existing stage_overwrite rewrite
// projection. Verified at the integration level by
// `apply_schema_drops_a_nullable_property_softly_preserves_prior_version`
// in `crates/omnigraph/tests/schema_apply.rs`.
let accepted = build_schema_ir(
&parse_schema(
r#"
node Person {
name: String @key
age: I32?
}
"#,
)
.unwrap(),
)
.unwrap();
let desired = build_schema_ir(
&parse_schema(
r#"
node Person {
name: String @key
}
"#,
)
.unwrap(),
)
.unwrap();

let plan = plan_schema_migration(&accepted, &desired).unwrap();
assert!(
plan.supported,
"drop-property plan must be supported: {plan:?}"
);
assert!(
plan.steps.iter().any(|step| matches!(
step,
SchemaMigrationStep::DropProperty {
type_kind: SchemaTypeKind::Node,
type_name,
property_name,
mode: DropMode::Soft,
..
} if type_name == "Person" && property_name == "age"
)),
"expected DropProperty {{ Soft }} step in plan: {plan:?}",
);
// Negative: no UnsupportedChange anywhere in the plan.
assert!(
!plan
.steps
.iter()
.any(|step| matches!(step, UnsupportedChange { .. })),
"soft drop must not emit UnsupportedChange: {plan:?}",
);
}

#[test]
fn plan_rejects_required_property_addition() {
let accepted = build_schema_ir(
Expand Down Expand Up @@ -935,4 +1063,56 @@ node Person @description("new") {
}],
}));
}

#[test]
fn drop_steps_round_trip_through_serde() {
// The DropType / DropProperty variants are dormant in this
// commit — the planner doesn't emit them yet — but their
// serde shape needs to be stable from day one. A future
// SchemaIR JSON containing one of these must deserialize
// back to the same value. This test pins the wire format
// so a v0 schema-ir consumer never sees a surprise variant
// shape after v1 ships.
let steps = vec![
SchemaMigrationStep::DropType {
type_kind: SchemaTypeKind::Node,
name: "Person".to_string(),
mode: DropMode::Soft,
},
SchemaMigrationStep::DropType {
type_kind: SchemaTypeKind::Edge,
name: "Knows".to_string(),
mode: DropMode::Hard,
},
SchemaMigrationStep::DropProperty {
type_kind: SchemaTypeKind::Node,
type_name: "Person".to_string(),
property_name: "age".to_string(),
mode: DropMode::Soft,
},
SchemaMigrationStep::DropProperty {
type_kind: SchemaTypeKind::Interface,
type_name: "Named".to_string(),
property_name: "alias".to_string(),
mode: DropMode::Hard,
},
];

for step in steps {
let json = serde_json::to_string(&step).expect("serialize");
let round_trip: SchemaMigrationStep =
serde_json::from_str(&json).expect("deserialize");
assert_eq!(step, round_trip, "round-trip mismatch on {json}");
}
}

#[test]
fn drop_mode_serde_uses_snake_case() {
// External tools may write SchemaIR JSON by hand. Pin the
// wire form so we don't silently break them later.
assert_eq!(serde_json::to_string(&DropMode::Soft).unwrap(), "\"soft\"");
assert_eq!(serde_json::to_string(&DropMode::Hard).unwrap(), "\"hard\"");
let soft: DropMode = serde_json::from_str("\"soft\"").unwrap();
assert_eq!(soft, DropMode::Soft);
}
}
2 changes: 1 addition & 1 deletion crates/omnigraph-compiler/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ pub use catalog::schema_ir::{
schema_ir_pretty_json,
};
pub use catalog::schema_plan::{
SchemaMigrationPlan, SchemaMigrationStep, SchemaTypeKind, plan_schema_migration,
DropMode, SchemaMigrationPlan, SchemaMigrationStep, SchemaTypeKind, plan_schema_migration,
};
pub use lint::{DiagnosticCode, Family, SafetyTier, Severity};
pub use ir::ParamMap;
Expand Down
4 changes: 2 additions & 2 deletions crates/omnigraph/src/db/omnigraph.rs
Original file line number Diff line number Diff line change
Expand Up @@ -18,8 +18,8 @@ use omnigraph_compiler::catalog::{Catalog, EdgeType, NodeType};
use omnigraph_compiler::schema::parser::parse_schema;
use omnigraph_compiler::types::ScalarType;
use omnigraph_compiler::{
SchemaIR, SchemaMigrationPlan, SchemaMigrationStep, SchemaTypeKind, build_catalog_from_ir,
build_schema_ir, plan_schema_migration,
DropMode, SchemaIR, SchemaMigrationPlan, SchemaMigrationStep, SchemaTypeKind,
build_catalog_from_ir, build_schema_ir, plan_schema_migration,
};

use crate::db::graph_coordinator::{GraphCoordinator, PublishedSnapshot};
Expand Down
35 changes: 35 additions & 0 deletions crates/omnigraph/src/db/omnigraph/schema_apply.rs
Original file line number Diff line number Diff line change
Expand Up @@ -138,6 +138,41 @@ pub(super) async fn apply_schema_with_lock(
}
SchemaMigrationStep::UpdateTypeMetadata { .. }
| SchemaMigrationStep::UpdatePropertyMetadata { .. } => {}
SchemaMigrationStep::DropProperty {
type_kind,
type_name,
mode,
..
} => {
// Soft = reuse the existing stage_overwrite rewrite
// path. batch_for_schema_apply_rewrite iterates the
// *target* schema fields, so a property absent from
// desired_catalog is naturally projected away. The
// prior Lance version retains the dropped column,
// so reads at the previous snapshot still see it
// (time-travel reversibility). Hard mode (immediate
// compact_files + cleanup_old_versions for actual
// data deletion) lands in commit #5 gated by
// --allow-data-loss.
if !matches!(mode, DropMode::Soft) {
return Err(OmniError::manifest_internal(
"DropProperty { Hard } not yet implemented (commit #5)",
));
}
let table_key = schema_table_key(*type_kind, type_name);
if table_key.starts_with("edge:") {
changed_edge_tables = true;
}
rewritten_tables.insert(table_key);
}
SchemaMigrationStep::DropType { .. } => {
// DropType (whole-table drop via __manifest entry
// removal) lands in commit #4 — different mechanics
// from DropProperty.
return Err(OmniError::manifest_internal(
"DropType not yet implemented (commit #4)",
));
}
step @ SchemaMigrationStep::UnsupportedChange { .. } => {
return Err(OmniError::manifest(
step.unsupported_error_message()
Expand Down
Loading