Registry manifest and Schema diff #400
Conversation
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
# Conflicts: # crates/weaver_semconv/src/group.rs
lquerel
changed the title
# Conflicts: # .clippy.toml # Cargo.toml # crates/weaver_semconv_gen/src/lib.rs # src/registry/search.rs # src/registry/stats.rs # src/registry/update_markdown.rs
| }, | ||
| /// A top-level telemetry object from the baseline registry was marked as deprecated in the head | ||
| /// registry. | ||
| Deprecated { |
There was a problem hiding this comment.
Is this its own change or should it be attached to other changes?
I.e. is it just an implication of change?
I think this was called out verbally, but it's the one I'm least sure of belonging with other "semantic" changes, especialyl given "uncategorized" as an option.
There was a problem hiding this comment.
See my comment below.
There was a problem hiding this comment.
Responded. Once updated, this PR LGTM.
Structurally/rust-wise you have all the pieces I'd look for. It's just naming/surface syntax at this point.
docs/schema-changes.md
Outdated
| - `renamed`: A top-level telemetry object from the baseline registry was renamed in the head registry. | ||
| - `deprecated`: A top-level telemetry object from the baseline registry was marked as deprecated in the head registry. | ||
| - `updated`: One or more fields in a top-level telemetry object have been updated in the head registry. | ||
| - `removed`: A top-level telemetry object from the baseline registry was removed in the head registry. |
There was a problem hiding this comment.
For semconv specifically, we definitely don't want to allow this, instead we deprecate.
Also - my concern with "deprecated" is that when we rename, we're efecctively deprecating the old.
I'm reading this and think "deprecated" is too generic and too much of a catch-all. I'd rather use "uncategorized", where deprecation is a consequence of the change vs. the change itself.
I.e. we almost need a "removed" where we mark the type as deprecated and prevent further usage but don't remove our knowledge it once existed.
There was a problem hiding this comment.
The deprecated type is indeed probably too much of a catch-all. However, I believe these three types are truly distinct, and I probably didn’t do a great job explaining them.
Currently, the general concept of deprecation is used for several types of changes in semantic conventions (renaming, “soft” removal, and other exotic changes). I propose refining my initial suggestion and the corresponding definitions as follows:
- Rename the change type
deprecatedtoobsoletedto clearly indicate that this change corresponds to an attribute or a signal that is discontinued without a valid replacement. - In my view,
removedshould exist at the Weaver level, if only to identify that there has been an actual deletion in a registry under validation. This type of change should never be issued for a published registry, but it is clearly a transitional change that can occur during the development of a registry. We could even build a policy leveraging this type of change in the future. uncategorizedis the catch-all change type representing all complex types of changes that we haven’t precisely codified. The idea of this type, as you mentioned during the meeting, is that we should gradually eliminate it from the registry.
Do we agree on this definition of things?
There was a problem hiding this comment.
I like moving deprecated to obsoleted where deprecated can remain as a catch-all for "we changed this thing in a way" and obsoleted implies "do not use anymore, here for legacy reasons".
I agree we need to actually model removed in some way. obsoleted as soft-delete works for me.
So yes, I agree on this.
| @@ -41,6 +41,48 @@ | |||
| } | |||
| }, | |||
| "$defs": { | |||
| "Deprecated": { | |||
There was a problem hiding this comment.
could you please also update https://github.com/open-telemetry/weaver/blob/main/schemas/semconv-syntax.md - an informal and reader-friendly version of it?
| registry_attributes: | ||
| - name: http.server_name # attribute name | ||
| type: obsoleted # change type | ||
| note: This attribute is deprecated. |
There was a problem hiding this comment.
is the note populated from note or brief?
There was a problem hiding this comment.
@lmolkova @jerbly Currently, the note in the schema change event is directly derived from the brief field of the telemetry object. Alternatively, we could choose to use the note field of the telemetry object. However, what we cannot do is allow users to fill in either one, as Weaver would then not know which one to use.
One possible solution/extension, however, would be to introduce a note field within the deprecated field (in the semantic conventions). If this field is populated, it would be used to fill the note field in the schema change event. If the deprecated.note field is not present, Weaver could then fall back to brief.
For this PR, I propose using brief for now and introducing the deprecated.note field if this approach works for you. I will make this change in another PR if needed.
Co-authored-by: Jeremy Blythe <[email protected]>
Co-authored-by: Jeremy Blythe <[email protected]>
Note: The scope of this PR has been reduced to focus only focus on the schema diff feature. Github issues have been created to track the features that have been postponed #482, open-telemetry/semantic-conventions#1938.
This PR implements the command
registry diff, see the following example:In this example, the diff is displayed in markdown format. The following formats are supported: json, markdown, ansi, ansi_stats. YAML format will be supported once PR #525 is finalized.
A detailed description of the schema diff data model and the diffing process is visible here.
Notes:
weaver_otel_schemais not essential for this PR; it was initially included as part of the preparations for theregistry schema-updatecommand. We have decided to implement this command in a future PR. However, for simplicity, I prefer to keep the preparation code in place instead of removing it. Same thing forall_changesinweaver_version.List of modifications to apply to the semantic conventions repository after the release of the Weaver containing the current PR:
registry-manifest.yamlfile with the version of the next release.Closes: #186
The following command comparing the versions 1.29 and 1.30
produces the following markdown output:
Registry Attributes
New registry_attributes:
Deprecated registry_attributes:
code.column.number)code.function.nameinstead)code.line.numberinstead)cassandra.consistency.levelinstead.)cassandra.coordinator.dcinstead.)cassandra.coordinator.idinstead.)cassandra.query.idempotentinstead.)cassandra.page.sizeinstead.)cassandra.speculative_execution.countinstead.)azure.client.idinstead.)azure.cosmosdb.connection.modeinstead.)cosmosdb.consistency.levelinstead.)azure.cosmosdb.operation.contacted_regionsinstead.)azure.cosmosdb.operation.request_chargeinstead.)azure.cosmosdb.request.body.sizeinstead.)azure.cosmosdb.response.sub_status_codeinstead.)elasticsearch.node.nameinstead.)db.operation.parameterinstead.)db.system.nameinstead.)gen_ai.request.seed.)network.connection.stateinstead.)Metrics
New metrics:
Deprecated metrics:
Spans
New spans: