feat(semconv): update to v1.25 in a non-breaking way

[robbkidd]

Now that semconv won't have deprecated names removed without a major version bump, here's a go at updating semantic conventions to 1.25.

• no const_missing hijinks! (like in 🚧 WIP: update semantic conventions in a non-breaking way #1567)

• pulls from the new semconv repo

• updates jinja2 template for the new structure of convention data

• stable semconv names will appear in OpenTelemetry::SemanticConventions
  while all semconv names (experimental, deprecated, stable, etc) appear
  in OpenTelemetry::SemanticCandidates

  • This maps to what other langs (Java, Python, JS) are putting into
    "Incubating" modules. We opted for "candidates" because
    OpenTelemetry::SemanticConventions::Incubating::... was getting
    too long and OpenTelemetry::SemanticIncubating didn't make much
    sense.

• generates a module per root semconv namespace to make navigating docs
  more focused and to decrease the number of constants loaded when
  using more precise requires in downstream code

• generates only attribute name constants and to an attributes subfolder
  to prepare for metric name constants to be generated to metrics
  subfolders, also for decreasing the scope of what gets loaded when
  downstream instrumentation requires these constants

Some TODOs

A few of these appear in the code in this PR, too.

• usage examples. How weird is it to explicitly require the nested modules?
• test(s) to make sure the trace and resource constants are present in SemanticCandidates
  • See dc42bf5. I did the best I could with a test to find which existing v1.10 constants had no match in the entirety of 1.25. I used that list of missing consts to update their deprecation notices accordingly.
• test(s) to make sure the SemanticConventions (stable) constants are all still present in the SemanticCandidates constants
• add deprecation notices to all the SemanticConventions::(Resource|Trace) constants with pointers to their replacements
• figure out how to Firstcap or PascalCase the module names in the jinja2 template with exceptions for things like AspNetCore, HTTP, etc?
  • The module name will become a part of the constants interface, so the fancier we make the code generation logic, the more likely something breaks and we release a version where module names have a new backwards-breaking shape.
  • 👎🏻 I vote we don't do this and have non-idiomatic SCREAMING module names for simplicity of up-keep.
• Do something about JRuby
  • Turn off testing constants exist on JRuby?
  • Find a markdown parser/renderer (1) supported by Yard, (2) supports inline-code-blocks-inside-links in markdown content, and (3) works on JRuby?
    • Found kramdown. Using it instead of redcarpet.

[robbkidd]

Probably going to dial this back from semconv v1.26 to v1.25 because there's some name hijinks going on in db.* in 1.26. At least for this initial cut at catching Ruby's semconv library up to recent.

[robbkidd]

Guh. New markdown processor (redcarpet) for yard to use in doc generation doesn't support JRuby because native extensions.

An error occurred while installing redcarpet (3.6.0), and Bundler cannot
continue.

In Gemfile:
  redcarpet

[robbkidd]

Taking this out of draft. It may not be ready to merge, but it's ready for people to review and form opinions about.

There's a lot going on here†, so walking the commits might help. I left notes about the whats and whys of each change in the commit messages.

† 😅

[robbkidd]

I wondering if some const_missing hijinks ought to be introduced to these Modules to protect downstream developers from semconv schema mistakes or mind-changes that remove attribute names from a version which would result in a missing constant that existed previously. If only to warn-log the missing constant once and return something that (1) prevents runtime code from blowing up and (2) gives some indication of unhappiness in the emitted telemetry that humans actually look at.

[JamieDanielson]

• stable semconv names will appear in OpenTelemetry::SemanticConventions
  while all semconv names (experimental, deprecated, stable, etc) appear
  in OpenTelemetry::SemanticCandidates

  • This maps to what other langs (Java, Python, JS) are putting into
    "Incubating" modules. We opted for "candidates" because
    OpenTelemetry::SemanticConventions::Incubating::... was getting
    too long and OpenTelemetry::SemanticIncubating didn't make much
    sense.

Wanted to note while I'm looking at it - there's a chance we may go with Incubating in JS. If we do, it may be worth reconsidering "candidates" to help with consistency, and also to follow what seems to be guidance provided from semconv on artifact structure

• Artifact name:
  • opentelemetry-semconv - stable conventions
  • opentelemetry-semconv-incubating - (if applicable) the preview artifact containing all (stable and experimental) conventions
• Namespace: opentelemetry.semconv and opentelemetry.semconv.incubating

So maybe OpenTelemetry::SemConvIncubating:: as an option?

[kaylareopelle]

@JamieDanielson - I like the idea of mirroring the package name more closely and reducing SemanticConventions to SemConv. That'd give our fingers a bit of a break.

So to match the artifact namespace more closely, it could also be:
OpenTelemetry::SemConv - stable constants
OpenTelemetry::SemConv::Incubating - others

[robbkidd]

Yea. "Candidates" was probably never going to fly. This whole thing is about trying to be conventional.

I don't hate shortening SemanticConventions to SemConv, especially if we follow the guidance below and generate OpenTelemetry::SemConv::Incubating::Attributes::CLIENT::CLIENT_SOMETHING.

• Attributes, metrics, and other convention definitions should be grouped by the convention type and the root namespace. See the example below:

├── SchemaUrls.code
├── attributes
│   ├── ClientAttributes.code
│   ├── HttpAttributes.code
│   └── ...
├── metrics
│   ├── HttpMetrics.code
│   └── ...
└── events
    └── ...

[github-actions]

👋 This pull request has been marked as stale because it has been open with no activity. You can: comment on the issue or remove the stale label to hold stale off for a while, add the keep label to hold stale off permanently, or do nothing. If you do nothing this pull request will be closed eventually by the stale bot

[kaylareopelle]

Following up from Slack, here's a PR with my tweaks that bump the version to 1.36.0, which was the latest semconv version last week.

robbkidd#1

The script worked great! I made a few small tweaks to account for formatting (rubocop) and the tests. Otherwise, this seems very easy to maintain as long as weaver remains stable. I'm comfortable approving this PR with those changes. I think we should also update the README to reflect this new structure, which I would be happy to do.

What else do we need to consider before we merge this? What am I missing?

[robbkidd]

robbkidd#1

Merged!

What else do we need to consider before we merge this? What am I missing?

The questions we've asked ourselves in this PR that I don't know the answers to:

• in the description:
  • provide usage examples?
    • 🗳️ Maybe not a blocker, can be done later?
  • camelCase or SnakeCase constant names?
    • 🗳️ I vote no and for us to go with the current SCREAMING.
• in a pairing recap of old:
  • distinguish constants for attribute names vs metric names?
    • 🗳️ No? I think the only danger of not doing so is the risk that an attribute and a metric comes along at some point with the same name. 🤔 Which would mean one constant wins … but their names would be the same, so I think the result in telemetry produced would be indistinguishable. We'd just have an annoying warning at load time about a constant getting redefined.
  • making the generated documentation a little better with hyperlinking
    • 🗳️ I think at this point we shouldn't let gold-plated† docs block merging this PR to get up-to-date constants out to folks.

† And I think this as the primary gold-plater here.

[robbkidd]

Guh. Linters, amirite?

[kaylareopelle]

Guh. Linters, amirite?

Ughhhh I forgot to run rubocop locally before I opened the PR. 🫠

At least the problems are only related to human-written code and not the autogenerated stuff! I'll add some code suggestions to fix things up.

[kaylareopelle]

The questions we've asked ourselves in this PR that I don't know the answers to:

• in the description:

  • provide usage examples?

    • 🗳️ Maybe not a blocker, can be done later?

  • camelCase or SnakeCase constant names?

    • 🗳️ I vote no and for us to go with the current SCREAMING.

Yup. +1 here. Let's leave it.

• in a pairing recap of old:

  • distinguish constants for attribute names vs metric names?

    • 🗳️ No? I think the only danger of not doing so is the risk that an attribute and a metric comes along at some point with the same name. 🤔 Which would mean one constant wins … but their names would be the same, so I think the result in telemetry produced would be indistinguishable. We'd just have an annoying warning at load time about a constant getting redefined.

Also agree!

• making the generated documentation a little better with hyperlinking

  • 🗳️ I think at this point we shouldn't let gold-plated† docs block merging this PR to get up-to-date constants out to folks.

† And I think this as the primary gold-plater here.

I think that's a good call. We can create an issue to take care of this later on if we feel strongly about it.

[kaylareopelle]

🎊

[kaylareopelle]

Whenever we want to look at README improvements, here's a draft: robbkidd#2