Skip to content

feat(llmobs): auto-tag git commit and repository URL on traces and ex…#17939

Open
gsvigruha wants to merge 3 commits intomainfrom
gergely.svigruha/capture-git-commit
Open

feat(llmobs): auto-tag git commit and repository URL on traces and ex…#17939
gsvigruha wants to merge 3 commits intomainfrom
gergely.svigruha/capture-git-commit

Conversation

@gsvigruha
Copy link
Copy Markdown
Contributor

@gsvigruha gsvigruha commented May 6, 2026

Description

Auto detect git variables and add as tags.

  • For prod tracing use the default dd-trace mechanism
  • For experiment tracing use a fallback mechanism to detect these vars from git rev parse, because in a dev env the env var based setup is not necessarily working

Testing

  • Build wheel locally
  • Run the experiment onboarding notebook with new wheel
  • Verify tags

Risks

Performance: both mechanisms are computing the git vars once and caching it for the lifetime of the process so should not be an issue

…periments

Reuses ddtrace.internal.gitmetadata.get_git_tags() so LLM Observability spans
and Experiment._tags pick up git.commit.sha and git.repository_url from the
existing DD_GIT_* env vars or the main package's Project-URL metadata, gated
by DD_TRACE_GIT_METADATA_ENABLED. User-supplied experiment tags with the same
keys take precedence.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@gsvigruha gsvigruha requested review from a team as code owners May 6, 2026 23:53
@cit-pr-commenter-54b7da
Copy link
Copy Markdown

cit-pr-commenter-54b7da Bot commented May 6, 2026

Codeowners resolved as

tests/llmobs/test_experiments.py                                        @DataDog/ml-observability

@datadog-prod-us1-5

This comment has been minimized.

gsvigruha and others added 2 commits May 6, 2026 20:28
Adds an experiment-only fallback in _resolve_experiment_git_metadata that
shells out to `git rev-parse HEAD` / `git ls-remote --get-url` when the
standard gitmetadata source (DD_GIT_* env vars / package Project-URL) is
empty. Cached for the process lifetime; URL is scrubbed via
_filter_sensitive_info. Prod LLM Obs span tagging is unchanged.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Test the resolver _resolve_experiment_git_metadata directly instead of
through Experiment(...) for fallback scenarios; parametrize the three
gitmetadata-vs-fallback combinations; mock the resolver in the two
Experiment integration tests so they no longer reach down into
gitmetadata + extract_*. Same coverage, ~50 fewer lines.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
not repeatedly invoke ``git``.
"""
global _GIT_FALLBACK_CACHE
repository_url, commit_sha, _ = gitmetadata.get_git_tags()
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is not sufficient in a typical dev env

Copy link
Copy Markdown
Contributor

@Yun-Kim Yun-Kim left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure

cached for the process lifetime so repeated experiment construction does
not repeatedly invoke ``git``.
"""
global _GIT_FALLBACK_CACHE
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why are we reading/modifying a global scope cache variable for every experiment init? Why don't we just modify gitmetadata.get_git_tags() to include this subprocess fallback?

if _GIT_FALLBACK_CACHE is None:
fallback_url, fallback_sha = "", ""
try:
fallback_sha = git.extract_commit_sha()
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If DD_TRACE_GIT_METADATA_ENABLED=false, won't this still default to the git cli fallback and tag experiments anyway even though the user explicitly disabled git metadata tagging? Is this intentional

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants