docs: research GenAI instrumentation landscape (#154)

[LoveChauhan-18]

Description

This PR addresses Issue #154 by providing a comprehensive audit of the GenAI/LLM instrumentation landscape. It establishes a new research workspace under projects/ and documents the current state of OpenTelemetry support across the four primary ecosystem languages.

Motivation

As GenAI workloads become more prevalent, mapping the existing instrumentation landscape is the first step toward automating registry data extraction. This research identifies where the ecosystem stands regarding semantic convention adoption and where critical gaps remain.

Proposed Changes

• New Planning Workspace: Established projects/154-genai-ecosystem/ following standard repository conventions.
• Audit Findings: Created 00-research.md covering:
  • Python: Identified as the reference implementation with mature contrib support.
  • JS/TS: Highlighted a significant gap in OTel-native instrumentation for major SDKs (OpenAI/Anthropic).
  • Java & .NET: Identified a bridge-based strategy as the most viable path forward for frameworks like Spring AI and Semantic Kernel.
  • Semconv Coverage: Mapped stable vs. experimental attribute adoption.
• Roadmap: Created NEXT-STEPS.md to track the decision log and immediate actions for maintainer review.
• Index Registration: Registered the initiative in the root projects/_index.md.

Key Conclusions

1. Python as Reference: Registry schemas should be modeled after Python's *-contrib implementations.
2. The JS Gap: Direct OTel-native instrumentation for JS LLM SDKs is a high-priority opportunity.
3. conformance Dashboard: The genai-otel-conformance project should be the primary benchmark for verifying future instrumentation quality.

Verification

• All docs satisfy frontmatter.schema.json.
• All files pass bun run lint:md.
• All files pass scripts/check_copyright.py.
• All files are 100% formatted via prettier.

[netlify]

✅ Deploy Preview for otel-ecosystem-explorer ready!

Name	Link
🔨 Latest commit	4e0e2bc
🔍 Latest deploy log	https://app.netlify.com/projects/otel-ecosystem-explorer/deploys/6a01b25ce68b8600079dadab
😎 Deploy Preview	https://deploy-preview-388--otel-ecosystem-explorer.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

---

To edit notification comments on pull requests, go to your Netlify project configuration.

[LoveChauhan-18]

Hi @jaydeluca,

I have completed the initial research phase for Issue #154 regarding the GenAI/LLM instrumentation landscape.

Summary of Work:

• Audit Scope: Conducted a deep dive across Python, JS/TS, Java, and .NET, mapping existing instrumentations, signal support (Traces/Metrics/Logs), and OTel Semantic Convention adoption.

• Findings: Identified Python as the current reference implementation and surfaced a critical gap in native OTel support for the OpenAI JS SDK.

• Next Steps: Provided a rolling roadmap in NEXT-STEPS.md to coordinate maintainer feedback on how these findings should be integrated into the Explorer's registry.

Looking forward to your feedback on the audit findings and the proposed registry strategy!

[LoveChauhan-18]

Thanks for the correction, @jaydeluca! I’ve updated the research documentation to accurately reflect the Java ecosystem's status:

• Updated Java section in 00-research.md: Now includes the official opentelemetry-java-instrumentation agent's support for the OpenAI Java SDK and AWS Bedrock (via AWS SDK).

• Revised observations: Corrected the assessment of Java as the 'weakest area'; it’s now noted as rapidly catching up to Python with solid SDK-level coverage.

• Updated NEXT-STEPS: Included Java as a viable target for the first cut of the registry alongside Python and JS/TS.

The PR has been updated with these changes. Ready for another look!

[LoveChauhan-18]

@jaydeluca , Please review the PR once and let me know if the PR is in good shape or do I need to make some changes into the PR ?

[jaydeluca]

hey @LoveChauhan-18 thanks for doing this, I just need a little more time to review. it's on my list and will get to it soon. thanks for your patience!

in the meantime, i started an issue to discuss the approach of this type of task: #416

[LoveChauhan-18]

hey @LoveChauhan-18 thanks for doing this, I just need a little more time to review. it's on my list and will get to it soon. thanks for your patience!

in the meantime, i started an issue to discuss the approach of this type of task: #416

Sure, I am also working on the issue mentioned by you (#416) .