From a0eb86250294005eaaaf03d0fff1fc64c3cfbfdd Mon Sep 17 00:00:00 2001 From: qapdex-maker Date: Sun, 23 Aug 2026 03:05:38 +0200 Subject: [PATCH 1/5] docs: add local-dev constraint (glibc/.NET 11 preview) and lightweight-telemetry skill - docs/LOCAL-DEVELOPMENT.md: documents that the pinned .NET 11 preview SDK is glibc-only and cannot run on Bionic hosts (Termux/Android); lists supported environments (Codespaces, Docker, WSL2, glibc VM) and what does NOT work. - plugins/dotnet11/skills/lightweight-telemetry: small dependency-free .NET 11 telemetry sample using System.Diagnostics.Metrics + TimeProvider. - docs/slides/README.md: reserved briefing for the deferred visual manual. - README.md: link the local-dev doc and the website/dashboard. --- README.md | 13 +++ docs/LOCAL-DEVELOPMENT.md | 88 +++++++++++++++++++ docs/slides/README.md | 34 +++++++ .../skills/lightweight-telemetry/SKILL.md | 70 +++++++++++++++ .../lightweight-telemetry/sample/Program.cs | 61 +++++++++++++ .../sample/telemetry.csproj | 11 +++ 6 files changed, 277 insertions(+) create mode 100644 docs/LOCAL-DEVELOPMENT.md create mode 100644 docs/slides/README.md create mode 100644 plugins/dotnet11/skills/lightweight-telemetry/SKILL.md create mode 100644 plugins/dotnet11/skills/lightweight-telemetry/sample/Program.cs create mode 100644 plugins/dotnet11/skills/lightweight-telemetry/sample/telemetry.csproj diff --git a/README.md b/README.md index 446f266ec3..6ad8e90cce 100644 --- a/README.md +++ b/README.md @@ -118,6 +118,19 @@ $ skill-installer install https://github.com/dotnet/skills/tree/main/plugins/ +- Agent Skills standard: + ## License See [LICENSE](LICENSE) for details. diff --git a/docs/LOCAL-DEVELOPMENT.md b/docs/LOCAL-DEVELOPMENT.md new file mode 100644 index 0000000000..7739f694e7 --- /dev/null +++ b/docs/LOCAL-DEVELOPMENT.md @@ -0,0 +1,88 @@ +# Local development on non-glibc hosts + +This repository pins a **.NET 11 preview SDK** in the root `global.json`: + +```json +{ + "sdk": { + "version": "11.0.100-preview.3.26207.106", + "rollForward": "latestMajor" + }, + "test": { "runner": "Microsoft.Testing.Platform" } +} +``` + +The official .NET SDK builds distributed from and the +`dotnet-install.sh` script are **glibc** binaries. They will not run on platforms +whose system loader is not glibc. This document records the verified constraint and +the supported ways to develop locally. + +## Verified constraint + +The following was confirmed on a Termux / Android (aarch64) host: + +| Check | Result | +|-------|--------| +| `dotnet` on PATH before install | not found | +| Install `.NET 11 preview` via `dotnet-install.sh --version 11.0.100-preview.3.26207.106` | "Installation finished successfully" | +| Run `./.dotnet/dotnet --version` | `cannot execute: required file not found` | +| ELF interpreter requested by the SDK muxer | `/lib/ld-linux-aarch64.so.1` (glibc) | +| glibc `libc.so.6` present on device | **none** | +| System loader present | `/system/bin/linker64` (Android Bionic) | +| System `dotnet-sdk-10.0` from the distro package | **works** (it is a Bionic build: interpreter `/system/bin/linker64`) | + +Conclusion: the SDK tarballs from dotnet.net are unusable on Bionic-only hosts +(Termux on Android, some minimal containers). `patchelf` does not help because the +glibc loader itself is absent. This is a platform ABI limit, not a configuration +problem in `global.json`. + +## Recommended local development environments + +Use a host that provides glibc. In priority order: + +### 1. GitHub Codespaces / GitHub Actions (already used by CI) +The repository's workflows (for example `skill-validator.yml`) use +`actions/setup-dotnet@v5` with `global-json-file: global.json`, which installs the +exact pinned preview on `ubuntu-latest`, `windows-latest`, and `macos-latest`. +This is the source of truth for a green build. + +### 2. Docker (Linux, glibc) +```bash +docker run --rm -it -v "$PWD":/repo -w /repo mcr.microsoft.com/dotnet/sdk:11.0-preview +dotnet build eng/skill-validator/SkillValidator.slnx +``` +The Microsoft `dotnet/sdk` images are glibc-based and resolve the pinned preview. + +### 3. WSL 2 on Windows +```bash +wsl -d Ubuntu +sudo apt update && sudo apt install -y dotnet-sdk-11.0 # or use dotnet-install.sh +cd /mnt/c/path/to/repo +dotnet build +``` +WSL2 runs a real glibc Linux kernel/userspace, so the preview SDK works. + +### 4. A glibc Linux VM or remote host +Any x64/arm64 Linux with glibc (Ubuntu, Fedora, Debian) can run +`dotnet-install.sh --version 11.0.100-preview.3.26207.106` and build the repo. + +## What does NOT work (do not waste time) + +- Installing the preview SDK under Termux / Android and expecting it to run. +- `patchelf`-ing the SDK to the Bionic loader — the glibc runtime is missing. +- `rollForward` tweaks in `global.json` — they only change SDK *version* selection, + not the ABI. They cannot make a glibc binary run on Bionic. + +## Why the .NET 10 distro package is not enough here + +`dotnet-sdk-10.0` installs and builds `net10.0` projects (for example the +`foundry-agent-webapp` backend), but it does **not** satisfy this repository's +`global.json`, which requires the `.NET 11 preview`. The preview is only published +as a glibc SDK, so this repo cannot be built on a Bionic-only host regardless of +which SDK version is installed. + +## See also + +- Repository website / dashboard: +- Skill authoring guide: +- `CONTRIBUTING.md` for how to add or change a plugin. diff --git a/docs/slides/README.md b/docs/slides/README.md new file mode 100644 index 0000000000..615b2bd44c --- /dev/null +++ b/docs/slides/README.md @@ -0,0 +1,34 @@ +# Visual manual — slides (authoring deferred) + +This folder is reserved for the **visual manual / slide deck** for the .NET Skills +repository. Content authoring is deferred to a later pass (owner: repo maintainer). + +## Goal of the deck + +A short, human-friendly companion to the written docs that explains: + +1. **What this repo is** — the .NET team's curated Agent Skills & custom agents + (see `README.md` and ). +2. **The plugin map** — one slide per plugin family + (dotnet, dotnet-ai, dotnet-msbuild, dotnet-aspnetcore, dotnet11, ...). +3. **Local dev constraint** — why the .NET 11 preview only builds on glibc hosts + (Termux/Android is Bionic) and the supported paths (Codespaces, Docker, + WSL2, glibc VM). Source of truth: `docs/LOCAL-DEVELOPMENT.md`. +4. **Lightweight telemetry add-on** — the `lightweight-telemetry` skill + (`plugins/dotnet11/skills/lightweight-telemetry/`): what it measures and how + to run the sample. +5. **Green CI** — the `skill-validator.yml` build/test matrix and the dashboard + at . + +## Suggested format + +- 8–12 slides. +- Diagrams: dark theme, architecture/flow style (see repo `docs/design/`). +- Keep code snippets minimal; link to the skill files for full source. + +## Status + +- [ ] Outline approved +- [ ] Slides drafted +- [ ] Reviewed against `docs/LOCAL-DEVELOPMENT.md` and the telemetry skill +- [ ] Published / linked from README diff --git a/plugins/dotnet11/skills/lightweight-telemetry/SKILL.md b/plugins/dotnet11/skills/lightweight-telemetry/SKILL.md new file mode 100644 index 0000000000..8136bef147 --- /dev/null +++ b/plugins/dotnet11/skills/lightweight-telemetry/SKILL.md @@ -0,0 +1,70 @@ +--- +name: lightweight-telemetry +description: Emit small, structured telemetry from a .NET 11 console app using System.Diagnostics.Metrics and the new .NET 11 TimeProvider/keyed services ergonomics. Use when a tool or CLI needs to report duration, counts, or build metadata without pulling in a full APM SDK. +license: MIT +--- + +# Lightweight telemetry in .NET 11 + +A minimal, dependency-free way to expose operational metrics from a CLI or tool. +No OpenTelemetry SDK, no external collector required — metrics are written to the +console as structured lines and can be scraped or redirected. + +## When to use + +- A build tool, CLI, or local agent needs to report timing/counts. +- You want structured telemetry without an APM vendor SDK. +- The host may be resource-constrained (no background collector). + +## When not to use + +- You need distributed tracing across services → use the + `configuring-opentelemetry-dotnet` skill instead. +- You need cloud ingestion (Application Insights) → use the vendor SDK. + +## The pattern + +Use `System.Diagnostics.Metrics.Meter` to define a counter and a histogram, drive +time measurement with `TimeProvider.System`, and flush a snapshot on exit. + +```csharp +using System.Diagnostics; +using System.Diagnostics.Metrics; + +var meter = new Meter("MyTool", "1.0.0"); +var runs = meter.CreateCounter("tool.runs", "runs", "Number of executions"); +var duration = meter.CreateHistogram("tool.duration.ms", "ms", "Execution duration"); + +var clock = TimeProvider.System; +var start = clock.GetTimestamp(); + +// ... work ... + +var elapsedMs = clock.GetElapsedTime(start).TotalMilliseconds; +runs.Add(1); +duration.Record(elapsedMs); + +// Snapshot is emitted via a console listener (see sample). +``` + +## Sample (runnable) + +See `sample/Program.cs` and `sample/telemetry.csproj`. Build and run: + +```bash +dotnet run --project sample +``` + +It prints one JSON line per metric reading, e.g.: + +```json +{"meter":"MyTool","instrument":"tool.duration.ms","value":12.4,"unit":"ms","timestamp":"2026-08-23T..."} +``` + +## Notes + +- `Meter`/`Counter`/`Histogram` are built into `System.Diagnostics.DiagnosticSource` + (no extra NuGet package for the API itself). +- For production scraping, attach an `IMetricsListener` or export to OTLP; this + skill intentionally stays at the smallest useful surface. +- Keep the meter name stable — it becomes the metric namespace downstream. diff --git a/plugins/dotnet11/skills/lightweight-telemetry/sample/Program.cs b/plugins/dotnet11/skills/lightweight-telemetry/sample/Program.cs new file mode 100644 index 0000000000..cbba827725 --- /dev/null +++ b/plugins/dotnet11/skills/lightweight-telemetry/sample/Program.cs @@ -0,0 +1,61 @@ +using System.Diagnostics; +using System.Diagnostics.Metrics; +using System.Text.Json; + +var meter = new Meter("MyTool", "1.0.0"); +var runs = meter.CreateCounter("tool.runs", "runs", "Number of executions"); +var duration = meter.CreateHistogram("tool.duration.ms", "ms", "Execution duration"); + +// Console listener: emits one structured line per measurement. +var listener = new MetricListener(meter); + +var clock = TimeProvider.System; +var start = clock.GetTimestamp(); + +// Simulate work so the metric has a non-zero value. +await Task.Delay(120); + +var elapsedMs = clock.GetElapsedTime(start).TotalMilliseconds; +runs.Add(1); +duration.Record(elapsedMs); + +listener.Dispose(); +meter.Dispose(); + +/// +/// Minimal console telemetry sink. Attaches to a Meter and prints JSON lines. +/// No external dependencies — uses the built-in diagnostic source listener API. +/// +sealed class MetricListener : IDisposable +{ + private readonly MeterListener _inner = new(); + private readonly Meter _meter; + + public MetricListener(Meter meter) + { + _meter = meter; + _inner.InstrumentPublished = (instrument, _) => + { + if (instrument.Meter.Name == _meter.Name) + _inner.EnableMeasurementEvents(instrument); + }; + _inner.SetMeasurementEventCallback(OnMeasurement); + _inner.SetMeasurementEventCallback(OnMeasurement); + _inner.Start(); + } + + private static void OnMeasurement(Instrument instrument, T measurement, ReadOnlySpan> tags, object? state) + { + var line = new + { + meter = instrument.Meter.Name, + instrument = instrument.Name, + unit = instrument.Unit, + value = measurement?.ToString(), + timestamp = DateTimeOffset.UtcNow.ToString("O") + }; + Console.WriteLine(JsonSerializer.Serialize(line)); + } + + public void Dispose() => _inner.Dispose(); +} diff --git a/plugins/dotnet11/skills/lightweight-telemetry/sample/telemetry.csproj b/plugins/dotnet11/skills/lightweight-telemetry/sample/telemetry.csproj new file mode 100644 index 0000000000..00d6a7d04a --- /dev/null +++ b/plugins/dotnet11/skills/lightweight-telemetry/sample/telemetry.csproj @@ -0,0 +1,11 @@ + + + + Exe + net11.0 + enable + enable + MyTool + + + From 5dd44af9fd0f484bf9d4364f1edf2d098ccea51c Mon Sep 17 00:00:00 2001 From: qapdex-maker Date: Sun, 23 Aug 2026 03:30:45 +0200 Subject: [PATCH 2/5] docs: document running .NET 11 preview inside ubuntu-termux (PRoot) - docs/LOCAL-DEVELOPMENT.md: add a verified walkthrough for running the .NET 11 preview SDK inside the glibc Ubuntu 24.04 guest of qapdex-maker/ubuntu-termux, including the required PRoot workaround (DOTNET_SYSTEM_GLOBALIZATION_INVARIANT, DOTNET_GCHeapHardLimit, ulimit -v) and the worked lightweight-telemetry sample run with its actual output. - plugins/dotnet11/skills/lightweight-telemetry/SKILL.md: add a "Running the sample inside ubuntu-termux (PRoot)" section so the skill carries the same workaround. All steps and output were verified on an arm64 Termux/PRoot host. --- docs/LOCAL-DEVELOPMENT.md | 60 +++++++++++++++++++ .../skills/lightweight-telemetry/SKILL.md | 17 ++++++ 2 files changed, 77 insertions(+) diff --git a/docs/LOCAL-DEVELOPMENT.md b/docs/LOCAL-DEVELOPMENT.md index 7739f694e7..807ea29a7d 100644 --- a/docs/LOCAL-DEVELOPMENT.md +++ b/docs/LOCAL-DEVELOPMENT.md @@ -81,6 +81,66 @@ Any x64/arm64 Linux with glibc (Ubuntu, Fedora, Debian) can run as a glibc SDK, so this repo cannot be built on a Bionic-only host regardless of which SDK version is installed. +## Running the .NET 11 preview inside `ubuntu-termux` + +The companion repo [qapdex-maker/ubuntu-termux](https://github.com/qapdex-maker/ubuntu-termux) +boots a real **glibc** Ubuntu 24.04 guest via PRoot on an Android/Termux host. +Because the guest is glibc (not Bionic), the .NET 11 preview SDK runs there — +this is the practical way to build this repository's `net11.0` projects on a phone. + +Verified steps (arm64, Ubuntu 24.04 guest): + +```bash +# 1. In Termux, install + launch the guest (see that repo's README) +git clone https://github.com/qapdex-maker/ubuntu-termux.git +cd ubuntu-termux && ./install.sh -y +./startubuntu.sh + +# 2. Inside the guest, install the preview SDK. +# The minimal rootfs has no curl/wget, so fetch the tarball from the +# Termux host (it is bind-mounted at /data/data/com.termux) and extract. +mkdir -p /root/dotnet +tar -xzf /data/data/com.termux/files/home/ubuntu-termux/dotnet-sdk.tar.gz -C /root/dotnet +/root/dotnet/dotnet --version # -> 11.0.100-preview.3.26207.106 +``` + +### PRoot-specific runtime workaround (required) + +Under PRoot the .NET runtime tries to reserve ~256 GiB of virtual address space, +which PRoot blocks, so `dotnet` aborts with +`GC: Reserving 274877906944 bytes ... failed` / `0x8007000E`. Fix it with: + +```bash +export DOTNET_SYSTEM_GLOBALIZATION_INVARIANT=1 # no libicu needed +export DOTNET_GCHeapHardLimit=134217728 # 128 MiB hard GC limit +ulimit -v 8388608 # cap virtual memory at 8 GiB +``` + +With these set, the runtime starts, builds, and runs normally on arm64 PRoot. + +### Worked example: the `lightweight-telemetry` sample + +The skill `plugins/dotnet11/skills/lightweight-telemetry/` ships a runnable +sample. Inside the guest: + +```bash +export DOTNET_CLI_TELEMETRY_OPTOUT=1 DOTNET_NOLOGO=1 \ + DOTNET_SYSTEM_GLOBALIZATION_INVARIANT=1 \ + DOTNET_GCHeapHardLimit=134217728 +ulimit -v 8388608 +cp -r /data/data/com.termux/files/home/github/repo/dotnet-skills/plugins/dotnet11/skills/lightweight-telemetry/sample /root/telemetry-sample +cd /root/telemetry-sample +/root/dotnet/dotnet build -c Release # Build succeeded, 0 warnings/0 errors +/root/dotnet/dotnet run -c Release --no-build +``` + +Verified output (structured telemetry as designed): + +```json +{"meter":"MyTool","instrument":"tool.runs","unit":"runs","value":"1","timestamp":"2026-08-23T01:26:12.0213552+00:00"} +{"meter":"MyTool","instrument":"tool.duration.ms","unit":"ms","value":"135.9351","timestamp":"2026-08-23T01:26:12.0991198+00:00"} +``` + ## See also - Repository website / dashboard: diff --git a/plugins/dotnet11/skills/lightweight-telemetry/SKILL.md b/plugins/dotnet11/skills/lightweight-telemetry/SKILL.md index 8136bef147..398a18ea70 100644 --- a/plugins/dotnet11/skills/lightweight-telemetry/SKILL.md +++ b/plugins/dotnet11/skills/lightweight-telemetry/SKILL.md @@ -61,6 +61,23 @@ It prints one JSON line per metric reading, e.g.: {"meter":"MyTool","instrument":"tool.duration.ms","value":12.4,"unit":"ms","timestamp":"2026-08-23T..."} ``` +## Running the sample inside `ubuntu-termux` (PRoot) + +This skill is verified to run inside the glibc Ubuntu 24.04 guest of +[qapdex-maker/ubuntu-termux](https://github.com/qapdex-maker/ubuntu-termux) on an +Android/Termux host — the practical way to execute `net11.0` code on a phone. +PRoot blocks .NET's default ~256 GiB virtual-address reservation, so set: + +```bash +export DOTNET_SYSTEM_GLOBALIZATION_INVARIANT=1 # no libicu in minimal rootfs +export DOTNET_GCHeapHardLimit=134217728 # 128 MiB hard GC limit +ulimit -v 8388608 # cap virtual memory at 8 GiB +``` + +Then `dotnet build -c Release` (succeeded with 0 warnings/0 errors) and +`dotnet run -c Release --no-build` produce the designed structured output. See +`docs/LOCAL-DEVELOPMENT.md` for the full walkthrough. + ## Notes - `Meter`/`Counter`/`Histogram` are built into `System.Diagnostics.DiagnosticSource` From 36f3bfcfdf35da05ec5fb94fa50e600f37593c5e Mon Sep 17 00:00:00 2001 From: qapdex-maker Date: Tue, 25 Aug 2026 21:02:07 +0200 Subject: [PATCH 3/5] tests: add eval.yaml + CODEOWNERS for dotnet11/lightweight-telemetry Adds the skill test the PR review (dotnet/skills#1036) requested. The eval uses the Vally schema mirrored from tests/dotnet11/system-text-json-net11: - 6 distinct stimuli (>=5 floor for statistical power) - 4 activation scenarios: built-in System.Diagnostics.Metrics + Meter, TimeProvider timing, stable Meter name, MeterListener JSON sink - 2 non-activation scenarios: distributed tracing and cloud ingestion are correctly routed to OpenTelemetry / vendor SDK instead check_eval_quality.py passes with "No errors." CODEOWNERS: add explicit entries for the new skill and its test, matching the existing system-text-json-net11 pattern. --- .github/CODEOWNERS | 2 + .../dotnet11/lightweight-telemetry/eval.yaml | 168 ++++++++++++++++++ 2 files changed, 170 insertions(+) create mode 100644 tests/dotnet11/lightweight-telemetry/eval.yaml diff --git a/.github/CODEOWNERS b/.github/CODEOWNERS index a1469dfc21..a0d078cf68 100644 --- a/.github/CODEOWNERS +++ b/.github/CODEOWNERS @@ -140,3 +140,5 @@ /plugins/dotnet11/skills/system-text-json-net11/ @dotnet/skills-csharp-language-reviewers /tests/dotnet11/system-text-json-net11/ @dotnet/skills-csharp-language-reviewers +/plugins/dotnet11/skills/lightweight-telemetry/ @dotnet/skills-csharp-language-reviewers +/tests/dotnet11/lightweight-telemetry/ @dotnet/skills-csharp-language-reviewers diff --git a/tests/dotnet11/lightweight-telemetry/eval.yaml b/tests/dotnet11/lightweight-telemetry/eval.yaml new file mode 100644 index 0000000000..7cb17f59c6 --- /dev/null +++ b/tests/dotnet11/lightweight-telemetry/eval.yaml @@ -0,0 +1,168 @@ +name: lightweight-telemetry +description: Evaluates the dotnet11/lightweight-telemetry skill +type: capability +config: + timeout: 3m +stimuli: + - name: Emit structured telemetry from a .NET 11 CLI + prompt: | + I'm building a .NET 11 console tool and I want it to report how many times + it ran and how long each run took, as structured telemetry. I do NOT want to + pull in OpenTelemetry or any external APM SDK — the host has no collector and + is resource constrained. Show me a minimal `net11.0` program that uses the + framework's built-in metrics API and prints one structured line per measurement. + graders: + - type: exit-success + - type: output-contains + config: + substring: System.Diagnostics.Metrics + - type: output-matches + config: + pattern: new Meter\( + - type: output-matches + config: + pattern: CreateCounter + - type: output-matches + config: + pattern: CreateHistogram + - type: output-matches + config: + pattern: net11\.0 + - type: output-not-matches + config: + pattern: OpenTelemetry + - type: prompt + rubric: + - Uses System.Diagnostics.Metrics.Meter (built into the framework, no extra NuGet package for the API) rather than OpenTelemetry or a vendor APM SDK + - Defines a counter and a histogram from the Meter + - Targets net11.0 + - Emits structured (JSON) output per measurement rather than only a plaintext log line + + - name: Drive timing with TimeProvider instead of DateTime.Now + prompt: | + In a .NET 11 tool I need to measure how long an operation takes and record it + as a metric. I want the timing to go through the framework's TimeProvider + abstraction rather than reaching for DateTime.Now everywhere. Show a small + `net11.0` snippet that starts a timestamp with a TimeProvider and computes the + elapsed milliseconds, then records it on a histogram. + graders: + - type: exit-success + - type: output-matches + config: + pattern: TimeProvider + - type: output-matches + config: + pattern: GetTimestamp + - type: output-matches + config: + pattern: GetElapsedTime + - type: output-not-matches + config: + pattern: DateTime\.Now + - type: output-matches + config: + pattern: net11\.0 + - type: prompt + rubric: + - Uses TimeProvider.System (or a TimeProvider instance) to obtain and compute elapsed time + - Calls GetTimestamp to start and GetElapsedTime to compute the duration + - Does NOT use DateTime.Now for the measurement + - Targets net11.0 + + - name: Consistent meter name for downstream scraping + prompt: | + I'm adding telemetry to several .NET 11 tools and I plan to scrape the + emitted metrics with an external system later. What should I be careful about + with the Meter name, and give me a `net11.0` example that creates a Meter with + a stable name and version. + graders: + - type: exit-success + - type: output-matches + config: + pattern: new Meter\( + - type: output-matches + config: + pattern: net11\.0 + - type: output-not-matches + config: + pattern: Guid\.NewGuid + - type: output-matches + config: + pattern: (stable|namespace|consisten) + - type: prompt + rubric: + - Creates a Meter with an explicit, stable name (and version) rather than a randomly generated one + - Explains that the meter name becomes the metric namespace downstream, so it must stay stable + - Targets net11.0 + + - name: Zero-dependency sink via built-in MeterListener + prompt: | + I have a .NET 11 tool that uses System.Diagnostics.Metrics. I want to print the + measurements to the console as JSON lines with no extra packages. Show me a + `net11.0` console listener that attaches to a Meter, enables measurement events, + and writes one JSON line per reading. + graders: + - type: exit-success + - type: output-matches + config: + pattern: MeterListener + - type: output-matches + config: + pattern: InstrumentPublished + - type: output-matches + config: + pattern: EnableMeasurementEvents + - type: output-matches + config: + pattern: JsonSerializer + - type: output-matches + config: + pattern: net11\.0 + - type: output-not-matches + config: + pattern: OpenTelemetry + - type: prompt + rubric: + - Uses the built-in MeterListener (no external dependency) to subscribe to instrument measurements + - Enables measurement events via InstrumentPublished / EnableMeasurementEvents + - Serializes each reading to JSON (e.g. via System.Text.Json) and writes it to the console + - Targets net11.0 + + - name: Non-activation — distributed tracing belongs elsewhere + prompt: | + My .NET 11 service needs distributed tracing across multiple microservices, + with spans, context propagation, and a backend like Jaeger. Should I use the + lightweight built-in metrics approach from the telemetry skill for this? + If not, what should I reach for instead? + expect_activation: false + graders: + - type: output-matches + config: + pattern: OpenTelemetry + - type: output-not-matches + config: + pattern: System\.Diagnostics\.Metrics\.Meter + - type: prompt + rubric: + - Recognizes that cross-service distributed tracing is out of scope for the lightweight-telemetry skill + - Recommends OpenTelemetry (or the configuring-opentelemetry-dotnet skill) instead of System.Diagnostics.Metrics + - Does NOT load the lightweight-telemetry skill as the solution for this request + + - name: Non-activation — cloud ingestion belongs elsewhere + prompt: | + I want to send my .NET 11 app's telemetry to Application Insights in the cloud + so I get dashboards and alerting. Is the dependency-free, console-emitting + telemetry approach the right fit here? + expect_activation: false + graders: + - type: output-matches + config: + pattern: Application Insights|ApplicationInsights|app insights|vendor SDK + - type: output-not-matches + config: + pattern: System\.Diagnostics\.Metrics\.Meter + - type: prompt + rubric: + - Recognizes that cloud ingestion / managed dashboards are out of scope for the lightweight-telemetry skill + - Recommends the vendor SDK (Application Insights) rather than the built-in console sink + - Does NOT load the lightweight-telemetry skill as the solution for this request From cd907052ba5370fe754e07bf5eeebb192754f5a5 Mon Sep 17 00:00:00 2001 From: qapdex-maker Date: Tue, 25 Aug 2026 21:13:40 +0200 Subject: [PATCH 4/5] tests: strengthen dotnet11/lightweight-telemetry eval to 10 stimuli The previous 6-stimulus eval sat in the statistically fragile 5-7 band (any loss is fatal, a tie can drop below 5 discordant votes). Bump to 10 distinct stimuli for a survivable one-loss margin: - keep the 6 existing scenarios (built-in metrics, TimeProvider timing, stable Meter name, MeterListener JSON sink, two non-activation cases) - add: gauge for live scalars, tagged measurements, instrument unit/description, non-activation for log aggregation (logging != metrics) check_eval_quality.py passes with "No errors." (10 distinct stimuli). --- .../dotnet11/lightweight-telemetry/eval.yaml | 92 +++++++++++++++++++ 1 file changed, 92 insertions(+) diff --git a/tests/dotnet11/lightweight-telemetry/eval.yaml b/tests/dotnet11/lightweight-telemetry/eval.yaml index 7cb17f59c6..4b2b2268e5 100644 --- a/tests/dotnet11/lightweight-telemetry/eval.yaml +++ b/tests/dotnet11/lightweight-telemetry/eval.yaml @@ -166,3 +166,95 @@ stimuli: - Recognizes that cloud ingestion / managed dashboards are out of scope for the lightweight-telemetry skill - Recommends the vendor SDK (Application Insights) rather than the built-in console sink - Does NOT load the lightweight-telemetry skill as the solution for this request + + - name: Gauge for a live scalar value + prompt: | + I have a .NET 11 tool and I want to expose a current scalar value (like the + number of items currently queued) as telemetry. I don't want a cumulative + counter for this — I want the instantaneous value. Show me a `net11.0` + snippet that creates the right instrument type from a Meter and records a + value into it. + graders: + - type: exit-success + - type: output-matches + config: + pattern: CreateGauge + - type: output-matches + config: + pattern: new Meter\( + - type: output-matches + config: + pattern: net11\.0 + - type: output-not-matches + config: + pattern: CreateCounter + - type: prompt + rubric: + - Uses Meter.CreateGauge for an instantaneous/live scalar value rather than a cumulative counter + - Records the current value into the gauge + - Targets net11.0 + + - name: Tag measurements for dimensions + prompt: | + In a .NET 11 tool I'm recording how long build steps take with a histogram. + I want each measurement to carry a dimension so I can tell which step it + belongs to (e.g. "restore" vs "compile"). Show me a `net11.0` example that + records the duration with a tag/label on the measurement. + graders: + - type: exit-success + - type: output-matches + config: + pattern: Record\( + - type: output-matches + config: + pattern: KeyValuePair + - type: output-matches + config: + pattern: net11\.0 + - type: prompt + rubric: + - Records the measurement with a tag/key-value dimension (e.g. via KeyValuePair on Record) so values can be split by step + - Uses the histogram Record call with the tagged value + - Targets net11.0 + + - name: Unit and description on instruments + prompt: | + I'm defining metrics in a .NET 11 tool and I want the instruments to carry a + unit (like "ms" for duration) and a human description, so downstream scraping + shows sensible metadata. Show me a `net11.0` snippet that creates a counter and + a histogram with explicit unit and description arguments. + graders: + - type: exit-success + - type: output-matches + config: + pattern: CreateCounter + - type: output-matches + config: + pattern: CreateHistogram + - type: output-matches + config: + pattern: net11\.0 + - type: prompt + rubric: + - Passes a unit (e.g. "ms", "runs") and a description to CreateCounter / CreateHistogram + - Shows awareness that instrument metadata improves downstream scraping + - Targets net11.0 + + - name: Non-activation — log aggregation is not metrics + prompt: | + I just want my .NET 11 app to ship its existing log lines (info/warn/error) + to a central place like Seq or Elasticsearch for searching. Does the + lightweight built-in metrics skill cover this? + expect_activation: false + graders: + - type: output-matches + config: + pattern: log|Seq|Elasticsearch|Serilog|ILogger + - type: output-not-matches + config: + pattern: System\.Diagnostics\.Metrics\.Meter + - type: prompt + rubric: + - Recognizes that shipping/searching log lines is a logging concern, not the metrics approach the skill describes + - Does not present System.Diagnostics.Metrics as the solution for log aggregation + - Does NOT load the lightweight-telemetry skill as the solution for this request From 0ac100e542aa004e383a5d87625c11a1718b9d81 Mon Sep 17 00:00:00 2001 From: qapdex-maker Date: Sat, 29 Aug 2026 20:33:58 +0200 Subject: [PATCH 5/5] Strengthen dotnet11/lightweight-telemetry skill and eval MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The eval sat at 7 preference-eligible stimuli, inside the 5-7 fragile band the quality gate warns about (one loss fatal, a tie can drop below the discordant floor). It also asserted the skill's own vocabulary (CreateGauge, MeterListener, InstrumentPublished), which is technique/vocabulary overfitting rather than outcome measurement. Eval: - 10 preference stimuli (was 7) + 3 dormancy contracts, each discriminating a different decision: instrument choice for a level vs a monotonic total, dimension via tag vs instrument-per-value, unit/description metadata, cheap measurement path when nothing collects, listener lifetime in a short-lived process, testable clock seam, stable metric identity. - Rubrics rewritten as outcomes; prompts no longer leak API names, so the baseline arm is not cued. - Dormancy guards now carry explicit anti-hijack rubric items (clears the gate's dormancy warning) and answer the real question instead of only declining. - config: -> defaults: (config is the deprecated alias), timeout 6m for code-generating stimuli. Skill: added the content the new stimuli demand and the baseline gets wrong — an instrument-selection table, tag-vs-name dimensions with a cardinality warning, Instrument.Enabled guarding + TagList to keep the hot path cheap, and listener lifetime (Start before first measurement, RecordObservableInstruments before exit). Verified, not asserted: the sample and every API claim were compiled and run. No net11.0 preview SDK is available on this host, so the code was exercised on net10.0 (these System.Diagnostics.Metrics APIs are unchanged) - build succeeded with 0 warnings/0 errors and the run emits tagged JSON lines carrying unit and description. The lifetime claim is from observed behaviour: a measurement recorded before listener.Start() produced no output line; the same measurement after it produced exactly one. check_eval_quality.py reports "No errors." and its 27 self-tests pass. skill-validator could not be run here (global.json pins the net11 preview SDK). --- .../skills/lightweight-telemetry/SKILL.md | 91 +++++- .../lightweight-telemetry/sample/Program.cs | 67 ++-- .../dotnet11/lightweight-telemetry/eval.yaml | 285 +++++++++--------- 3 files changed, 272 insertions(+), 171 deletions(-) diff --git a/plugins/dotnet11/skills/lightweight-telemetry/SKILL.md b/plugins/dotnet11/skills/lightweight-telemetry/SKILL.md index 398a18ea70..81917eddc6 100644 --- a/plugins/dotnet11/skills/lightweight-telemetry/SKILL.md +++ b/plugins/dotnet11/skills/lightweight-telemetry/SKILL.md @@ -1,6 +1,6 @@ --- name: lightweight-telemetry -description: Emit small, structured telemetry from a .NET 11 console app using System.Diagnostics.Metrics and the new .NET 11 TimeProvider/keyed services ergonomics. Use when a tool or CLI needs to report duration, counts, or build metadata without pulling in a full APM SDK. +description: Emit structured metrics from a .NET 11 console app, CLI, or build tool using the built-in System.Diagnostics.Metrics API with no OpenTelemetry, APM, or collector dependency. Use when asked to "report how long it took", "count how many times it ran", expose queue depth or a running total, pick between a counter, gauge, and histogram, split one metric by a tag/dimension, keep the measurement path cheap when nothing is listening, or print readings as JSON lines from a short-lived process. Do not use for distributed tracing across services (use configuring-opentelemetry-dotnet), cloud ingestion into Application Insights or Azure Monitor, or shipping log lines to Seq/Elasticsearch. license: MIT --- @@ -22,6 +22,68 @@ console as structured lines and can be scraped or redirected. `configuring-opentelemetry-dotnet` skill instead. - You need cloud ingestion (Application Insights) → use the vendor SDK. +## Pick the instrument first + +The instrument type is the decision that is most often wrong, and it is not +recoverable downstream — a consumer cannot turn a gauge back into a rate. + +| The value is | Use | Never use | Because | +|---|---|---|---| +| A total that only grows (bytes processed, runs) | `CreateCounter` | a gauge | the consumer derives the rate from the increasing total; a gauge that resets destroys it | +| The value right now (queue depth, open handles) | `CreateGauge` | a counter | a cumulative sum misrepresents a level that goes down again | +| A per-operation duration or size you want percentiles for | `CreateHistogram` | a counter | summing durations loses the distribution | +| A level you can only sample when asked | `CreateObservableGauge` | recording in a hot loop | the callback runs at collection time | + +Always pass the unit and description — put the unit in the **metadata**, not only +in a `.ms` name suffix, or a consumer cannot tell seconds from milliseconds: + +```csharp +meter.CreateHistogram("tool.step.duration", "ms", "Duration per build step"); +``` + +## Split a metric by a dimension, not by name + +One instrument plus a tag, never one instrument per value: + +```csharp +stepDuration.Record(elapsedMs, new TagList { { "step", "restore" } }); +``` + +Tag **values** must come from a bounded set (step names, status codes). Never tag +with a user id, path, or timestamp — each distinct value is a separate time +series downstream. + +## Keep the hot path cheap + +`Record`/`Add` are cheap, but building the tags and formatting values is not. +Guard the expensive part when nothing is collecting: + +```csharp +if (stepDuration.Enabled) // false when no listener is attached + stepDuration.Record(elapsedMs, new TagList { { "step", step } }); +``` + +Use `TagList` (a struct) rather than allocating a `KeyValuePair[]` per iteration. + +## Lifetime: set up the listener before the first measurement + +A `MeterListener` only sees measurements recorded **after** `Start()`. In a +short-lived CLI this is the difference between output and silence: + +```csharp +var listener = BuildListener(meter); // Start() called inside +// ... all recording happens after this point ... +listener.RecordObservableInstruments(); // pull observable gauges once before exit +listener.Dispose(); +meter.Dispose(); +``` + +Verified on .NET 10 (`System.Diagnostics.Metrics` is unchanged for these APIs on +net11.0): a measurement recorded before `listener.Start()` produces **no** output +line, one recorded after it produces exactly one. Observable instruments emit +nothing at all unless `RecordObservableInstruments()` is called, so a process that +exits without it reports nothing for them. + ## The pattern Use `System.Diagnostics.Metrics.Meter` to define a counter and a histogram, drive @@ -31,22 +93,28 @@ time measurement with `TimeProvider.System`, and flush a snapshot on exit. using System.Diagnostics; using System.Diagnostics.Metrics; -var meter = new Meter("MyTool", "1.0.0"); -var runs = meter.CreateCounter("tool.runs", "runs", "Number of executions"); -var duration = meter.CreateHistogram("tool.duration.ms", "ms", "Execution duration"); +var meter = new Meter("MyTool", "1.0.0"); // stable name = metric identity +var runs = meter.CreateCounter("tool.runs", "{run}", "Number of executions"); +var duration = meter.CreateHistogram("tool.step.duration", "ms", "Duration per step"); + +using var listener = new MetricListener(meter); // BEFORE the first measurement -var clock = TimeProvider.System; +var clock = TimeProvider.System; // injectable, testable clock var start = clock.GetTimestamp(); // ... work ... -var elapsedMs = clock.GetElapsedTime(start).TotalMilliseconds; +if (duration.Enabled) // skip tag building when idle + duration.Record(clock.GetElapsedTime(start).TotalMilliseconds, + new TagList { { "step", "compile" } }); runs.Add(1); -duration.Record(elapsedMs); -// Snapshot is emitted via a console listener (see sample). +listener.Flush(); // pull observables before exit ``` +Substitute a test `TimeProvider` (e.g. `Microsoft.Extensions.Time.Testing.FakeTimeProvider`) +to assert on recorded durations without sleeping. + ## Sample (runnable) See `sample/Program.cs` and `sample/telemetry.csproj`. Build and run: @@ -58,7 +126,7 @@ dotnet run --project sample It prints one JSON line per metric reading, e.g.: ```json -{"meter":"MyTool","instrument":"tool.duration.ms","value":12.4,"unit":"ms","timestamp":"2026-08-23T..."} +{"meter":"MyTool","instrument":"tool.step.duration","unit":"ms","description":"Duration per step","value":58.6,"tags":{"step":"compile"},"timestamp":"2026-08-29T18:32:07+00:00"} ``` ## Running the sample inside `ubuntu-termux` (PRoot) @@ -84,4 +152,7 @@ Then `dotnet build -c Release` (succeeded with 0 warnings/0 errors) and (no extra NuGet package for the API itself). - For production scraping, attach an `IMetricsListener` or export to OTLP; this skill intentionally stays at the smallest useful surface. -- Keep the meter name stable — it becomes the metric namespace downstream. +- Keep the meter name stable — it becomes the metric namespace downstream. The + meter *version* string is safe to bump; the name is not. +- One instrument + a tag beats one instrument per value, but keep tag values + bounded — unbounded values (ids, paths) create a time series each. diff --git a/plugins/dotnet11/skills/lightweight-telemetry/sample/Program.cs b/plugins/dotnet11/skills/lightweight-telemetry/sample/Program.cs index cbba827725..8b77128aa1 100644 --- a/plugins/dotnet11/skills/lightweight-telemetry/sample/Program.cs +++ b/plugins/dotnet11/skills/lightweight-telemetry/sample/Program.cs @@ -3,28 +3,42 @@ using System.Text.Json; var meter = new Meter("MyTool", "1.0.0"); -var runs = meter.CreateCounter("tool.runs", "runs", "Number of executions"); -var duration = meter.CreateHistogram("tool.duration.ms", "ms", "Execution duration"); -// Console listener: emits one structured line per measurement. -var listener = new MetricListener(meter); +// Counter: a total that only grows — the consumer derives a rate from it. +var runs = meter.CreateCounter("tool.runs", "{run}", "Number of executions"); +// Histogram: per-operation duration, so percentiles stay available. +var duration = meter.CreateHistogram("tool.step.duration", "ms", "Duration per step"); +// Gauge: the value right now, not a cumulative sum. +var queueDepth = meter.CreateGauge("tool.queue.depth", "{item}", "Items currently queued"); + +// The listener must start BEFORE the first measurement — anything recorded +// earlier is never observed. +using var listener = new MetricListener(meter); var clock = TimeProvider.System; -var start = clock.GetTimestamp(); -// Simulate work so the metric has a non-zero value. -await Task.Delay(120); +foreach (var step in new[] { "restore", "compile" }) +{ + var start = clock.GetTimestamp(); + await Task.Delay(60); // stand-in for real work + + // Cheap path: only assemble tags when something is actually collecting. + if (duration.Enabled) + { + var elapsedMs = clock.GetElapsedTime(start).TotalMilliseconds; + duration.Record(elapsedMs, new TagList { { "step", step } }); + } +} -var elapsedMs = clock.GetElapsedTime(start).TotalMilliseconds; runs.Add(1); -duration.Record(elapsedMs); +queueDepth.Record(3); -listener.Dispose(); -meter.Dispose(); +// Pull observable instruments once so nothing is lost at exit. +listener.Flush(); /// /// Minimal console telemetry sink. Attaches to a Meter and prints JSON lines. -/// No external dependencies — uses the built-in diagnostic source listener API. +/// No external dependencies — uses the built-in metrics listener API. /// sealed class MetricListener : IDisposable { @@ -34,28 +48,47 @@ sealed class MetricListener : IDisposable public MetricListener(Meter meter) { _meter = meter; - _inner.InstrumentPublished = (instrument, _) => + _inner.InstrumentPublished = (instrument, l) => { if (instrument.Meter.Name == _meter.Name) - _inner.EnableMeasurementEvents(instrument); + l.EnableMeasurementEvents(instrument); }; _inner.SetMeasurementEventCallback(OnMeasurement); _inner.SetMeasurementEventCallback(OnMeasurement); + _inner.SetMeasurementEventCallback(OnMeasurement); _inner.Start(); } - private static void OnMeasurement(Instrument instrument, T measurement, ReadOnlySpan> tags, object? state) + /// Records observable instruments once, e.g. just before exit. + public void Flush() => _inner.RecordObservableInstruments(); + + private static void OnMeasurement( + Instrument instrument, + T measurement, + ReadOnlySpan> tags, + object? state) + where T : struct { + var dimensions = new Dictionary(tags.Length); + foreach (var tag in tags) + dimensions[tag.Key] = tag.Value; + var line = new { meter = instrument.Meter.Name, instrument = instrument.Name, unit = instrument.Unit, - value = measurement?.ToString(), + description = instrument.Description, + value = measurement, + tags = dimensions, timestamp = DateTimeOffset.UtcNow.ToString("O") }; Console.WriteLine(JsonSerializer.Serialize(line)); } - public void Dispose() => _inner.Dispose(); + public void Dispose() + { + _inner.Dispose(); + _meter.Dispose(); + } } diff --git a/tests/dotnet11/lightweight-telemetry/eval.yaml b/tests/dotnet11/lightweight-telemetry/eval.yaml index 4b2b2268e5..11591bcad3 100644 --- a/tests/dotnet11/lightweight-telemetry/eval.yaml +++ b/tests/dotnet11/lightweight-telemetry/eval.yaml @@ -1,260 +1,257 @@ name: lightweight-telemetry description: Evaluates the dotnet11/lightweight-telemetry skill type: capability -config: - timeout: 3m +defaults: + timeout: 6m + runs: 1 stimuli: - - name: Emit structured telemetry from a .NET 11 CLI + - name: Structured metrics from a console tool without an APM SDK prompt: | I'm building a .NET 11 console tool and I want it to report how many times it ran and how long each run took, as structured telemetry. I do NOT want to pull in OpenTelemetry or any external APM SDK — the host has no collector and - is resource constrained. Show me a minimal `net11.0` program that uses the - framework's built-in metrics API and prints one structured line per measurement. + is resource constrained. Show me a minimal `net11.0` program that reports a + count and a duration distribution and prints one structured line per + measurement. graders: - type: exit-success - type: output-contains config: substring: System.Diagnostics.Metrics - - type: output-matches - config: - pattern: new Meter\( - - type: output-matches - config: - pattern: CreateCounter - - type: output-matches - config: - pattern: CreateHistogram - type: output-matches config: pattern: net11\.0 - type: output-not-matches config: - pattern: OpenTelemetry + pattern: PackageReference.*OpenTelemetry - type: prompt rubric: - - Uses System.Diagnostics.Metrics.Meter (built into the framework, no extra NuGet package for the API) rather than OpenTelemetry or a vendor APM SDK - - Defines a counter and a histogram from the Meter + - Solves the problem with the metrics API that ships in the framework rather than an added OpenTelemetry or vendor APM package + - Provides both a cumulative count and a duration distribution, not just one of the two - Targets net11.0 - - Emits structured (JSON) output per measurement rather than only a plaintext log line + - Emits machine-readable (structured/JSON) output per measurement rather than only a plaintext log line - - name: Drive timing with TimeProvider instead of DateTime.Now + - name: Elapsed time measured through the framework clock abstraction prompt: | In a .NET 11 tool I need to measure how long an operation takes and record it - as a metric. I want the timing to go through the framework's TimeProvider - abstraction rather than reaching for DateTime.Now everywhere. Show a small - `net11.0` snippet that starts a timestamp with a TimeProvider and computes the - elapsed milliseconds, then records it on a histogram. + as a metric, and I want the timing to be testable — I don't want DateTime.Now + scattered through the code. Show a small `net11.0` snippet that takes the + start timestamp, computes the elapsed milliseconds, and records it. graders: - type: exit-success - type: output-matches config: - pattern: TimeProvider - - type: output-matches - config: - pattern: GetTimestamp - - type: output-matches - config: - pattern: GetElapsedTime + pattern: net11\.0 - type: output-not-matches config: pattern: DateTime\.Now - - type: output-matches - config: - pattern: net11\.0 - type: prompt rubric: - - Uses TimeProvider.System (or a TimeProvider instance) to obtain and compute elapsed time - - Calls GetTimestamp to start and GetElapsedTime to compute the duration - - Does NOT use DateTime.Now for the measurement + - Obtains the elapsed time from an injectable clock abstraction (or an equivalent testable seam) instead of DateTime.Now + - Records the resulting duration as a metric measurement + - The timing code can be driven by a fake/controlled clock in a test without changing production code - Targets net11.0 - - name: Consistent meter name for downstream scraping + - name: Metric identity stays stable across releases prompt: | - I'm adding telemetry to several .NET 11 tools and I plan to scrape the - emitted metrics with an external system later. What should I be careful about - with the Meter name, and give me a `net11.0` example that creates a Meter with - a stable name and version. + I'm adding metrics to several .NET 11 tools and I plan to scrape the emitted + values with an external system later. Give me a `net11.0` example of setting + up the metric source, and tell me what I must be careful about so my + dashboards and queries don't break when I ship the next version. graders: - type: exit-success - - type: output-matches - config: - pattern: new Meter\( - type: output-matches config: pattern: net11\.0 - type: output-not-matches config: pattern: Guid\.NewGuid - - type: output-matches - config: - pattern: (stable|namespace|consisten) - type: prompt rubric: - - Creates a Meter with an explicit, stable name (and version) rather than a randomly generated one - - Explains that the meter name becomes the metric namespace downstream, so it must stay stable + - Uses an explicit, fixed metric source name rather than a generated or environment-derived one + - Explains that the source and instrument names form the identity downstream consumers query, so renaming them breaks existing dashboards + - Distinguishes the version string (safe to change) from the name (not safe to change) - Targets net11.0 - - name: Zero-dependency sink via built-in MeterListener + - name: Consume own measurements in-process with no extra packages prompt: | - I have a .NET 11 tool that uses System.Diagnostics.Metrics. I want to print the - measurements to the console as JSON lines with no extra packages. Show me a - `net11.0` console listener that attaches to a Meter, enables measurement events, - and writes one JSON line per reading. + I have a .NET 11 tool that already records metrics with the built-in metrics + API, but nothing consumes them so I see no output. Without adding any NuGet + package, how do I subscribe to my own measurements in the same process and + write each reading to the console as a JSON line? Show a `net11.0` example. graders: - type: exit-success - type: output-matches config: - pattern: MeterListener - - type: output-matches - config: - pattern: InstrumentPublished - - type: output-matches - config: - pattern: EnableMeasurementEvents - - type: output-matches - config: - pattern: JsonSerializer + pattern: net11\.0 - type: output-matches config: - pattern: net11\.0 + pattern: (Json|Serialize|serializ) - type: output-not-matches config: - pattern: OpenTelemetry + pattern: PackageReference.*OpenTelemetry - type: prompt rubric: - - Uses the built-in MeterListener (no external dependency) to subscribe to instrument measurements - - Enables measurement events via InstrumentPublished / EnableMeasurementEvents - - Serializes each reading to JSON (e.g. via System.Text.Json) and writes it to the console + - Subscribes to measurements in-process using a framework-provided listener, adding no NuGet package + - Explicitly opts the instruments in so callbacks actually fire (subscription alone produces nothing) + - Serializes each reading to JSON and writes it to the console - Targets net11.0 - - name: Non-activation — distributed tracing belongs elsewhere + - name: Instrument choice for an instantaneous value prompt: | - My .NET 11 service needs distributed tracing across multiple microservices, - with spans, context propagation, and a backend like Jaeger. Should I use the - lightweight built-in metrics approach from the telemetry skill for this? - If not, what should I reach for instead? - expect_activation: false + I have a .NET 11 tool and I want to expose the number of items currently + queued as telemetry. A running total is wrong here — I need whatever the + value is right now. Show me a `net11.0` snippet that picks the right + instrument for that and publishes the value. graders: + - type: exit-success - type: output-matches config: - pattern: OpenTelemetry - - type: output-not-matches - config: - pattern: System\.Diagnostics\.Metrics\.Meter + pattern: net11\.0 - type: prompt rubric: - - Recognizes that cross-service distributed tracing is out of scope for the lightweight-telemetry skill - - Recommends OpenTelemetry (or the configuring-opentelemetry-dotnet skill) instead of System.Diagnostics.Metrics - - Does NOT load the lightweight-telemetry skill as the solution for this request + - Chooses an instrument that reports the current value rather than a monotonically increasing sum + - Explains why a cumulative counter would misrepresent a queue depth + - Shows the value being published/observed, not merely the instrument being created + - Targets net11.0 - - name: Non-activation — cloud ingestion belongs elsewhere + - name: Instrument choice for a monotonic total prompt: | - I want to send my .NET 11 app's telemetry to Application Insights in the cloud - so I get dashboards and alerting. Is the dependency-free, console-emitting - telemetry approach the right fit here? - expect_activation: false + My .NET 11 tool processes files and I want to report the total number of + bytes it has processed since start, so an external system can compute a rate + from it. Show me a `net11.0` snippet with the right instrument for that and + explain why it is the right one. graders: + - type: exit-success - type: output-matches config: - pattern: Application Insights|ApplicationInsights|app insights|vendor SDK - - type: output-not-matches - config: - pattern: System\.Diagnostics\.Metrics\.Meter + pattern: net11\.0 - type: prompt rubric: - - Recognizes that cloud ingestion / managed dashboards are out of scope for the lightweight-telemetry skill - - Recommends the vendor SDK (Application Insights) rather than the built-in console sink - - Does NOT load the lightweight-telemetry skill as the solution for this request + - Chooses a monotonically increasing cumulative instrument for the byte total + - Explains that the consumer derives the rate from the increasing total, so the app must not reset or gauge it + - Does not model the running total as a distribution/percentile instrument + - Targets net11.0 - - name: Gauge for a live scalar value + - name: Split one metric by a dimension prompt: | - I have a .NET 11 tool and I want to expose a current scalar value (like the - number of items currently queued) as telemetry. I don't want a cumulative - counter for this — I want the instantaneous value. Show me a `net11.0` - snippet that creates the right instrument type from a Meter and records a - value into it. + In a .NET 11 tool I'm recording how long each build step takes. Right now all + the durations land in one bucket and I cannot tell "restore" from "compile". + I don't want a separate metric per step name. Show me a `net11.0` example + that fixes this. graders: - type: exit-success - - type: output-matches - config: - pattern: CreateGauge - - type: output-matches - config: - pattern: new Meter\( - type: output-matches config: pattern: net11\.0 - - type: output-not-matches - config: - pattern: CreateCounter - type: prompt rubric: - - Uses Meter.CreateGauge for an instantaneous/live scalar value rather than a cumulative counter - - Records the current value into the gauge + - Attaches a key/value dimension to each measurement instead of creating one instrument per step + - Keeps a single instrument and passes the step name as the dimension value + - Warns (or by construction avoids) unbounded dimension values that would explode cardinality - Targets net11.0 - - name: Tag measurements for dimensions + - name: Metadata that makes readings interpretable downstream prompt: | - In a .NET 11 tool I'm recording how long build steps take with a histogram. - I want each measurement to carry a dimension so I can tell which step it - belongs to (e.g. "restore" vs "compile"). Show me a `net11.0` example that - records the duration with a tag/label on the measurement. + I'm defining metrics in a .NET 11 tool and a colleague scraping them cannot + tell whether a duration value is seconds or milliseconds, or what a metric + means. Show me a `net11.0` snippet that fixes that at the point where the + metrics are defined. graders: - type: exit-success - type: output-matches config: - pattern: Record\( - - type: output-matches - config: - pattern: KeyValuePair + pattern: net11\.0 + - type: prompt + rubric: + - Declares the unit and a human-readable description on the instruments themselves rather than documenting them elsewhere + - Puts the unit in the metadata instead of relying only on a suffix in the metric name + - Shows the metadata reaching the consumer/reading output + - Targets net11.0 + + - name: Keep the measurement path cheap when nothing is listening + prompt: | + I want metrics in a hot loop in my .NET 11 tool, but most of the time nobody + is collecting them and I don't want to pay for building tag arrays and + formatting values on every iteration. How do I keep that path cheap? Show a + `net11.0` snippet. + graders: + - type: exit-success - type: output-matches config: pattern: net11\.0 - type: prompt rubric: - - Records the measurement with a tag/key-value dimension (e.g. via KeyValuePair on Record) so values can be split by step - - Uses the histogram Record call with the tagged value + - Checks whether the instrument is actually being collected before doing the expensive work of assembling the measurement + - Keeps the always-executed path allocation-light (no per-iteration allocation of tag collections or strings) + - Still records correctly when a consumer is attached - Targets net11.0 - - name: Unit and description on instruments + - name: Nothing is lost when the tool exits prompt: | - I'm defining metrics in a .NET 11 tool and I want the instruments to carry a - unit (like "ms" for duration) and a human description, so downstream scraping - shows sensible metadata. Show me a `net11.0` snippet that creates a counter and - a histogram with explicit unit and description arguments. + My short-lived .NET 11 CLI records metrics, but when the process exits I + sometimes see no output at all for the last operations. Show me how to + structure a `net11.0` program so the recorded values are all accounted for + before it terminates. graders: - type: exit-success - type: output-matches config: - pattern: CreateCounter + pattern: net11\.0 + - type: prompt + rubric: + - Ensures the consumer/listener is set up before the first measurement is recorded, not after + - Disposes or flushes the metric source and its consumer deterministically before the process exits + - Explains that a short-lived process can terminate before pull-based collection ever happens + - Targets net11.0 + + - name: Cross-service tracing request stays out of scope + prompt: | + My .NET 11 service needs distributed tracing across several microservices, + with spans, context propagation, and a Jaeger backend so I can follow one + request end to end. How should I set that up? + expect_activation: false + graders: - type: output-matches config: - pattern: CreateHistogram + pattern: (OpenTelemetry|Activity|ActivitySource) + - type: prompt + rubric: + - Answers with a distributed tracing solution (spans, context propagation, an OTLP/Jaeger exporter) + - Does not answer a tracing question with a dependency-free in-process metrics recipe + - Does not claim that console-printed metrics give end-to-end request correlation + - Treats a dependency-free in-process metrics recipe as out of scope for this request + + - name: Cloud ingestion request stays out of scope + prompt: | + I want my .NET 11 app's telemetry to land in Application Insights so my team + gets hosted dashboards, retention, and alerting without running anything + ourselves. What should I use? + expect_activation: false + graders: - type: output-matches config: - pattern: net11\.0 + pattern: (Application Insights|ApplicationInsights|Azure Monitor|OpenTelemetry) - type: prompt rubric: - - Passes a unit (e.g. "ms", "runs") and a description to CreateCounter / CreateHistogram - - Shows awareness that instrument metadata improves downstream scraping - - Targets net11.0 + - Recommends the hosted ingestion path (the vendor/Azure Monitor SDK or an OTLP exporter pointed at it) + - Does not propose printing measurements to the console as a substitute for hosted dashboards and alerting + - Addresses that the data must leave the process to reach the cloud service + - Treats a console-only, dependency-free metrics recipe as out of scope for this request - - name: Non-activation — log aggregation is not metrics + - name: Log shipping request stays out of scope prompt: | - I just want my .NET 11 app to ship its existing log lines (info/warn/error) - to a central place like Seq or Elasticsearch for searching. Does the - lightweight built-in metrics skill cover this? + I just want my .NET 11 app to ship its existing info/warn/error log lines to + a central place like Seq or Elasticsearch so I can search them. How do I do + that? expect_activation: false graders: - type: output-matches config: - pattern: log|Seq|Elasticsearch|Serilog|ILogger - - type: output-not-matches - config: - pattern: System\.Diagnostics\.Metrics\.Meter + pattern: (ILogger|Serilog|Seq|Elasticsearch|logging) - type: prompt rubric: - - Recognizes that shipping/searching log lines is a logging concern, not the metrics approach the skill describes - - Does not present System.Diagnostics.Metrics as the solution for log aggregation - - Does NOT load the lightweight-telemetry skill as the solution for this request + - Answers with a logging pipeline (a logger plus a sink for the target system) + - Does not convert the request into numeric instruments, which would discard the log message text + - Keeps the searchable log lines intact rather than replacing them with aggregated values + - Treats a numeric metrics recipe as out of scope for a log-shipping request