Add LLM model server metrics (#1103)

Co-authored-by: Liudmila Molkova <[email protected]> Co-authored-by: Drew Robbins <[email protected]> Co-authored-by: Joao Grassi <[email protected]>
open-telemetry · Jun 27, 2024 · a328d73 · a328d73
1 parent 5a2836b
commit a328d73
Show file tree

Hide file tree

Showing 6 changed files with 298 additions and 12 deletions.
diff --git a/.chloggen/1102.yaml b/.chloggen/1102.yaml
@@ -0,0 +1,22 @@
+# Use this changelog template to create an entry for release notes.
+#
+# If your change doesn't affect end users you should instead start
+# your pull request title with [chore] or use the "Skip Changelog" label.
+
+# One of 'breaking', 'deprecation', 'new_component', 'enhancement', 'bug_fix'
+change_type: enhancement
+
+# The name of the area of concern in the attributes-registry, (e.g. http, cloud, db)
+component: gen-ai
+
+# A brief description of the change. Surround your text with quotes ("") if it needs to start with a backtick (`).
+note: Add GenAI model server metrics for measuring LLM serving latency
+
+# Mandatory: One or more tracking issues related to the change. You can use the PR number here if no issue exists.
+# The values here must be integers.
+issues: [1102]
+
+# (Optional) One or more lines of additional information to render under the primary note.
+# These lines will be padded with 2 spaces and then inserted directly into the document.
+# Use pipe (|) for multiline entries.
+subtext:
diff --git a/docs/attributes-registry/gen-ai.md b/docs/attributes-registry/gen-ai.md
@@ -26,7 +26,7 @@ This document defines the attributes used to describe telemetry in the context o
 | `gen_ai.response.finish_reasons`   | string[] | Array of reasons the model stopped generating tokens, corresponding to each generation received. | `["stop"]`                                                              | ![Experimental](https://img.shields.io/badge/-experimental-blue) |
 | `gen_ai.response.id`               | string   | The unique identifier for the completion.                                                        | `chatcmpl-123`                                                          | ![Experimental](https://img.shields.io/badge/-experimental-blue) |
 | `gen_ai.response.model`            | string   | The name of the model that generated the response.                                               | `gpt-4-0613`                                                            | ![Experimental](https://img.shields.io/badge/-experimental-blue) |
-| `gen_ai.system`                    | string   | The Generative AI product as identified by the client instrumentation. [3]                       | `openai`                                                                | ![Experimental](https://img.shields.io/badge/-experimental-blue) |
+| `gen_ai.system`                    | string   | The Generative AI product as identified by the client or server instrumentation. [3]             | `openai`                                                                | ![Experimental](https://img.shields.io/badge/-experimental-blue) |
 | `gen_ai.token.type`                | string   | The type of token being counted.                                                                 | `input`; `output`                                                       | ![Experimental](https://img.shields.io/badge/-experimental-blue) |
 | `gen_ai.usage.completion_tokens`   | int      | The number of tokens used in the GenAI response (completion).                                    | `180`                                                                   | ![Experimental](https://img.shields.io/badge/-experimental-blue) |
 | `gen_ai.usage.prompt_tokens`       | int      | The number of tokens used in the GenAI input or prompt.                                          | `100`                                                                   | ![Experimental](https://img.shields.io/badge/-experimental-blue) |