Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

GenAI: Clarify what gen_ai.request.top_k means #1397

Open
lmolkova opened this issue Sep 9, 2024 · 0 comments
Open

GenAI: Clarify what gen_ai.request.top_k means #1397

lmolkova opened this issue Sep 9, 2024 · 0 comments
Assignees

Comments

@lmolkova
Copy link
Contributor

lmolkova commented Sep 9, 2024

OpenAI top_logprobs:

An integer between 0 and 20 specifying the number of most likely tokens to return at each token position, each with an associated log probability. logprobs must be set to true if this parameter is used.

Cohere k:

Ensures only the top k most likely tokens are considered for generation at each step. Defaults to 0, min value of 0, max value of 500.

Anthropic top_k:

Only sample from the top K options for each subsequent token.
Used to remove "long tail" low probability responses. Learn more technical details here.

We should document what this parameter is, currently it's not clear

| [`gen_ai.request.top_k`](/docs/attributes-registry/gen-ai.md) | double | The top_k sampling setting for the GenAI request. | `1.0` | `Recommended` | ![Experimental](https://img.shields.io/badge/-experimental-blue) |

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Development

No branches or pull requests

2 participants