Skip to content

Conversation

@ihower
Copy link
Contributor

@ihower ihower commented Nov 22, 2025

Resolved: #2100

This PR adds three useful fields to RequestUsage: model_name, agent_name, and response_id.

These fields make it easier to:

  • analyze usage across different models
  • track usage by agent
  • correlate each usage record with the exact response ID for inspection

Note

model_name is taken from the Model#model attribute. All real subclasses of Model (such as OpenAIResponsesModel, OpenAIChatCompletionsModel, and LitellmModel) do initialize a model attribute.

However, the parent Model class does not define this field, which leads to type-checking errors. So this PR adds the attribute to the base class. To avoid breaking custom Model implementations (for example, FakeModel in tests that do not set a model name), the default value is an blank string.

@ihower ihower changed the title Add model_name, agent_name, and request_id to RequestUsage for better tracking Add model_name, agent_name, and response_id to RequestUsage for better tracking Nov 22, 2025
@seratch seratch added enhancement New feature or request feature:core labels Nov 22, 2025
"""The base interface for calling an LLM."""

# The model name. Subclasses can set this in __init__.
model: str = ""
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

in fact, it seems available models have self.model but this could be potentially a breaking change for some custom models. it will require a minor version upgrade and release note annoucements

def add(
self,
other: Usage,
model_name: str,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

these properties are in the request usage data, so i don't think these data should be passed this way. at least, we should avoid passing these in order to keep flexibility of future enhancement (meaning having additional-properties argument, which can accept more properties in the future)

@seratch seratch added this to the 0.7.x milestone Nov 22, 2025
@ihower
Copy link
Contributor Author

ihower commented Nov 22, 2025

@seratch Thanks for the code review.

I’ve updated the PR so that all of these fields go into a single metadata dict. This should keep future enhancements flexible.

Regarding model.model, I’m not fully sure about the breaking-change risk. Since the base Model class now provides a default value "", any custom model that inherits from Model will have the attribute and won’t raise errors.

Are you concerned about custom models that don’t inherit from Model at all (basically duck typing)?

If we want to avoid touching the Model class to fully eliminate this risk, I can switch to a runtime check when calling add(), something like:

if hasattr(model, "model") and model.model:
    metadata["model_name"] = model.model

I can update the PR if you prefer this approach. Just let me know. Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request feature:core

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Track Agent and Model Identifiers in Usage Statistics

2 participants