Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

server : add system_fingerprint to chat/completion #10917

Open
wants to merge 2 commits into
base: master
Choose a base branch
from

Conversation

ngxson
Copy link
Collaborator

@ngxson ngxson commented Dec 20, 2024

While working with compatible between llama.cpp and HF inference library, I noticed that llama.cpp is missing system_fingerprint compared the OAI compat response.

While this field is not required (and indeed we can remove the check for this field from HF library), I think it's more useful to implement it here in llama.cpp and return the build number (+ the commit hash).

This should be useful for downstream projects to know if the current server is up-to-date or not. We can additionally display this on the webui in the future.

For details of OAI API, see: https://platform.openai.com/docs/api-reference/chat/object

@ngxson ngxson requested a review from ggerganov December 20, 2024 11:46
@github-actions github-actions bot added examples python python script changes server labels Dec 20, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
examples python python script changes server
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant