feat: add verbose request tracing and structured API error responses.#2007
Draft
zhang-minchao wants to merge 10 commits into
Draft
feat: add verbose request tracing and structured API error responses.#2007zhang-minchao wants to merge 10 commits into
zhang-minchao wants to merge 10 commits into
Conversation
zhang-minchao
requested review from
Clement-Wang26,
DongheJin,
DragonFive,
JimHsiung,
Kang-Meng,
RobbieLeung,
XuZhang99,
liujinguang0125,
liutongxuan,
walsonyang,
xiao-yu-chen,
yingxudeng and
yq33victor
as code owners
July 22, 2026 12:07
Contributor
|
Caution The consumer version of Gemini Code Assist on GitHub has been sunset. All code review activity has officially ceased. |
zhang-minchao
marked this pull request as draft
July 22, 2026 13:30
zhang-minchao
force-pushed
the
feat/verbose_error_trace_id
branch
from
July 22, 2026 13:33
673dae3 to
81ef010
Compare
…onses. - VerboseTraceLogger: spdlog async rotating logger; thread-local buffer eliminates per-call heap allocation. XLLM_VERBOSE_TRACE() << ... macro is zero-cost when disabled (one relaxed atomic load). - Resolve a single x_request_id per request in Call::init (client header or server-generated), pass directly to engine for end-to-end correlation. - Replace scattered ctrl->SetFailed + LOG(ERROR) with fail_http_request: structured JSON error body, proper HTTP status codes, x-request-id in logs, and verbose trace of failed context with raw request_body. - New gflags: enable_verbose_trace_log, verbose_trace_log_path (default log/verbose_trace.log), verbose_trace_log_max_size_mb, verbose_trace_log_max_files.
- AnthropicCall::write_and_finish: add missing request_completed trace event so every request_received has a matching completion in the log. - ModelsHttp: ProtoMessageToJson failure now returns a structured error response instead of an empty HTTP 200. - Call::init: reuse get_header_x_request_id() instead of duplicating the header extraction logic inline.
- request_completed: http=200 - request_failed_context: http=<code> (from status_code_to_http_status) - request_error: already had http=<code>, unchanged
…-id in response header. - Return const std::string& from Call accessors and mark them const. - Rename get_x_request_time() to x_request_time() for consistency with x_request_id(). - Use std::string_view for message/x_request_id params in api_error functions. - Add override and = default to ~AnthropicCall(). - Echo x-request-id in HTTP response header for end-to-end tracing.
…ency. - Add :block and :prefix_cache as explicit DEPS for stopping_checker_test to resolve undefined reference to create_prefix_cache caused by request ↔ prefix_cache circular static-archive link ordering.
- Update text_generation_service_impl.cpp to use renamed accessors. - Remove spurious :request dep from prefix_cache to break circular static-archive link ordering (prefix_cache ↔ request ↔ block).
zhang-minchao
force-pushed
the
feat/verbose_error_trace_id
branch
from
July 23, 2026 06:15
81ef010 to
38d7488
Compare
- preserve the configured path for single-rank serving - derive independent rotating paths for every multi-rank process - add path-resolution coverage and document per-rank retention
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
This PR improves API request observability and makes HTTP error responses consistent across OpenAI-compatible and Anthropic-compatible endpoints.
Request tracing
x-request-idfor each request, preferring the client-providedx-request-idorx-ms-client-request-idheader and generating an ID when neither is present.request_received,request_completed, andrequest_error) with the HTTP status code.spdlogasync queue. If the queue is full, the oldest queued entry is dropped rather than blocking request processing.Verbose tracing is disabled by default and can be configured with:
--enable_verbose_trace_logfalse--verbose_trace_log_pathlog/verbose_trace.log--verbose_trace_log_max_size_mb1024--verbose_trace_log_max_files100000Structured API errors
StatusCodeto HTTP status and API error-type mapping.brpc::Controller::SetFailed, which can replace the custom response with a generic HTTP 500.{"error":{"message":...,"type":...,"code":...,"param":null}}.{"type":"error","error":{"type":...,"message":...}}.Build fixes
stopping_checker_teststatic-library link dependencies.prefix_cache -> requestdependency that introduced circular static-archive link ordering after rebasing onto the latestmain.Related Issues
N/A
Change Type
Validation
format-checkpassed.Pull Request Checklist
Thank you for contributing to xLLM. Before requesting review, please make sure the following items are complete.
PR Title and Commit Messages
<type>: <subject>.Pre-commit Checks
pre-commitby runningpip install pre-commitor an equivalent command.pre-commit install.pre-commit run --all-filesand fixed any reported issues.Self Review
.agents/skills/code-review/references/custom-code-style.md, especially code written or assisted by AI.mainbranch.Build and Test Coverage
python setup.py build testhas passed on a CUDA machine.python setup.py build testhas passed on an NPU machine.python setup.py build testhas passed on an MLU machine.Reviewer Notes
StatusCodeto HTTP mappings, streaming terminal-error behavior, and the non-blocking logging path.