Repository navigation
extproc: send content-length as a decimal string on retried requests - #2944
Open
3431481073 wants to merge 1 commit into
Open
3431481073 wants to merge 1 commit into
3431481073 wants to merge 1 commit into
Conversation
buildContentLengthDynamicMetadataOnRequest stored the request body length as a NumberValue. On retries the header mutation filter substitutes %DYNAMIC_METADATA(...:content_length)%, and Envoy renders a double in shortest form, so a round length like 100000 went out as 1e+05. Strict HTTP parsers (uvicorn/vLLM) reject that with 400. Store the length with strconv.Itoa so the retry sends plain decimal digits. Add unit tests for the metadata builder and for the request header path a retry re-runs, using a 100000-byte body. Fixes theagentrouter#2930 Signed-off-by: xby <3431481073@qq.com>
✅ Deploy Preview for theagentrouter canceled.
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
On a retry the upstream ext_proc replaces the request body, so Envoy clears
content-lengthand restamps it from dynamic metadata.buildContentLengthDynamicMetadataOnRequeststored the length as aNumberValue, and Envoy renders a double in shortest form when substituting%DYNAMIC_METADATA(io.envoy.ai_gateway:content_length)%. A 100000-byte body therefore went out ascontent-length: 1e+05, and strict HTTP parsers such as uvicorn/vLLM rejected the request with400 Invalid HTTP request received.Only retries were affected; the first attempt kept the client'scontent-length: 100000.Store the length as a decimal string (
strconv.Itoa) instead. Envoy then substitutes plain digits, so the retry carries the same header value as the original request.Related Issues/PRs
Fixes #2930
Special notes for reviewers
Only the dynamic metadata value type changes. The header mutation filter that consumes it is unchanged and substitutes the string verbatim. A repository search found no other reader of the
content_lengthmetadata key.Test_buildContentLengthDynamicMetadataOnRequestcovers 0, 1, 1000, 99999, 100000 and 32000000.Test_chatCompletionProcessorUpstreamFilter_ProcessRequestHeaders_contentLengthexercises the request-header phase a retry re-runs with a 100000-byte body.go test ./internal/extproc/ ./internal/extensionserver/ -count=1andgo vet ./internal/extproc/pass. End-to-end Envoy + vLLM was not available in this environment.