Request forwarding with large files #350
-
|
I am working on a reverse-proxy-like thing where a user uploads a file to a central endpoint, which in turn forwards the file to an appropriate micro-service in the backend. The reverse-proxy works pretty well in its current state, however I have concerns related to handling large files, especially related to handling multiple simultaneous requests that may include large files. How do I ensure that the reverse-proxy is able to process multiple simultaneous requests without blowing up the memory-footprint of the app? |
Beta Was this translation helpful? Give feedback.
Replies: 8 comments 18 replies
-
|
Hi @thomafred
This way, files are never read whole in memory at once. I do NOT recommend using multipart/form-data for this kind of scenario, although it is the default used by HTML having To read the input in chunks in blacksheep (server part), you can do: from blacksheep import Application, Request, Response
app = Application()
post = app.router.post
@post("/upload")
async def save_big_file(request: Request):
async for chunk in request.stream():
# TODO: do something with the chunk, in this case forward the chunk to a back-end
...
# TODO: send a request to the back-end, return the desired response
return Response(200)
if __name__ == "__main__":
import uvicorn
uvicorn.run(app, port=44555)This is documented here. Are you using the BlackSheep HTTP client or another client? If you are using the client in BlackSheep, I can prepare an example for uploading files in chunks, too. Processing the files in chunks ensures that things are done efficiently. Then I would test with the expected workload - if things work, fine, otherwise I would consider two options:
As a side note, this depends on how your micro-service is implemented. For example, the Azure Storage Blob API offers two operations: one to upload a single chunk, one to communicate the end of the upload process. That approach is straightforward, and it forces clients to sends files in exact chunks, with multiple web requests. |
Beta Was this translation helpful? Give feedback.
-
|
@RobertoPrevato I am using the blacksheep client for this. Based on your input I have cobbled together the following example: # Request forwarder
def strip_headers(request):
headers = {}
for header, value in request.headers.items():
lheader = header.lower()
if lheader in (
b"content-length",
b"transfer-encoding",
b"content-encoding",
b"connection",
b"host"
):
continue # Ignore
return [(key, value) for key, value in headers.items()]
@post("/upload")
async def forward_big_file(request: Request):
async def data_generator():
async for chunk in request.stream():
yield chunk
# TODO: Handle params
content = StreamedContent(request.content_type(), data_generator)
req = Request(request.method, b"http://localhost:44556/upload", strip_headers(request)).with_content(content)
async with ClientSession() as client:
response = await client.send(req)
# TODO: send a request to the back-end, return the desired response
return Response(response.status)
if __name__ == "__main__":
import uvicorn
uvicorn.run(app, port=44555)# File request handler
from blacksheep import Application, Request, Response
app = Application()
post = app.router.post
@post("/upload")
async def save_big_file(request: Request):
bytes_received = 0
async for chunk in request.stream():
bytes_received += len(chunk)
print(f"Received {bytes_received} bytes")
# TODO: send a request to the back-end, return the desired response
return Response(200)
if __name__ == "__main__":
import uvicorn
uvicorn.run(app, port=44556)The request-forwarder will strip the header values for I am using the following command to test: curl --request POST "localhost:44555/upload" --data-binary @./myfile.binThis works fine when uploading the file directly to the file request handler, but fails when using the request-forwarder. From the log, I can see the following warning: From what I can tell, the warning is not originating from blacksheep, but possibly from uvicorn. I was able to reproduce this using Postman setting the |
Beta Was this translation helpful? Give feedback.
-
|
Hi @RobertoPrevato. Hope you are doing well :) I have had a second look at this, and I have been unable to resolve the Do you have any ideas on how we might work around this? |
Beta Was this translation helpful? Give feedback.
-
|
Hi @thomafred I can reproduce the error you reported about the content-length. I did not find the root cause of the problem, yet, but I think to be on a good track. PS. lately I might have seemed inactive on GitHub. I had less time (I was busier than usual), and I am working on something privately, still (a CLI to bootstrap BlackSheep projects). |
Beta Was this translation helpful? Give feedback.
-
|
Hi @thomafred The BlackSheep server in my example handles files of any size properly (lazily, in chunks) - it is the proxy part. The example is not 100% complete, as it should handle better some cases, but it describes what you asked in this discussion. |
Beta Was this translation helpful? Give feedback.
-
|
Hi @RobertoPrevato, This is awesome, thank you so much! Based on your example, I can see the request coming in the backend service and I can see the backend service response being returned (A Are there any special considerations we need to do in the backend service too? |
Beta Was this translation helpful? Give feedback.
-
|
@thomafred thanks for your patience! I will check more, and using your example, too. By the way, while preparing the example I realized that maybe BlackSheep should support async generators to prepare chunked responses while not using 'Transfer-Encoding: chunked'. |
Beta Was this translation helpful? Give feedback.
-
Yes, I agree this might help in this case. |
Beta Was this translation helpful? Give feedback.
-
|
Hi @thomafred I will try again using |
Beta Was this translation helpful? Give feedback.
-
|
Running the I do not understand why this is happening because the Flask server in my first example does not complain about anything, and everything else works. I will try to look more into this, but it might be an issue in |
Beta Was this translation helpful? Give feedback.
-
|
Hi @thomafred |
Beta Was this translation helpful? Give feedback.
-
|
This is amazing, thank you so much! I will make another attempt in our proxy implementation and let you know how it goes :) |
Beta Was this translation helpful? Give feedback.
-
|
This worked like a charm! Once again - thank you so much 🚀 |
Beta Was this translation helpful? Give feedback.
-
|
Thank You @thomafred, hadn't you created this discussion, I would have not noticed there was something wrong. :) I published the updated examples here: All issues were boiling down to the fact that BlackSheep supported only |
Beta Was this translation helpful? Give feedback.
-
|
Sorry to be pulling at this thread. I am seeing some issues related to small bodies with the Setting a breakpoint at This does not appear to be supported by Should the body be populated when dealing small requests? |
Beta Was this translation helpful? Give feedback.
-
|
I attempted to update the |
Beta Was this translation helpful? Give feedback.
-
Setting the Exerpt from the uvicorn logs: |
Beta Was this translation helpful? Give feedback.
-
|
After some more testing, it seems that the forwarded HTTP request is processed and file properly stored. The request handler returns the expected response, but it seems that uvicorn somehow changes to response to the The reverse proxy then raises an exception on the response, as the header is indicating a content-length of 28 bytes (my original response), but receives 30 bytes (The |
Beta Was this translation helpful? Give feedback.
-
|
Thanks for investigating and all the insight. I will try to understand what is going on here. |
Beta Was this translation helpful? Give feedback.
-
|
I tried uploading a 5.2 GB file using an HTML form with Firefox, to my proxy server in https://github.com/Neoteroi/BlackSheep-Examples/tree/main/proxy-2, with the fix applied locally (to use I will investigate more. |
Beta Was this translation helpful? Give feedback.
-
|
I think the invalid http request message came from an issue with my own
implementation, and that I may have been generating more data than the
content length I provided to the StreamedContent constructor.
The fix alone seems to be enough :)
…On Thu, 13 Jul 2023 at 18:38, Roberto Prevato ***@***.***> wrote:
I tried uploading a 5.2 GB file using an HTML form with Firefox, to my
proxy server in
https://github.com/Neoteroi/BlackSheep-Examples/tree/main/proxy-2, with
the fix applied locally (to use long long for Content.length) and could
not reproduce the error Invalid HTTP request received. with uvicorn, even
using httptools for it.
I will investigate more.
—
Reply to this email directly, view it on GitHub
<#350 (reply in thread)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AB3NZPG7VSPTSI775MG6U7DXQAQABANCNFSM6AAAAAAXUXOXOA>
.
You are receiving this because you were mentioned.Message ID:
***@***.***>
|
Beta Was this translation helpful? Give feedback.
Hi @thomafred
I prepared a working example, and published it here: https://github.com/Neoteroi/BlackSheep-Examples/tree/main/proxy
The BlackSheep server in my example handles files of any size properly (lazily, in chunks) - it is the proxy part.
Please take a look at how I implemented it, and the notes I left there.
The example is not 100% complete, as it should handle better some cases, but it describes what you asked in this discussion.