Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Stop reading the request body in the fibers web server #85

Draft
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

cbaines
Copy link

@cbaines cbaines commented May 3, 2023

This is a rather out there suggestion of a breaking change to the (fibers web server). I want to be able to handle large file uploads (several GB) and don't want that several GB of data to sit in memory while the request is being processed.

Instead, the handler must read the request body if there is one to read.

The intention here is to make it possible to process the request body in different ways. For example, use a chunked input port if it's chunked, or maybe just write it to disk without first reading it all in to memory. This flexibility will make it feasible to use the fibers web server to handle requests where the body is of an unknown size, or large enough that you wouldn't want it in memory.

The biggest downside of this change is that it breaks the handler interface as it's just passed a single argument, rather than two, I'm not sure how to mitigate that though.

Instead, the handler must read the request body if there is one to
read.

The intention here is to make it possible to process the request body
in different ways. For example, use a chunked input port if it's
chunked, or maybe just write it to disk without first reading it all
in to memory. This flexibility will make it feasible to use the fibers
web server to handle requests where the body is of an unknown size, or
large enough that you wouldn't want it in memory.
@emixa-d
Copy link
Collaborator

emixa-d commented Sep 4, 2023

The biggest downside of this change is that it breaks the handler interface
as it's just passed a single argument, rather than two, I'm not sure how to
mitigate that though.

Maybe add a #:request-body-streaming? #true/#false argument to select between the different interfaces.

@emixa-d emixa-d added enhancement needs-changes This PR isn't quite right yet. labels Sep 4, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement needs-changes This PR isn't quite right yet.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants