Skip to content

Commit

Permalink
Add support for streaming uploads
Browse files Browse the repository at this point in the history
Supports both seekable and unseekable streams
  • Loading branch information
kyleknap committed Mar 17, 2016
1 parent 2b4a2b9 commit eb40ac5
Show file tree
Hide file tree
Showing 8 changed files with 995 additions and 294 deletions.
32 changes: 30 additions & 2 deletions s3transfer/manager.py
Original file line number Diff line number Diff line change
Expand Up @@ -92,8 +92,36 @@ def __init__(self, client, config=None):
def upload(self, fileobj, bucket, key, extra_args=None, subscribers=None):
"""Uploads a file to S3
:type fileobj: str
:param fileobj: The name of a file to upload.
:type fileobj: str or filelike object
:param fileobj: The file to upload. Valid value types are as follows:
* name of file (str): This format is recommended because it
results in much better memory management and handles the
file management for you.
* seekable file-like object: The file like object **must**
produce binary data. Note that there are memory implications
with this format. In the worst possible scenario, you can
expect based on the TransferConfig::
multipart_chunksize * min(max_queue_size, max_concurrency)
So configure the manager accordingly.
* unseekable file-like object: The file like object **must**
produce binary data. Note that there are memory implications
with this format. In the worst possible scenario, you can
expect based on the TransferConfig::
multipart_threshold + (
multipart_chunksize * (min(
max_queue_size, max_concurrency))
If a transfer size is provided, the worst case scenario becomes::
multipart_chunksize * min(max_queue_size, max_concurrency)
So configure the manager accordingly.
:type bucket: str
:param bucket: The name of the bucket to upload to
Expand Down
Loading

0 comments on commit eb40ac5

Please sign in to comment.