Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(blob): Provide onUploadProgress({ loaded, total, percentage }) #782

Merged
merged 43 commits into from
Nov 6, 2024

Conversation

vvo
Copy link
Member

@vvo vvo commented Oct 24, 2024

This commit introduces an onUploadProgress callback to put/upload*. Here's how to use it:

// also works with upload()
const blob = await put('file.pdf', file, {
  onUploadProgress(event) {
    console.log(event.loaded, event.total, event.percentage);
  }
});

How does it work?

By default we use fetch. When you pass onUploadProgress, we turn the file into a stream to track the stream consumption's progress by fetch.
For browsers not supporting fetch with streams (Safari, Firefox), we use XMLHttpRequest and its progress events.

This means we re-implemented parts of https://github.com/axios/axios or other fetch libraries using xhr as backup. But remember, progress events are somehow lying as they only measure what gets sent from your computer to the server, not what the server really acknowledged. Still, they are a good indicator for progress of the upload.

This commit introduces an `onUploadProgress` callback to put/upload*. Since
we're using fetch, this works by making sure every body we send via fetch is a
ReadableStream (web). Once it's a ReadableStream we can follow its consumption.

Caveats:
- This only measures what gets sent to the server, not what was received by the
  server
- Safari & Firefox are not supporting bodies as ReadableStreams, we may need to
  introduce.. xhr!

TODO:
- [] Add XHR fallback for runtimes not supporting streams as bodies but having
  XHR
- [] Add real browser tests ensuring we get multiple onUploadProgress events for
  big files
- [] Ensure we send one event at the start of the upload (progress: 0)
@vvo vvo requested a review from luismeyer October 24, 2024 07:36
@vvo vvo self-assigned this Oct 24, 2024
Copy link

changeset-bot bot commented Oct 24, 2024

🦋 Changeset detected

Latest commit: 682d4ba

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 2 packages
Name Type
@vercel/blob Minor
vercel-storage-integration-test-suite Minor

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

Copy link

vercel bot commented Oct 24, 2024

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name Status Preview Updated (UTC)
vercel-storage-next-integration-test-suite ✅ Ready (Inspect) Visit Preview Nov 6, 2024 9:30am

Copy link

socket-security bot commented Oct 24, 2024

New and removed dependencies detected. Learn more about Socket for GitHub ↗︎

Package New capabilities Transitives Size Publisher
npm/@playwright/[email protected] None +2 10.8 MB dgozman-ms, mxschmitt, pavelfeldman, ...1 more
npm/@tailwindcss/[email protected] None +1 64.5 kB adamwathan
npm/@types/[email protected] None +1 14 kB types
npm/@types/[email protected] Transitive: environment, filesystem, unsafe +48 1.76 MB types
npm/@types/[email protected] None 0 4.76 kB types
npm/@types/[email protected] None +1 2.36 MB types
npm/@types/[email protected] None 0 0 B
npm/@types/[email protected] None +2 1.69 MB types
npm/[email protected] environment Transitive: filesystem, shell +10 2.72 MB ai
npm/[email protected] network Transitive: environment, filesystem +8 2.51 MB jasonsaayman
npm/[email protected] environment, filesystem 0 79.1 kB motdotla
npm/[email protected] unsafe Transitive: environment, eval, filesystem, shell +230 25.3 MB vercel-release-bot
npm/[email protected] environment, filesystem, network, shell, unsafe +68 101 MB vercel-release-bot
npm/[email protected] environment, filesystem +3 376 kB ai
npm/[email protected] environment +3 4.63 MB react-bot
npm/[email protected] environment +2 339 kB react-bot
npm/[email protected] Transitive: environment +2 659 kB vercel-release-bot
npm/[email protected] Transitive: environment, filesystem, network, shell, unsafe +101 14.9 MB adamwathan, malfaitrobin, reinink
npm/[email protected] environment, filesystem, unsafe +16 2.09 MB blakeembrey
npm/[email protected] eval, filesystem Transitive: environment, network, shell, unsafe +73 10 MB egoist
npm/[email protected] environment, network 0 1.16 MB ronag

🚮 Removed packages: npm/@playwright/[email protected], npm/@tailwindcss/[email protected], npm/@types/[email protected], npm/@types/[email protected], npm/@types/[email protected], npm/@types/[email protected], npm/@types/[email protected], npm/[email protected], npm/[email protected], npm/[email protected], npm/[email protected], npm/[email protected], npm/[email protected], npm/[email protected], npm/[email protected], npm/[email protected], npm/[email protected], npm/[email protected], npm/[email protected]

View full report↗︎

packages/blob/src/api.ts Outdated Show resolved Hide resolved
packages/blob/src/api.ts Outdated Show resolved Hide resolved
packages/blob/src/api.ts Show resolved Hide resolved
packages/blob/src/api.ts Show resolved Hide resolved
packages/blob/src/helpers.ts Outdated Show resolved Hide resolved
packages/blob/src/helpers.ts Outdated Show resolved Hide resolved
packages/blob/src/helpers.ts Outdated Show resolved Hide resolved
packages/blob/src/helpers.ts Outdated Show resolved Hide resolved
packages/blob/src/helpers.ts Outdated Show resolved Hide resolved
packages/blob/src/multipart/upload.ts Show resolved Hide resolved
@vvo vvo merged commit c3afec3 into main Nov 6, 2024
14 checks passed
@vvo vvo deleted the feat/blob/on-progress branch November 6, 2024 15:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants