feat: HTTP to UCAN bridge #325
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
travis
added a commit
to storacha/w3cli
that referenced
this pull request
Feb 7, 2024
In service of the UCAN bridge implementation in storacha/w3infra#325 this PR proposes a new `bridge generate-tokens` command that will generate values for the `Authorization` header and `proof` field of the HTTP request users make to the bridge. This PR is currently intended as a stake-in-the-ground and is expected to change significantly. TODO - [ ] clean up and refactor bridge - [ ] finalize command names - [ ] decouple from coupon code (?)
travis
requested review from
alanshaw,
vasco-santos,
mikeal,
Gozala,
olizilla and
gobengo
gobengo
reviewed
Feb 7, 2024
alanshaw
requested changes
Feb 7, 2024
There was a problem hiding this comment.
❤️ love it - thanks for pushing this!
Worth mentioning what you're doing is storacha/RFC#1 (comment)
Can you please define response format?
Originally I read this as meaning it was base64 encoded with padding. https://datatracker.ietf.org/doc/html/rfc4648#section-4 It could be good to clarify you mean it is multibase encoded using the multibase encoding for base64+padding. https://www.ietf.org/archive/id/draft-multiformats-multibase-07.html#name-base-64-with-padding-and-mi |
<3
Yes! Though there's a pretty big difference in the authorization - I'm not "passing UCAN delegation as JWT bearer token" because of the 8KB limit I mention in the thread with Ben above, instead passing the secret there and embedding the delegation in the JSON body.
Yep definitely - added a TODO! |
To support users in languages that do not have existing UCAN invocation implementations, we are going to launch a bridge that allows them to make simple HTTP requests with JSON bodies that we transform into proper UCAN invocations. So far this PR has: 1) an untested (but type-checking!) implementation of such a bridge 2) a markdown description of the bridge protocol, intended to be the first draft of an eventual specification Notable design choices: 1) I chose to include JUST the base64pad-encoded "secret" in the Authorization header to avoid running afoul of maximum header size restrictions that exist in [some HTTP environments](https://stackoverflow.com/questions/686217/maximum-on-http-header-values). 2) The `proof` field of the JSON body is a base64pad encoded "delegation archive" (created with `ucanto`'s `Delegation.archive` function) TODO - [ ] factor core bridge logic out to a separate library - [ ] factor HTTP input wrangling out to a separate function - [ ] rename `UPLOAD_API_DID` and `ACCESS_SERVICE_URL` environment variables to `W3UP_SERVICE_DID` and `W3UP_SERVICE_URL` - [ ] add tests - [ ] expand and formalize bridge specification, move it to the specs repo (?)
travis
force-pushed
the
feat/ucan-bridge
branch
from
ec6bc44
to
5ebd1ea
Compare
View stack outputs
|
update spec to use https:// github.com/ucan-wg/ucan-http-bearer-token?tab=readme-ov-file#ucan-as-bearer-token-specification-v030 and move the secret to a new `X-Auth-Secret` header
update implementation per this spec - currently typechecks cleanly but throws an error when I actually try to invoke:
```
TypeError: proof.export is not a function or its return value is not iterable
at delegate (/Users/travis/dev/pl/w3infra/node_modules/@ucanto/core/src/delegation.js:466:33)
at IssuedInvocation.buildIPLDView (/Users/travis/dev/pl/w3infra/node_modules/@ucanto/core/src/invocation.js:101:12)
at writeInvocations (/Users/travis/dev/pl/w3infra/node_modules/@ucanto/core/src/message.js:114:35)
at MessageBuilder.buildIPLDView (/Users/travis/dev/pl/w3infra/node_modules/@ucanto/core/src/message.js:79:52)
at Object.build (/Users/travis/dev/pl/w3infra/node_modules/@ucanto/core/src/message.js:30:49)
at execute (/Users/travis/dev/pl/w3infra/node_modules/@ucanto/client/src/connection.js:48:31)
at Connection.execute (/Users/travis/dev/pl/w3infra/node_modules/@ucanto/client/src/connection.js:35:12)
at IssuedInvocation.execute (/Users/travis/dev/pl/w3infra/node_modules/@ucanto/core/src/invocation.js:115:39)
at handlerFn (/Users/travis/dev/pl/w3infra/upload-api/functions/bridge.js:124:38)
at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
```
I think this is because the `@ipld/dag-ucan` `Ucan.View` is not a full `Delegation`
Gozala
previously requested changes
Feb 12, 2024
1) add content negotiation and support CBOR 2) better parsing abstractions 3) validate tasks before casting one slightly weird thing I'm doing is converting the string I get from the AWS lambda to a ReadableStream and then reading it back again - seems silly in this implementation, but is necessary because I plan to pull out the abstraction and support other deployment environments like Cloudflare workers or Node's `fetch` API.
also error on unsupported content type
The bridge was configured to point at staging even when it was running in a PR environment, because we don't customize the `ACCESS_SERVICE_URL` in those environments. It would be nice to figure out how to set the proper environment variables in different environments, but I suspect the right way to do that might change with the SST upgrade, so for now let's just add a function that encodes the rules we know, since these aren't likely to change soon and would require some code changes anyway.
alanshaw
approved these changes
Mar 5, 2024
| message: `${contentType} is not supported` | ||
| } | ||
| } | ||
| } |
There was a problem hiding this comment.
Try/catch around this incase decode throws?
upload-api/functions/bridge.js
Outdated
Comment on lines
147
to
148
| p: receipt.root.data?.ocm, | ||
| s: receipt.root.data?.sig |
There was a problem hiding this comment.
I hope typescript understands that you just checked for existence?
Suggested change
| p: receipt.root.data?.ocm, | |
| s: receipt.root.data?.sig | |
| p: receipt.root.data.ocm, | |
| s: receipt.root.data.sig |
upload-api/functions/bridge.js
Outdated
| if (!authorizationHeader) { | ||
| return { | ||
| statusCode: 401, | ||
| body: 'request has no Authorization header' |
There was a problem hiding this comment.
I like to say "missing" rather than "request has no" 😬
| return bridgeReceipts.ok ? { | ||
| statusCode: 200, | ||
| headers: { | ||
| "Content-Type": "application/vnd.ipld.dag-json" |
There was a problem hiding this comment.
Would be nice if these had a content type export like CAR...how do you feel about sending a couple of PRs?
There was a problem hiding this comment.
oo I like that - will dig into that today!
travis
added a commit
to storacha/w3cli
that referenced
this pull request
Mar 5, 2024
In service of the UCAN bridge implementation in storacha/w3infra#325 this PR introduces a new `bridge generate-tokens` command that will generate values for the `X-Auth-Secret` and `Authorization` headers of the HTTP request users make to the bridge.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
To support users in languages that do not have existing UCAN invocation implementations, we are going to launch a bridge that allows them to make simple HTTP requests with JSON bodies that we transform into proper UCAN invocations.
This follows the specification here: storacha/specs#112
Values for authorization headers can be generated using the
bridge generate-tokensw3cli command proposed here:storacha/w3cli#175
UPLOAD_API_DIDandACCESS_SERVICE_URLenvironment variables toW3UP_SERVICE_DIDandW3UP_SERVICE_URL(filed asUPLOAD_SERVICE_DIDandACCESS_SERVICE_URLshould be deprecated #337)