Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

refactor: filecoin api services events and tests #974

Merged
merged 21 commits into from
Oct 23, 2023
Merged

refactor: filecoin api services events and tests #974

merged 21 commits into from
Oct 23, 2023

Conversation

vasco-santos
Copy link
Contributor

@vasco-santos vasco-santos commented Oct 12, 2023 •
edited
Loading

Implements filecoin-api services, events and tests all over the place.

Note that some small follow ups are still needed and will be added as follow up:

  • CRON for piece/accept
Screenshot 2023-10-19 at 23 50 14

@vasco-santos vasco-santos force-pushed the refactor/filecoin-api-2 branch 12 times, most recently from 1ec5761 to 2d860d4 Compare October 19, 2023 16:53
@vasco-santos vasco-santos changed the title Refactor/filecoin api 2 refactor: filecoin api events and services Oct 19, 2023
@vasco-santos vasco-santos changed the title refactor: filecoin api events and services refactor: filecoin api services events and tests Oct 19, 2023
@vasco-santos vasco-santos changed the base branch from main to refactor/filecoin-api October 19, 2023 17:37
@vasco-santos vasco-santos changed the base branch from refactor/filecoin-api to refactor/filecoin-client October 19, 2023 17:37
@vasco-santos vasco-santos force-pushed the refactor/filecoin-api-2 branch 2 times, most recently from ee803fa to 3289cc1 Compare October 19, 2023 21:50
@vasco-santos vasco-santos marked this pull request as ready for review October 19, 2023 21:52
@vasco-santos vasco-santos force-pushed the refactor/filecoin-api-2 branch 4 times, most recently from c78b7e3 to def20a3 Compare October 20, 2023 15:44
alanshaw
alanshaw requested changes Oct 23, 2023
View reviewed changes
Copy link
Member

@alanshaw alanshaw left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wow what a PR. In general this is great. My biggest concern is wrapping errors and losing valuable context.

packages/capabilities/src/types.ts Outdated Show resolved Hide resolved
packages/capabilities/src/types.ts Show resolved Hide resolved
packages/capabilities/src/types.ts
export type FilecoinAcceptFailure =
| InvalidContentPiece
| ProofNotFound
| Ucanto.Failure
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I seem to recall gozala saying we shouldn't add generic Failure to these unions...(there are more instances of this).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

well this looks to be everywhere... Even in the recent capabilities like RateLimit, Customer, ...
If I remove it, there are a lot of typescript failures, so I will keep it and we can consider doing this removal across the board

packages/capabilities/src/types.ts Outdated Show resolved Hide resolved
packages/capabilities/src/types.ts Outdated Show resolved Hide resolved
packages/filecoin-api/src/storefront/service.js Outdated Show resolved Hide resolved
packages/filecoin-api/test/context/mocks.js Show resolved Hide resolved
packages/filecoin-api/test/events/aggregator.js
* @param {API.Test<AggregatorApi.TestEventsContext>} testFn
* @param {(context: AggregatorApi.TestEventsContext) => Promise<AggregatorApi.TestEventsContext>} mockContextFunction
*/
function wichMockableContext(testFn, mockContextFunction) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A helper?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I could not get this to work with the chain of type inference if I put this with templates to try to make it generic with all the contexts being inferred by caller. So keeping it for now to not spend more time

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can fix the typo though?

Suggested change
function wichMockableContext(testFn, mockContextFunction) {
function withMockableContext(testFn, mockContextFunction) {

packages/filecoin-client/src/dealer.js Show resolved Hide resolved
packages/filecoin-client/test/deal-tracker.test.js
provider: 'f099'
}
}
12_345: {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm...why?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

linter wants this...

@vasco-santos vasco-santos merged commit 3bab254 into refactor/filecoin-client Oct 23, 2023
13 of 14 checks passed
@vasco-santos vasco-santos deleted the refactor/filecoin-api-2 branch October 23, 2023 20:38
@vasco-santos vasco-santos mentioned this pull request Oct 23, 2023
Closed
vasco-santos added a commit to storacha/specs that referenced this pull request Oct 23, 2023
@vasco-santos
fix: filecoin spec aggregate accept receipt type (#83)
e469e57 
Fixes aggregate/accept receipt:

1. inclusion proof was wrongly here. It should only be in `piece/accept`
because it is piece specific proof within an aggregate
2. added `aggregate` in receipt to be coherent with other receipts for
`*/accept`

Fixes filecoin/accept receipt:
1. added `piece` and `aggregate` in receipt to be coherent with other
receipts for `*/accept`
2. swaped `aux` to be a property per
storacha/w3up#974 (comment)
vasco-santos added a commit that referenced this pull request Oct 23, 2023
@vasco-santos
refactor!: filecoin api services events and tests (#974)
c18abc8 
Implements `filecoin-api` services, events and tests all over the place.

BREAKING CHANGE: see latest specs https://github.com/web3-storage/specs/blob/cbdb706f18567900c5c24d7fb16ccbaf93d0d023/w3-filecoin.md
vasco-santos added a commit that referenced this pull request Oct 23, 2023
@vasco-santos
refactor!: filecoin api services events and tests (#974)
b80c8e6 
Implements `filecoin-api` services, events and tests all over the place.

BREAKING CHANGE: see latest specs https://github.com/web3-storage/specs/blob/cbdb706f18567900c5c24d7fb16ccbaf93d0d023/w3-filecoin.md
vasco-santos added a commit that referenced this pull request Oct 23, 2023
@vasco-santos
refactor!: filecoin api services events and tests (#974)
d86a667 
Implements `filecoin-api` services, events and tests all over the place.

BREAKING CHANGE: see latest specs https://github.com/web3-storage/specs/blob/cbdb706f18567900c5c24d7fb16ccbaf93d0d023/w3-filecoin.md
vasco-santos added a commit that referenced this pull request Oct 23, 2023
@vasco-santos
refactor!: filecoin api services events and tests (#974)
edb76a3 
Implements `filecoin-api` services, events and tests all over the place.

BREAKING CHANGE: see latest specs https://github.com/web3-storage/specs/blob/cbdb706f18567900c5c24d7fb16ccbaf93d0d023/w3-filecoin.md
vasco-santos added a commit that referenced this pull request Oct 24, 2023
@vasco-santos
refactor!: filecoin api services events and tests (#974)
953537b 
Implements `filecoin-api` services, events and tests all over the place.

BREAKING CHANGE: see latest specs https://github.com/web3-storage/specs/blob/cbdb706f18567900c5c24d7fb16ccbaf93d0d023/w3-filecoin.md
This was referenced Oct 24, 2023
Merged
Merged
Merged
vasco-santos pushed a commit that referenced this pull request Oct 24, 2023
@it-dag-house
chore(main): release capabilities 11.0.0 (#994)
76b0489 
🤖 I have created a release *beep* *boop*
---


##
[11.0.0](capabilities-v10.2.0...capabilities-v11.0.0)
(2023-10-24)


### ⚠ BREAKING CHANGES

* see latest specs
https://github.com/web3-storage/specs/blob/cbdb706f18567900c5c24d7fb16ccbaf93d0d023/w3-filecoin.md
* filecoin client to use new capabilities
* filecoin capabilities

### Bug Fixes

* add missing ContentNotFound definition for filecoin offer failure
([c0b97bf](c0b97bf))
* add missing filecoin submit success and failure types
([c0b97bf](c0b97bf))
* client tests
([b0d9c3f](b0d9c3f))
* type errors
([c0b97bf](c0b97bf))
* upgrade ucanto in filecoin api
([c95fb54](c95fb54))


### Code Refactoring

* filecoin api services events and tests
([#974](#974))
([953537b](953537b))
* filecoin capabilities
([c0b97bf](c0b97bf))
* filecoin client to use new capabilities
([b0d9c3f](b0d9c3f))

---
This PR was generated with [Release
Please](https://github.com/googleapis/release-please). See
[documentation](https://github.com/googleapis/release-please#release-please).
vasco-santos added a commit that referenced this pull request Oct 24, 2023
@it-dag-house @vasco-santos
chore(main): release filecoin-client 2.0.0 (#993)
e7521a8 
🤖 I have created a release *beep* *boop*
---


##
[2.0.0](filecoin-client-v1.4.0...filecoin-client-v2.0.0)
(2023-10-24)


### ⚠ BREAKING CHANGES

* see latest specs
https://github.com/web3-storage/specs/blob/cbdb706f18567900c5c24d7fb16ccbaf93d0d023/w3-filecoin.md
* filecoin client to use new capabilities

### Bug Fixes

* client tests
([b0d9c3f](b0d9c3f))
* upgrade ucanto in filecoin api
([c95fb54](c95fb54))


### Code Refactoring

* filecoin api services events and tests
([#974](#974))
([953537b](953537b))
* filecoin client to use new capabilities
([b0d9c3f](b0d9c3f))

---
This PR was generated with [Release
Please](https://github.com/googleapis/release-please). See
[documentation](https://github.com/googleapis/release-please#release-please).

Co-authored-by: Vasco Santos <[email protected]>
vasco-santos added a commit that referenced this pull request Oct 24, 2023
@it-dag-house @vasco-santos
chore(main): release filecoin-api 2.0.0 (#954)
f7ee580 
🤖 I have created a release *beep* *boop*
---


##
[2.0.0](filecoin-api-v1.4.4...filecoin-api-v2.0.0)
(2023-10-24)


### ⚠ BREAKING CHANGES

* see latest specs
https://github.com/web3-storage/specs/blob/cbdb706f18567900c5c24d7fb16ccbaf93d0d023/w3-filecoin.md

### Features

* upgrade to ucanto@9
([#951](#951))
([d72faf1](d72faf1))


### Bug Fixes

* upgrade ucanto in filecoin api
([c95fb54](c95fb54))


### Code Refactoring

* filecoin api services events and tests
([#974](#974))
([953537b](953537b))

---
This PR was generated with [Release
Please](https://github.com/googleapis/release-please). See
[documentation](https://github.com/googleapis/release-please#release-please).

Co-authored-by: Vasco Santos <[email protected]>
alanshaw pushed a commit to storacha/upload-service that referenced this pull request Nov 5, 2024
@github-actions
chore(main): release capabilities 1.0.0 (#16)
1c0902d 
🤖 I have created a release *beep* *boop*
---


## 1.0.0 (2024-11-05)


### ⚠ BREAKING CHANGES

* add `index/add` handler
([storacha#1421](https://github.com/storacha/upload-service/issues/1421))
* restrict store API to CARs
([storacha#1415](https://github.com/storacha/upload-service/issues/1415))
* **capabilities:** `BlobMultihash` type in `@web3-storage/capabilities`
renamed to `Multihash`.
* allocations storage interface now requires remove to be implemented
* return allocated bytes in `store/add` receipt
([storacha#1213](https://github.com/storacha/upload-service/issues/1213))
* coupon
([storacha#1136](https://github.com/storacha/upload-service/issues/1136))
* see latest specs
https://github.com/web3-storage/specs/blob/cbdb706f18567900c5c24d7fb16ccbaf93d0d023/w3-filecoin.md
* filecoin client to use new capabilities
* filecoin capabilities

### refactor

* filecoin api services events and tests
([storacha#974](https://github.com/storacha/upload-service/issues/974))
([953537b](953537b))
* filecoin capabilities
([c0b97bf](c0b97bf))
* filecoin client to use new capabilities
([b0d9c3f](b0d9c3f))


### Features

* add "plan/create-admin-session" capability
([storacha#1411](https://github.com/storacha/upload-service/issues/1411))
([50eeeb5](50eeeb5))
* add `index/add` handler
([storacha#1421](https://github.com/storacha/upload-service/issues/1421))
([cbe9524](cbe9524))
* add `initialize` method to `PlansStorage`
([storacha#1278](https://github.com/storacha/upload-service/issues/1278))
([6792126](6792126))
* add `store/get` and `upload/get` capabilities
([storacha#942](https://github.com/storacha/upload-service/issues/942))
([40c79eb](40c79eb))
* add `subscription/list` capability
([storacha#1088](https://github.com/storacha/upload-service/issues/1088))
([471d7e5](471d7e5))
* add a function to verify and return Abilities.
([storacha#1252](https://github.com/storacha/upload-service/issues/1252))
([2f026a2](2f026a2))
* add blob list and remove
([storacha#1385](https://github.com/storacha/upload-service/issues/1385))
([2f69946](2f69946))
* add blob protocol to upload-client
([storacha#1425](https://github.com/storacha/upload-service/issues/1425))
([49aef56](49aef56))
* add blob/get
([storacha#1484](https://github.com/storacha/upload-service/issues/1484))
([328039d](328039d))
* add revocation to access-client and w3up-client
([storacha#975](https://github.com/storacha/upload-service/issues/975))
([6c877aa](6c877aa))
* add usage/report capability
([storacha#1079](https://github.com/storacha/upload-service/issues/1079))
([6418b4b](6418b4b))
* blob, web3.storage and ucan conclude capabilities together with api
handlers
([storacha#1342](https://github.com/storacha/upload-service/issues/1342))
([00735a8](00735a8))
* **capabilities:** add `index/add` capability
([storacha#1410](https://github.com/storacha/upload-service/issues/1410))
([1b71b89](1b71b89))
* change `plan/update` to `plan/set` and use existing `PlansStorage#set`
to implement an invocation handler
([storacha#1258](https://github.com/storacha/upload-service/issues/1258))
([1ccbfe9](1ccbfe9))
* coupon
([storacha#1136](https://github.com/storacha/upload-service/issues/1136))
([1b94f2d](1b94f2d))
* filecoin info
([storacha#1091](https://github.com/storacha/upload-service/issues/1091))
([adb2442](adb2442))
* Generate Space proofs on the fly, on `access/claim`
([storacha#1555](https://github.com/storacha/upload-service/issues/1555))
([9e2b1d4](9e2b1d4))
* implement `plan/get` capability
([storacha#1005](https://github.com/storacha/upload-service/issues/1005))
([f0456d2](f0456d2))
* introduce capability for changing billing plan
([storacha#1253](https://github.com/storacha/upload-service/issues/1253))
([d33b3a9](d33b3a9))
* move aggregate information out of deals in filecoin/info
([storacha#1192](https://github.com/storacha/upload-service/issues/1192))
([18dc590](18dc590))
* publish index claim
([storacha#1487](https://github.com/storacha/upload-service/issues/1487))
([237b0c6](237b0c6))
* restrict store API to CARs
([storacha#1415](https://github.com/storacha/upload-service/issues/1415))
([e53aa87](e53aa87))
* return allocated bytes in `store/add` receipt
([storacha#1213](https://github.com/storacha/upload-service/issues/1213))
([5d52e44](5d52e44))
* router ([#11](#11))
([c810735](c810735))
* upgrade ucanto/transport to 9.1.0 in all packages to get more verbose
errors from HTTP transport on non-ok response
([storacha#1312](https://github.com/storacha/upload-service/issues/1312))
([d6978d7](d6978d7))
* usage/record capability definition
([storacha#1562](https://github.com/storacha/upload-service/issues/1562))
([98c8a87](98c8a87))
* wip router
([ffcd9c7](ffcd9c7))


### Fixes

* add missing ContentNotFound definition for filecoin offer failure
([c0b97bf](c0b97bf))
* add missing filecoin submit success and failure types
([c0b97bf](c0b97bf))
* capabilities should export blob caps
([storacha#1376](https://github.com/storacha/upload-service/issues/1376))
([460729e](460729e))
* client tests
([b0d9c3f](b0d9c3f))
* fix arethetypesworking errors in all packages
([storacha#1004](https://github.com/storacha/upload-service/issues/1004))
([2e2936a](2e2936a))
* issue where typedoc docs would only show full docs for w3up-client
([storacha#1141](https://github.com/storacha/upload-service/issues/1141))
([0b8d3f3](0b8d3f3))
* migrate repo
([storacha#1389](https://github.com/storacha/upload-service/issues/1389))
([475a287](475a287))
* one more tweak to the `PlanStorage` interface
([storacha#1280](https://github.com/storacha/upload-service/issues/1280))
([5a44565](5a44565))
* package metadata
([storacha#1161](https://github.com/storacha/upload-service/issues/1161))
([b8a1cc2](b8a1cc2))
* put access.session back
([storacha#1100](https://github.com/storacha/upload-service/issues/1100))
([10a1a4b](10a1a4b))
* rename blob and index client capabilities
([storacha#1478](https://github.com/storacha/upload-service/issues/1478))
([17e3a31](17e3a31))
* repo URLs
([storacha#1550](https://github.com/storacha/upload-service/issues/1550))
([e02ddf3](e02ddf3))
* tests
([b179910](b179910))
* trigger capabilities release
([storacha#1399](https://github.com/storacha/upload-service/issues/1399))
([7d9ab35](7d9ab35))
* type errors
([c0b97bf](c0b97bf))
* update data-segment dep
([228ff79](228ff79))
* upgrade @ucanto/validator with bugfix
([storacha#1151](https://github.com/storacha/upload-service/issues/1151))
([d4e961b](d4e961b))
* upgrade ucanto core
([storacha#1127](https://github.com/storacha/upload-service/issues/1127))
([5ce4d22](5ce4d22))
* upgrade ucanto in filecoin api
([c95fb54](c95fb54))
* upgrade ucanto libs and format filecoin api
([storacha#1359](https://github.com/storacha/upload-service/issues/1359))
([87ca098](87ca098))
* upload API test fixes
([6b0d72d](6b0d72d))


### Other Changes

* Add `pnpm dev` to watch-build all packages
([storacha#1533](https://github.com/storacha/upload-service/issues/1533))
([07970ef](07970ef))
* **main:** release capabilities 10.1.0
([storacha#979](https://github.com/storacha/upload-service/issues/979))
([bdd3970](bdd3970))
* **main:** release capabilities 10.2.0
([storacha#983](https://github.com/storacha/upload-service/issues/983))
([a906488](a906488))
* **main:** release capabilities 11.0.0
([storacha#994](https://github.com/storacha/upload-service/issues/994))
([76b0489](76b0489))
* **main:** release capabilities 11.0.1
([storacha#1008](https://github.com/storacha/upload-service/issues/1008))
([37cdc5a](37cdc5a))
* **main:** release capabilities 11.1.0
([storacha#1026](https://github.com/storacha/upload-service/issues/1026))
([7fdb541](7fdb541))
* **main:** release capabilities 11.2.0
([storacha#1084](https://github.com/storacha/upload-service/issues/1084))
([0e7b6dc](0e7b6dc))
* **main:** release capabilities 11.3.0
([storacha#1098](https://github.com/storacha/upload-service/issues/1098))
([7d671bd](7d671bd))
* **main:** release capabilities 11.3.1
([storacha#1101](https://github.com/storacha/upload-service/issues/1101))
([20b5b35](20b5b35))
* **main:** release capabilities 11.4.0
([storacha#1105](https://github.com/storacha/upload-service/issues/1105))
([1b6210f](1b6210f))
* **main:** release capabilities 11.4.1
([storacha#1131](https://github.com/storacha/upload-service/issues/1131))
([a5b7154](a5b7154))
* **main:** release capabilities 12.0.0
([storacha#1137](https://github.com/storacha/upload-service/issues/1137))
([bb23e9f](bb23e9f))
* **main:** release capabilities 12.0.1
([storacha#1147](https://github.com/storacha/upload-service/issues/1147))
([d1a9c78](d1a9c78))
* **main:** release capabilities 12.0.2
([storacha#1152](https://github.com/storacha/upload-service/issues/1152))
([b9d7ff5](b9d7ff5))
* **main:** release capabilities 12.0.3
([storacha#1163](https://github.com/storacha/upload-service/issues/1163))
([ec5c385](ec5c385))
* **main:** release capabilities 12.1.0
([storacha#1195](https://github.com/storacha/upload-service/issues/1195))
([a21c1a5](a21c1a5))
* **main:** release capabilities 13.0.0
([storacha#1230](https://github.com/storacha/upload-service/issues/1230))
([3d5b3ef](3d5b3ef))
* **main:** release capabilities 13.1.0
([storacha#1257](https://github.com/storacha/upload-service/issues/1257))
([85adc9a](85adc9a))
* **main:** release capabilities 13.1.1
([storacha#1283](https://github.com/storacha/upload-service/issues/1283))
([31c38e9](31c38e9))
* **main:** release capabilities 13.2.0
([storacha#1315](https://github.com/storacha/upload-service/issues/1315))
([0505458](0505458))
* **main:** release capabilities 13.2.1
([storacha#1362](https://github.com/storacha/upload-service/issues/1362))
([26b5751](26b5751))
* **main:** release capabilities 13.3.0
([storacha#1366](https://github.com/storacha/upload-service/issues/1366))
([d6fbc4a](d6fbc4a))
* **main:** release capabilities 13.3.1
([storacha#1377](https://github.com/storacha/upload-service/issues/1377))
([149f592](149f592))
* **main:** release capabilities 14.0.0
([storacha#1386](https://github.com/storacha/upload-service/issues/1386))
([69bfc08](69bfc08))
* **main:** release capabilities 14.0.1
([storacha#1395](https://github.com/storacha/upload-service/issues/1395))
([a76c970](a76c970))
* **main:** release capabilities 14.0.2
([storacha#1400](https://github.com/storacha/upload-service/issues/1400))
([7b46852](7b46852))
* **main:** release capabilities 15.0.0
([storacha#1412](https://github.com/storacha/upload-service/issues/1412))
([ec90b81](ec90b81))
* **main:** release capabilities 16.0.0
([storacha#1419](https://github.com/storacha/upload-service/issues/1419))
([50e3934](50e3934))
* **main:** release capabilities 17.0.0
([storacha#1428](https://github.com/storacha/upload-service/issues/1428))
([6ee21fd](6ee21fd))
* **main:** release capabilities 17.1.0
([storacha#1447](https://github.com/storacha/upload-service/issues/1447))
([8ee1fd9](8ee1fd9))
* **main:** release capabilities 17.1.1
([storacha#1483](https://github.com/storacha/upload-service/issues/1483))
([5394ed5](5394ed5))
* **main:** release capabilities 17.2.0
([storacha#1494](https://github.com/storacha/upload-service/issues/1494))
([99876a5](99876a5))
* **main:** release capabilities 17.3.0
([storacha#1503](https://github.com/storacha/upload-service/issues/1503))
([891c2a5](891c2a5))
* **main:** release capabilities 17.4.0
([storacha#1559](https://github.com/storacha/upload-service/issues/1559))
([feaea7a](feaea7a))
* no longer depends on hd-scripts, packages use/configure eslint
directly, fixes warnings from npm lint script
([storacha#1058](https://github.com/storacha/upload-service/issues/1058))
([ebdb99b](ebdb99b))
* package renames
([0f797ed](0f797ed))

---
This PR was generated with [Release
Please](https://github.com/googleapis/release-please). See
[documentation](https://github.com/googleapis/release-please#release-please).

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@alanshaw alanshaw alanshaw approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@vasco-santos @alanshaw