Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: market #301

Draft
wants to merge 45 commits into
base: main
Choose a base branch
from
Draft

feat: market #301

wants to merge 45 commits into from

Conversation

LexLuthr
Copy link
Contributor

@LexLuthr LexLuthr commented Oct 25, 2024

This PR replaces Boost with New Curio market module. Once this PR is merged, Boost will no longer be compatible with Curio.

  • Add deal itests (offline and online)
  • Add some IPNI itest
  • Add retrieval itests
  • Add MoveEscrow to UI
  • Add backpressure
  • Add filters
  • Rearrange pieces in sector before sealing
  • Add Piece Info page
  • Fix Label storage in DB

LexLuthr and others added 30 commits September 17, 2024 19:06
* webui: Jetbrains Mono, fix porep layout (#179)

* Move storiface into Curio (#183)

* temp: replace lotus to local

* move storiface to Curio

* fix build

* don't refer to storiface2

* make gen

* make remote path test happy

* cleanup storiface

* Batch deadline setting in SectorMetadata (#177)

* basic mk12 scafolding

* config comments, http header init

* fix gen test

* fix tests

* poller redesign

* remove indexing table, add defaults

* fix ingest, market tasks

* incomplete basic UI code

* finish libp2p init, fix tests

* fix test-all

* enable libp2p relay

* add url add command

* fix circleCI

* fix everything else

* apply suggestion from review, cleanup todos

* remove import-data

* unify libp2p, atomic market scheduler, config changes

* minor sql fixes

* undo makefile changes

* fix final review

---------

Co-authored-by: Łukasz Magiera <[email protected]>
* basic mk12 scafolding

* fix gen test

* poller redesign

* remove indexing table, add defaults

* fix ingest, market tasks

* incomplete basic UI code

* finish libp2p init, fix tests

* fix test-all

* ipni provider

* fix storeiface, publisher multiaddrs

* close rows

* apply suggestions

* fix error

* fix contextID type, move ipni in market

* gen, remove old market sql
* create http server

* remove loadbalancer, add db cache, set defaults

* update config

* fix lint err
* enables retrievals

* fix itest

* correct test path

* apply changes from review
* wip simpler ipni

* initial smart ipni chunker

* ipni: Complete smart chunker

* ipni: Use batching for initial chunker

* ipni: wire up the better chunker

* make gen

* uncached ipni entry server

* harmonydb: Add retry to .Exec

* ipni: Address review

* indexstore: Don't trim multihashes
* fix ipni http path, add cached reader

* cachedPieceReader library

* add getAdBytes method

* close reader
* fix DealPublishControl address usage

* fix forest test
* remove MarketAddBalance

* make gen
* libp2p migration and generation

* check libp2p rows

* row check in market cli
* cuhttp: Fix startup issues

* cachedreader: Don't reuse io.Reader

* dealdata: Fix URLPieceReader

* retrieval: Fix Routes

* make gen

* ipni: Fix construction

* ipni: More column name fixes

* fix libp2p key serialization

* ipni: task logic fixes

* ipni: get the task into working shape

* make gen
* ipni: Fixed endpoints, working head

* ipni: Fix Ad cids

* ipni: fix chunk output, addrs in ad addresses

* make gen

* I think we need this

* Or not
* libp2p: Fix startup, cleanup config

* ipni: Separate key table

* libp2p: Working single Libp2p

* cuhttp: Libp2p redirector

* make gen

* libp2p: Automatically set actor addresses
* webui: Ask UI

* webui market: Prices in tib/mo

* webui market: Fix price calc

* webui market: Working ask setting

* fix libp2p safehandle

* more fixes to get deals working

* fix market navlink

* last fix to create_indexing_task
* controlled indexing after migration

* shared max counter, ipni complete

* limit count
* deal list page, deal detail page

* fix SQL typos
* feat: ipni ui

* make gen

---------

Co-authored-by: Łukasz Magiera <[email protected]>
* Add MoveEscrow to UI

* fix gen
LexLuthr and others added 9 commits October 30, 2024 23:13
* add backpressure to mk12 deals

* fix backpressure CTE
* rearrange pieces in sector for no padding

* replace transfer funcs on go side

* remove stateful piece offset
* deal filters

* replace price rule number with name
Copy link
Collaborator

@magik6k magik6k left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks basically done, some small comments.

cmd/curio/guidedsetup/guidedsetup.go Outdated Show resolved Hide resolved
cmd/curio/tasks/tasks.go Outdated Show resolved Hide resolved
market/indexstore/create.cql Outdated Show resolved Hide resolved
tasks/seal/task_submit_commit.go Outdated Show resolved Hide resolved
tasks/storage-market/task_find_deal.go Outdated Show resolved Hide resolved
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants