-
Notifications
You must be signed in to change notification settings - Fork 2.2k
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
Showing
58 changed files
with
7,921 additions
and
168 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,82 @@ | ||
| name: Blobstorage | ||
|
|
||
| on: | ||
| push: | ||
| branches: [main] | ||
| paths: | ||
| - "packages/blobstorage/**" | ||
| pull_request: | ||
| paths: | ||
| - "packages/blobstorage/**" | ||
|
|
||
| jobs: | ||
| test: | ||
| runs-on: [taiko-runner] | ||
| steps: | ||
| - name: Cancel Previous Runs | ||
| uses: styfle/[email protected] | ||
| with: | ||
| access_token: ${{ github.token }} | ||
|
|
||
| - uses: actions/checkout@v4 | ||
| - uses: actions/setup-go@v5 | ||
| with: | ||
| go-version: "1.21.0" | ||
|
|
||
| - name: blobstorage - Unit Tests | ||
| working-directory: ./packages/blobstorage | ||
| run: go test `go list ./... | grep -v ./bindings | grep -v ./mock | grep -v ./cmd` -coverprofile=coverage.txt -covermode=atomic | ||
|
|
||
| - name: blobstorage - Upload coverage to Codecov | ||
| uses: codecov/codecov-action@v4 | ||
| with: | ||
| files: ./packages/blobstorage/coverage.txt | ||
| flags: blobstorage | ||
|
|
||
| push-docker-image: | ||
| # only push docker image on PR merge to main | ||
| if: ${{ github.event }} == 'push' | ||
| name: Build and push docker image | ||
| runs-on: [taiko-runner] | ||
|
|
||
| steps: | ||
| - name: Checkout | ||
| uses: actions/checkout@v4 | ||
| with: | ||
| ref: ${{ github.ref }} | ||
|
|
||
| - name: Login to GCR | ||
| uses: docker/login-action@v3 | ||
| with: | ||
| registry: gcr.io | ||
| username: _json_key | ||
| password: ${{ secrets.GCR_JSON_KEY }} | ||
|
|
||
| - name: Set up QEMU | ||
| uses: docker/setup-qemu-action@v3 | ||
|
|
||
| - name: Set up Docker Buildx | ||
| uses: docker/setup-buildx-action@v3 | ||
|
|
||
| - name: Docker meta | ||
| id: meta | ||
| uses: docker/metadata-action@v5 | ||
| with: | ||
| images: | | ||
| gcr.io/evmchain/blobstorage | ||
| tags: | | ||
| type=ref,event=branch | ||
| type=ref,event=pr | ||
| type=ref,event=tag | ||
| type=sha | ||
| - name: Build and push | ||
| uses: docker/build-push-action@v5 | ||
| with: | ||
| platforms: linux/amd64 | ||
| push: true | ||
| context: . | ||
| tags: ${{ steps.meta.outputs.tags }} | ||
| labels: ${{ steps.meta.outputs.labels }} | ||
| build-args: | | ||
| PACKAGE=blobstorage |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -30,6 +30,7 @@ jobs: | |
| main | ||
| deps | ||
| deps-dev | ||
| blobstorage | ||
| branding | ||
| bridge-ui | ||
| eventindexer | ||
|
|
||
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,9 @@ | ||
| RPC_URL=wss://l1ws.internal.taiko.xyz | ||
| BEACON_URL=https://l1beacon.internal.taiko.xyz/eth/v1/beacon/blob_sidecars/ | ||
| TAIKO_L1_CONTRACT_ADDRESS=0xC069c3d2a9f2479F559AD34485698ad5199C555f | ||
| DATABASE_HOST=localhost | ||
| DATABASE_PORT=27017 | ||
| DATABASE_USER=root | ||
| DATABASE_PASSWORD=root | ||
| DATABASE_NAME=blobs | ||
| METRICS_HTTP_PORT=7472 |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,7 @@ | ||
| HTTP_PORT=3282 | ||
| DATABASE_HOST=localhost | ||
| DATABASE_PORT=27017 | ||
| DATABASE_USER=root | ||
| DATABASE_PASSWORD=root | ||
| DATABASE_NAME=blobs | ||
| METRICS_HTTP_PORT=7471 |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,21 @@ | ||
| # If you prefer the allow list template instead of the deny list, see community template: | ||
| # https://github.com/github/gitignore/blob/main/community/Golang/Go.AllowList.gitignore | ||
| # | ||
| # Binaries for programs and plugins | ||
| *.exe | ||
| *.exe~ | ||
| *.dll | ||
| *.so | ||
| *.dylib | ||
|
|
||
| # Test binary, built with `go test -c` | ||
| *.test | ||
|
|
||
| # Output of the go coverage tool, specifically when used with LiteIDE | ||
| *.out | ||
|
|
||
| # Dependency directories (remove the comment below to include it) | ||
| # vendor/ | ||
|
|
||
| # Go workspace file | ||
| go.work |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,55 @@ | ||
| # blob-storage | ||
|
|
||
| Repo for BLOB storage (archive and serve data) | ||
|
|
||
| ## how to run ? | ||
|
|
||
| Prerequisite is to have docker engine up and running. | ||
|
|
||
| 1. Start the mongoDB | ||
|
|
||
| ```bash | ||
| cd local_docker && docker-compose up -d | ||
| ``` | ||
|
|
||
| 2. Start the `indexer` | ||
|
|
||
| ```bash | ||
| ENV_FILE=.default.indexer.env run cmd/main.go indexer | ||
| ``` | ||
|
|
||
| By default the above command starts the app from the latest block height. If we want to specifiy a previous blockheight, we can run change it from the `.default.indexer.env` file, by adding a `STARTING_BLOCK_ID` variable. | ||
|
|
||
| 2. Start the `server`. | ||
|
|
||
| ```bash | ||
| ENV_FILE=.default.server.env run cmd/main.go server | ||
| ``` | ||
|
|
||
| ## how to test / use ? | ||
|
|
||
| When the `DB`, `blob-catcher` and `server` is running, the `blob-catcher` is outputting the `blobHash` to the terminal (with the `networkName` variable too, tho it is not written into the DB). Use that `blobHash` (including the 0x) in | ||
|
|
||
| 1. Either in a curl command like this (you can query multiple blobHashes - comma separated - with one go and the result will be a respective array): | ||
|
|
||
| ```bash | ||
| curl -X GET "http://localhost:27001/getBlob?blobHash=0x01a2a1cdc7ad221934061642a79a760776a013d0e6fa1a1c6b642ace009c372a,0xWRONG_HASH" | ||
| ``` | ||
|
|
||
| The result will be something like this: | ||
|
|
||
| ```bash | ||
| {"data":[{"blob":"0x123...00","kzg_commitment":"0xabd68b406920aa74b83cf19655f1179d373b5a8cba21b126b2c18baf2096c8eb9ab7116a89b375546a3c30038485939e"}, {"blob":"NOT_FOUND","kzg_commitment":"NOT_FOUND"}]} | ||
| ``` | ||
|
|
||
| 2. Or to backtest, use the simple python script below, after overwriting the `blob_hash` variable: | ||
|
|
||
| ```bash | ||
| python3 python_query.py | ||
| ``` | ||
|
|
||
| ## todos | ||
|
|
||
| What is still missing is: | ||
|
|
||
| - small refinements and DevOps (prod-grade DB with creditentials, proper containerization) |
Oops, something went wrong.