Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Sprint 1.16 #1107

Open
wants to merge 70 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
70 commits
Select commit Hold shift + click to select a range
f2060d9
go mod tidy
Jayashsatolia403 Mar 10, 2024
da47b3c
Restricted blobber tests
Jayashsatolia403 Mar 20, 2024
a9422df
Changed repo snapshots
Jayashsatolia403 Mar 20, 2024
27ac46f
Merge branch 'master' into sprint-1.14
Jayashsatolia403 Mar 26, 2024
e0994b0
Merge branch 'refs/heads/master' into sprint-1.14
Jayashsatolia403 Apr 2, 2024
d21f717
Merge branch 'refs/heads/master' into sprint-1.14
Jayashsatolia403 Apr 2, 2024
803ba50
Skipped authorizer tests
Jayashsatolia403 Apr 2, 2024
49f095b
Merge branch 'refs/heads/master' into sprint-1.14
Jayashsatolia403 Apr 2, 2024
f1cbede
add owner api calls (#1047)
Sanchit011 Apr 7, 2024
444f542
Added feature blobber auth tests
Jayashsatolia403 Apr 8, 2024
6a6e660
Fix API tests create allocation
Jayashsatolia403 Apr 8, 2024
4a99711
Updated gosdk
Jayashsatolia403 Apr 8, 2024
37d9664
Merge branch 'refs/heads/sprint-1.14' into feature/blobber-auth
Jayashsatolia403 Apr 8, 2024
90f92ec
Updated gosdk
Jayashsatolia403 Apr 8, 2024
3990f94
Fix
Jayashsatolia403 Apr 8, 2024
32f1bc1
Fix
Jayashsatolia403 Apr 8, 2024
eff1e13
Fix blobber config update test =
Jayashsatolia403 Apr 8, 2024
48f7f65
Fix blobber config update cleanup
Jayashsatolia403 Apr 8, 2024
ff1b743
Fix
Jayashsatolia403 Apr 8, 2024
d5cb623
Hotfix/0box (#1049)
Sanchit011 Apr 9, 2024
b114d8e
Updated gosdk
Jayashsatolia403 Apr 11, 2024
c0218cd
fix commit err message (#1052)
Hitenjain14 Apr 11, 2024
274393e
Fix
Jayashsatolia403 Apr 14, 2024
5d494ef
Fix
Jayashsatolia403 Apr 14, 2024
f7d76d8
Fix
Jayashsatolia403 Apr 14, 2024
03b3e24
Fix
Jayashsatolia403 Apr 14, 2024
f557e07
Fix
Jayashsatolia403 Apr 14, 2024
5906a73
Fix
Jayashsatolia403 Apr 14, 2024
c4b29b2
Fix lint
Jayashsatolia403 Apr 14, 2024
67ae7ee
Fix
Jayashsatolia403 Apr 14, 2024
cd10097
Merge pull request #1031 from 0chain/feature/blobber-auth
dabasov Apr 14, 2024
619a423
tests to list stakable providers (#1036)
storybehind Apr 15, 2024
43f0884
Fix 0box tests with create allocation
Jayashsatolia403 Apr 15, 2024
069c96a
fix 0box tests
Sanchit011 Apr 15, 2024
9ba204a
Empty commit
Jayashsatolia403 Apr 16, 2024
1e20724
Fix restricted blobber tests
Jayashsatolia403 Apr 16, 2024
be7287d
Merge pull request #1054 from 0chain/fix/0box-tests
dabasov Apr 16, 2024
6a49c1d
Merge branch 'refs/heads/master' into sprint-1.14
Jayashsatolia403 Apr 22, 2024
11a6f88
Merge branch 'master' into sprint-1.14
Jayashsatolia403 Apr 22, 2024
a455567
Merge branch 'refs/heads/master' into sprint-1.14
Jayashsatolia403 Apr 22, 2024
ca51ad2
Update gosdk
Jayashsatolia403 Apr 23, 2024
b57c34f
Fix sharder config update
Jayashsatolia403 Apr 24, 2024
c826906
fix delete wallet (#1057)
Sanchit011 Apr 27, 2024
a64f648
Fixed file format and restricted blobbers test
Jayashsatolia403 Apr 28, 2024
055f288
Fix resume upload
Jayashsatolia403 Apr 28, 2024
6f8e983
Fix restricted blobbers
Jayashsatolia403 Apr 28, 2024
763c02c
fix resume download test (#1058)
Hitenjain14 Apr 28, 2024
22c07a9
Fix restricted blobbers
Jayashsatolia403 Apr 29, 2024
6581305
Fix restricted blobbers
Jayashsatolia403 Apr 29, 2024
df4597d
Fix restricted blobbers
Jayashsatolia403 Apr 29, 2024
15d8c16
added repair allocation with diff file size (#1034)
Zorro30 Apr 30, 2024
2772ccc
added test suite for dropbox
pewssh May 1, 2024
967e5cf
test suite for gdrive
pewssh May 1, 2024
3dba307
updated for workflow
pewssh May 2, 2024
a17b590
lint fix
pewssh May 2, 2024
acc5176
changes for dropbox tests
pewssh May 5, 2024
66cdcd6
Fix: Issue previous test cases
pewssh May 5, 2024
fe1aeb9
Added lib
pewssh Jun 13, 2024
3a32014
Resolve conflicts
pewssh Jun 13, 2024
4a7553e
disable lint
pewssh Jun 13, 2024
8eb4645
add TestRepairSize
storybehind Jun 14, 2024
cf72129
add cli tests
storybehind Jun 14, 2024
8e451f9
update gosdk version
storybehind Jun 14, 2024
5a98103
apply gofmt
storybehind Jun 14, 2024
06eee02
Merge pull request #1071
Jayashsatolia403 Jun 16, 2024
ce23fa8
Merge branch 'master' into sprint-1.16
Jayashsatolia403 Jun 18, 2024
8ef432a
Merge pull request #1072 from 0chain/repair-size
dabasov Jun 21, 2024
50b905a
Feature: migrate from the usage of Tenderly forks in favour of Tender…
YarikRevich Jul 6, 2024
679cdc7
hotfix: applied tenderly virtual network setup for all the workflows …
YarikRevich Jul 6, 2024
3640817
Merge branch 'master' into sprint-1.16
Jayashsatolia403 Jul 7, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -166,6 +166,8 @@ jobs:
run_smoke_tests: ${{ env.RUN_SMOKE_TESTS }}
S3_ACCESS_KEY: ${{ secrets.S3_ACCESS_KEY }}
S3_SECRET_KEY: ${{ secrets.S3_SECRET_KEY }}
DROPBOX_ACCESS_TOKEN: ${{ secrets.DROPBOX_ACCESS_TOKEN }}
GDRIVE_ACCESS_TOKEN: ${{ secrets.GDRIVE_ACCESS_TOKEN }}

- name: "Remove Tenderly virtual testnet"
if: always()
Expand Down
17 changes: 10 additions & 7 deletions go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -2,9 +2,11 @@ module github.com/0chain/system_test

go 1.21

toolchain go1.22.1

require (
github.com/0chain/errors v1.0.3
github.com/0chain/gosdk v1.14.0-RC2
github.com/0chain/gosdk v1.15.2-0.20240614082721-1b8ad4f48bc6
github.com/go-resty/resty/v2 v2.7.0
github.com/herumi/bls-go-binary v1.31.0
github.com/shopspring/decimal v1.3.1
Expand All @@ -13,17 +15,18 @@ require (
github.com/stretchr/testify v1.8.4
github.com/tyler-smith/go-bip39 v1.1.0
github.com/ybbus/jsonrpc/v3 v3.1.5 // nolint
golang.org/x/crypto v0.17.0
golang.org/x/crypto v0.24.0
gopkg.in/errgo.v2 v2.1.0
gopkg.in/yaml.v3 v3.0.1
gorm.io/gorm v1.24.1
)

require (
github.com/andybalholm/brotli v1.0.5 // indirect
github.com/google/go-cmp v0.6.0 // indirect
github.com/hashicorp/golang-lru v0.5.5-0.20210104140557-80c98217689d // indirect
github.com/hashicorp/golang-lru/v2 v2.0.1 // indirect
github.com/hitenjain14/fasthttp v0.0.0-20240229173600-722723e15e17 // indirect
github.com/hitenjain14/fasthttp v0.0.0-20240527123209-06019e79bff9 // indirect
github.com/klauspost/compress v1.17.0 // indirect
github.com/lithammer/shortuuid/v3 v3.0.7 // indirect
github.com/minio/sha256-simd v1.0.1 // indirect
Expand Down Expand Up @@ -96,10 +99,10 @@ require (
github.com/yusufpapurcu/wmi v1.2.3 // indirect
go.dedis.ch/fixbuf v1.0.3 // indirect
go.dedis.ch/kyber/v3 v3.1.0 // indirect
golang.org/x/net v0.19.0 // indirect
golang.org/x/sync v0.5.0 // indirect
golang.org/x/sys v0.15.0 // indirect
golang.org/x/text v0.14.0 // indirect
golang.org/x/net v0.26.0 // indirect
golang.org/x/sync v0.7.0 // indirect
golang.org/x/sys v0.21.0 // indirect
golang.org/x/text v0.16.0 // indirect
google.golang.org/appengine v1.6.7 // indirect
google.golang.org/protobuf v1.28.1 // indirect
gopkg.in/ini.v1 v1.67.0 // indirect
Expand Down
27 changes: 15 additions & 12 deletions go.sum
Original file line number Diff line number Diff line change
Expand Up @@ -42,6 +42,8 @@ github.com/0chain/errors v1.0.3 h1:QQZPFxTfnMcRdt32DXbzRQIfGWmBsKoEdszKQDb0rRM=
github.com/0chain/errors v1.0.3/go.mod h1:xymD6nVgrbgttWwkpSCfLLEJbFO6iHGQwk/yeSuYkIc=
github.com/0chain/gosdk v1.14.0-RC2 h1:OIpjj7mDKxVqJVlxJKm8/fPzdJRN5roXO7/plBCtmTg=
github.com/0chain/gosdk v1.14.0-RC2/go.mod h1:tgAiVAuIy+Vs1tGfKCPEuuWWARwNQBEw32y950LrqrU=
github.com/0chain/gosdk v1.15.2-0.20240614082721-1b8ad4f48bc6 h1:f8fN6PuvE/Iu1LqrJH5q20eWOFB/NjlDA8TIDMqZgAg=
github.com/0chain/gosdk v1.15.2-0.20240614082721-1b8ad4f48bc6/go.mod h1:Hzl56JJ66ZmoyNS7CbTJue7wUugBYvNx8/qJzTRWmkI=
github.com/BurntSushi/toml v0.3.1/go.mod h1:xHWCNGjB5oqiDr8zfno3MHue2Ht5sIBksp03qcyfWMU=
github.com/BurntSushi/xgb v0.0.0-20160522181843-27f122750802/go.mod h1:IVnqGOEym/WlBOVXweHU+Q+/VP0lqqI8lqeDx9IjBqo=
github.com/Luzifer/go-openssl/v3 v3.1.0 h1:QqKqo6kYXGGUsvtUoCpRZm8lHw+jDfhbzr36gVj+/gw=
Expand Down Expand Up @@ -197,8 +199,8 @@ github.com/google/go-cmp v0.5.1/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/
github.com/google/go-cmp v0.5.2/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
github.com/google/go-cmp v0.5.4/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
github.com/google/go-cmp v0.5.5/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
github.com/google/go-cmp v0.5.9 h1:O2Tfq5qg4qc4AmwVlvv0oLiVAGB7enBSJ2x2DqQFi38=
github.com/google/go-cmp v0.5.9/go.mod h1:17dUlkBOakJ0+DkrSSNjCkIjxS6bF9zb3elmeNGIjoY=
github.com/google/go-cmp v0.6.0 h1:ofyhxvXcZhMsU5ulbFiLKl/XBFqE1GSq7atu8tAmTRI=
github.com/google/go-cmp v0.6.0/go.mod h1:17dUlkBOakJ0+DkrSSNjCkIjxS6bF9zb3elmeNGIjoY=
github.com/google/martian v2.1.0+incompatible/go.mod h1:9I4somxYTbIHy5NJKHRl3wXiIaQGbYVAs8BPL6v8lEs=
github.com/google/martian/v3 v3.0.0/go.mod h1:y5Zk1BBys9G+gd6Jrk0W3cC1+ELVxBWuIGO+w/tUAp0=
github.com/google/martian/v3 v3.1.0/go.mod h1:y5Zk1BBys9G+gd6Jrk0W3cC1+ELVxBWuIGO+w/tUAp0=
Expand Down Expand Up @@ -240,6 +242,7 @@ github.com/herumi/bls-go-binary v1.31.0 h1:L1goQ2tMtGgpXCg5AwHAdJQpLs/pfnWWEc3Wo
github.com/herumi/bls-go-binary v1.31.0/go.mod h1:O4Vp1AfR4raRGwFeQpr9X/PQtncEicMoOe6BQt1oX0Y=
github.com/hitenjain14/fasthttp v0.0.0-20240229173600-722723e15e17 h1:FbyIK0BfvXVZTOxKOe2dlxJqSPSF2ZXOv2Mc7dvS7sc=
github.com/hitenjain14/fasthttp v0.0.0-20240229173600-722723e15e17/go.mod h1:RZMcXy7u4S+E97IXYTe7WHZ3+mCYOh4vys8PkIGZeXk=
github.com/hitenjain14/fasthttp v0.0.0-20240527123209-06019e79bff9/go.mod h1:RZMcXy7u4S+E97IXYTe7WHZ3+mCYOh4vys8PkIGZeXk=
github.com/holiman/bloomfilter/v2 v2.0.3 h1:73e0e/V0tCydx14a0SCYS/EWCxgwLZ18CZcZKVu0fao=
github.com/holiman/bloomfilter/v2 v2.0.3/go.mod h1:zpoh+gs7qcpqrHr3dB55AMiJwo0iURXE7ZOP9L9hSkA=
github.com/holiman/uint256 v1.2.2-0.20230321075855-87b91420868c h1:DZfsyhDK1hnSS5lH8l+JggqzEleHteTYfutAiVlSUM8=
Expand Down Expand Up @@ -447,8 +450,8 @@ golang.org/x/crypto v0.0.0-20200622213623-75b288015ac9/go.mod h1:LzIPMQfyMNhhGPh
golang.org/x/crypto v0.0.0-20210421170649-83a5a9bb288b/go.mod h1:T9bdIzuCu7OtxOm1hfPfRQxPLYneinmdGuTeoZ9dtd4=
golang.org/x/crypto v0.0.0-20210921155107-089bfa567519/go.mod h1:GvvjBRRGRdwPK5ydBHafDWAxML/pGHZbMvKqRZ5+Abc=
golang.org/x/crypto v0.0.0-20211108221036-ceb1ce70b4fa/go.mod h1:GvvjBRRGRdwPK5ydBHafDWAxML/pGHZbMvKqRZ5+Abc=
golang.org/x/crypto v0.17.0 h1:r8bRNjWL3GshPW3gkd+RpvzWrZAwPS49OmTGZ/uhM4k=
golang.org/x/crypto v0.17.0/go.mod h1:gCAAfMLgwOJRpTjQ2zCCt2OcSfYMTeZVSRtQlPC7Nq4=
golang.org/x/crypto v0.24.0 h1:mnl8DM0o513X8fdIkmyFE/5hTYxbwYOjDS/+rK6qpRI=
golang.org/x/crypto v0.24.0/go.mod h1:Z1PMYSOR5nyMcyAVAIQSKCDwalqy85Aqn1x3Ws4L5DM=
golang.org/x/exp v0.0.0-20190121172915-509febef88a4/go.mod h1:CJ0aWSM057203Lf6IL+f9T1iT9GByDxfZKAQTCR3kQA=
golang.org/x/exp v0.0.0-20190306152737-a1d7652674e8/go.mod h1:CJ0aWSM057203Lf6IL+f9T1iT9GByDxfZKAQTCR3kQA=
golang.org/x/exp v0.0.0-20190510132918-efd6b22b2522/go.mod h1:ZjyILWgesfNpC6sMxTJOJm9Kp84zZh5NQWvqDGG3Qr8=
Expand Down Expand Up @@ -524,8 +527,8 @@ golang.org/x/net v0.0.0-20220722155237-a158d28d115b/go.mod h1:XRhObCWvk6IyKnWLug
golang.org/x/net v0.1.0/go.mod h1:Cx3nUiGt4eDBEyega/BKRp+/AlGL8hYe7U9odMt2Cco=
golang.org/x/net v0.5.0/go.mod h1:DivGGAXEgPSlEBzxGzZI+ZLohi+xUj054jfeKui00ws=
golang.org/x/net v0.7.0/go.mod h1:2Tu9+aMcznHK/AK1HMvgo6xiTLG5rD5rZLDS+rp2Bjs=
golang.org/x/net v0.19.0 h1:zTwKpTd2XuCqf8huc7Fo2iSy+4RHPd10s4KzeTnVr1c=
golang.org/x/net v0.19.0/go.mod h1:CfAk/cbD4CthTvqiEl8NpboMuiuOYsAr/7NOjZJtv1U=
golang.org/x/net v0.26.0 h1:soB7SVo0PWrY4vPW/+ay0jKDNScG2X9wFeYlXIvJsOQ=
golang.org/x/net v0.26.0/go.mod h1:5YKkiSynbBIh3p6iOc/vibscux0x38BZDkn8sCUPxHE=
golang.org/x/oauth2 v0.0.0-20180821212333-d2e6202438be/go.mod h1:N/0e6XlmueqKjAGxoOufVs8QHGRruUQn6yWY3a++T0U=
golang.org/x/oauth2 v0.0.0-20190226205417-e64efc72b421/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw=
golang.org/x/oauth2 v0.0.0-20190604053449-0f29369cfe45/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw=
Expand All @@ -546,8 +549,8 @@ golang.org/x/sync v0.0.0-20200625203802-6e8e738ad208/go.mod h1:RxMgew5VJxzue5/jJ
golang.org/x/sync v0.0.0-20201020160332-67f06af15bc9/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20201207232520-09787c993a3a/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20220722155255-886fb9371eb4/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.5.0 h1:60k92dhOjHxJkrqnwsfl8KuaHbn/5dl0lUPUklKo3qE=
golang.org/x/sync v0.5.0/go.mod h1:Czt+wKu1gCyEFDUtn0jG5QVvpJ6rzVqr5aXyt9drQfk=
golang.org/x/sync v0.7.0 h1:YsImfSBoP9QPYL0xyKJPq0gcaJdG3rInoqxTWbfQu9M=
golang.org/x/sync v0.7.0/go.mod h1:Czt+wKu1gCyEFDUtn0jG5QVvpJ6rzVqr5aXyt9drQfk=
golang.org/x/sys v0.0.0-20180830151530-49385e6e1522/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20180909124046-d0be0721c37e/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20190124100055-b90733256f2e/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
Expand Down Expand Up @@ -600,8 +603,8 @@ golang.org/x/sys v0.1.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.2.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.4.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.5.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.15.0 h1:h48lPFYpsTvQJZF4EKyI4aLHaev3CxivZmv7yZig9pc=
golang.org/x/sys v0.15.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA=
golang.org/x/sys v0.21.0 h1:rF+pYz3DAGSQAxAu1CbC7catZg4ebC4UIeIhKxBZvws=
golang.org/x/sys v0.21.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA=
golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo=
golang.org/x/term v0.0.0-20210927222741-03fcf44c2211/go.mod h1:jbD1KX2456YbFQfuXm/mYQcufACuNUgVhRMnK/tPxf8=
golang.org/x/term v0.1.0/go.mod h1:jbD1KX2456YbFQfuXm/mYQcufACuNUgVhRMnK/tPxf8=
Expand All @@ -618,8 +621,8 @@ golang.org/x/text v0.3.7/go.mod h1:u+2+/6zg+i71rQMx5EYifcz6MCKuco9NR6JIITiCfzQ=
golang.org/x/text v0.4.0/go.mod h1:mrYo+phRRbMaCq/xk9113O4dZlRixOauAjOtrjsXDZ8=
golang.org/x/text v0.6.0/go.mod h1:mrYo+phRRbMaCq/xk9113O4dZlRixOauAjOtrjsXDZ8=
golang.org/x/text v0.7.0/go.mod h1:mrYo+phRRbMaCq/xk9113O4dZlRixOauAjOtrjsXDZ8=
golang.org/x/text v0.14.0 h1:ScX5w1eTa3QqT8oi6+ziP7dTV1S2+ALU0bI+0zXKWiQ=
golang.org/x/text v0.14.0/go.mod h1:18ZOQIKpY8NJVqYksKHtTdi31H5itFRjB5/qKTNYzSU=
golang.org/x/text v0.16.0 h1:a94ExnEXNtEwYLGJSIUxnWoxoRz/ZcCsV63ROupILh4=
golang.org/x/text v0.16.0/go.mod h1:GhwF1Be+LQoKShO3cGOHzqOgRrGaYc9AvblQOmPVHnI=
golang.org/x/time v0.0.0-20181108054448-85acf8d2951c/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=
golang.org/x/time v0.0.0-20190308202827-9d24e82272b4/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=
golang.org/x/time v0.0.0-20191024005414-555d28b269f0/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=
Expand Down
2 changes: 2 additions & 0 deletions internal/api/util/config/config.go
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,8 @@ type Config struct {
S3BucketNameAlternate string `yaml:"s3_bucket_name_alternate"`
BlobberOwnerWalletMnemonics string `yaml:"blobber_owner_wallet_mnemonics"`
OwnerWalletMnemonics string `yaml:"owner_wallet_mnemonics"`
DropboxAccessToken string `yaml:"dropboxAccessToken"`
GdriveAccessToken string `yaml:"gdriveAccessToken"`
}

func Parse(configPath string) *Config {
Expand Down
8 changes: 8 additions & 0 deletions internal/cli/model/model.go
Original file line number Diff line number Diff line change
Expand Up @@ -733,6 +733,14 @@ type ReadMarker struct {
BlockNumber int64 `json:"block_number"`
}

// holds result of repair size
type RepairSize struct {
// upload size in bytes
UploadSize uint64 `json:"upload_size"`
// download size in bytes
DownloadSize uint64 `json:"download_size"`
}

var StorageKeySettings = []string{
"owner_id",
}
Expand Down
100 changes: 100 additions & 0 deletions tests/api_tests/repair_allocation_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -347,3 +347,103 @@ func TestRepairAllocation(testSetup *testing.T) {
}
})
}

func TestRepairSize(testSetup *testing.T) {
t := test.NewSystemTest(testSetup)
wallet := createWallet(t)
sdkClient.SetWallet(t, wallet)
apiClient.CreateReadPool(t, wallet, 0.5, client.TxSuccessfulStatus)

t.RunSequentiallyWithTimeout("repair size in case of no blobber failure should be zero", 5 * time.Minute, func(t *test.SystemTest) {
// create allocation with default blobber requirements
blobberRequirements := model.DefaultBlobberRequirements(wallet.Id, wallet.PublicKey)
allocationBlobbers := apiClient.GetAllocationBlobbers(t, wallet, &blobberRequirements, client.HttpOkStatus)
allocationID := apiClient.CreateAllocation(t, wallet, allocationBlobbers, client.TxSuccessfulStatus)
t.Logf("allocationID: %v", allocationID)

// create and upload a file of 2KB to allocation.
op := sdkClient.AddUploadOperation(t, "", "", int64(1024 * 2))
sdkClient.MultiOperation(t, allocationID, []sdk.OperationRequest{op})

// assert both upload and download size should be zero
alloc, err := sdk.GetAllocation(allocationID)
require.NoErrorf(t, err, "allocation ID %v is not found", allocationID)
rs, err := alloc.RepairSize("/")
require.Nil(t, err)
t.Logf("repair size: %v", rs)
require.Equal(t, uint64(0), rs.UploadSize, "upload size doesn't match")
require.Equal(t, uint64(0), rs.DownloadSize, "download size doesn't match")
})

t.RunSequentiallyWithTimeout("repair size on single blobber failure should match", 5 * time.Minute, func(t *test.SystemTest) {
// create allocation with default blobber requirements
blobberRequirements := model.DefaultBlobberRequirements(wallet.Id, wallet.PublicKey)
blobberRequirements.DataShards = 2
blobberRequirements.ParityShards = 2
blobberRequirements.Size = 2056
allocationBlobbers := apiClient.GetAllocationBlobbers(t, wallet, &blobberRequirements, client.HttpOkStatus)
allocationID := apiClient.CreateAllocation(t, wallet, allocationBlobbers, client.TxSuccessfulStatus)
t.Logf("allocationID: %v", allocationID)

// create and upload a file of 2KB to allocation.
// one blobber url is set invalid to mimic failure.
alloc, err := sdk.GetAllocation(allocationID)
require.NoErrorf(t, err, "allocation ID %v is not found", allocationID)
alloc.Blobbers[0].Baseurl = "http://0zus.com/"
op := sdkClient.AddUploadOperation(t, "", "", int64(1024 * 2))
sdkClient.MultiOperation(t, allocationID, []sdk.OperationRequest{op}, client.WithRepair(alloc.Blobbers))

// assert upload and download size should be 1KB and 2KB respectively
rs, err := alloc.RepairSize("/")
require.Nil(t, err)
t.Logf("repair size: %v", rs)
require.Equal(t, uint64(1024), rs.UploadSize, "upload size doesn't match")
require.Equal(t, uint64(1024 * 2), rs.DownloadSize, "download size doesn't match")
})

t.RunSequentiallyWithTimeout("repair size with nested directories and two blobber failure should match", 5 * time.Minute, func(t *test.SystemTest) {
// create allocation with default blobber requirements
blobberRequirements := model.DefaultBlobberRequirements(wallet.Id, wallet.PublicKey)
blobberRequirements.DataShards = 2
blobberRequirements.ParityShards = 4
allocationBlobbers := apiClient.GetAllocationBlobbers(t, wallet, &blobberRequirements, client.HttpOkStatus)
allocationID := apiClient.CreateAllocation(t, wallet, allocationBlobbers, client.TxSuccessfulStatus)
t.Logf("allocationID: %v", allocationID)

// create and upload two files of 1KB each to / and /dir1.
// two blobber url is set invalid to mimic failure.
alloc, err := sdk.GetAllocation(allocationID)
require.NoErrorf(t, err, "allocation ID %v is not found", allocationID)
alloc.Blobbers[0].Baseurl = "http://0zus.com/"
alloc.Blobbers[1].Baseurl = "http://0zus.com/"
ops := []sdk.OperationRequest{
sdkClient.AddUploadOperationWithPath(t, allocationID, "/dir1/"),
sdkClient.AddUploadOperationWithPath(t, allocationID, "/dir1/"),
sdkClient.AddUploadOperationWithPath(t, allocationID, "/"),
sdkClient.AddUploadOperationWithPath(t, allocationID, "/"),
}
sdkClient.MultiOperation(t, allocationID, ops, client.WithRepair(alloc.Blobbers))

// assert both upload and download size should be 2KB in /dir1
rs, err := alloc.RepairSize("/dir1")
require.Nilf(t, err, "error getting repair size in /dir1: %v", err)
t.Logf("repair size: %v", rs)
require.Equal(t, uint64(1024 * 2), rs.UploadSize, "upload size in directory /dir1 doesn't match")
require.Equal(t, uint64(1024 * 2), rs.DownloadSize, "download size in directory dir1 doesn't match")

// with trailing slash
// assert both upload and download size should be 2KB in /dir1/
rs, err = alloc.RepairSize("/dir1/")
require.Nilf(t, err, "error getting repair size in /dir1/: %v", err)
t.Logf("repair size: %v", rs)
require.Equal(t, uint64(1024 * 2), rs.UploadSize, "upload size in directory /dir1/ doesn't match")
require.Equal(t, uint64(1024 * 2), rs.DownloadSize, "download size in directory /dir1/ doesn't match")

// assert both upload and download size should be 4KB in root directory
rs, err = alloc.RepairSize("/")
require.Nilf(t, err, "error getting repair size in /: %v", err)
t.Logf("repair size: %v", rs)
require.Equal(t, uint64(1024 * 4), rs.UploadSize, "upload size in root directory doesn't match")
require.Equal(t, uint64(1024 * 4), rs.DownloadSize, "download size in root directory doesn't match")
})
}
Loading
Loading