Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

UT "TestSnapSyncWithBlobs" random fail #2489

Closed
zzzckck opened this issue May 22, 2024 · 2 comments
Closed

UT "TestSnapSyncWithBlobs" random fail #2489

zzzckck opened this issue May 22, 2024 · 2 comments
Assignees
@MatusKysel @zzzckck

Comments

@zzzckck
Copy link
Collaborator

zzzckck commented May 22, 2024 •
edited
Loading

System information

Geth version: v1.4.8
OS & Version: Windows/Linux/OSX
Commit hash : (if develop)

Expected behaviour

UT should not failed

Actual behaviour

it failed

Steps to reproduce the behaviour

Random fail, check the failed log: https://github.com/bnb-chain/bsc/actions/runs/9184422166/job/25256710320?pr=2487

Backtrace


panic: test timed out after 1h0m0s
running tests:
	TestSnapSyncWithBlobs (1h0m0s)

goroutine 70838 [running]:
testing.(*M).startAlarm.func1()
	/opt/hostedtoolcache/go/1.21.11/x64/src/testing/testing.go:2259 +0x3b9
created by time.goFunc
	/opt/hostedtoolcache/go/1.21.11/x64/src/time/sleep.go:176 +0x2d

goroutine 1 [chan receive, 59 minutes]:
testing.tRunner.func1()
	/opt/hostedtoolcache/go/1.21.11/x64/src/testing/testing.go:1561 +0x489
testing.tRunner(0xc00158c1a0, 0xc001f97c48)
	/opt/hostedtoolcache/go/1.21.11/x64/src/testing/testing.go:1601 +0x138
testing.runTests(0xc00014c1e0?, {0x2d72bc0, 0x1b, 0x1b}, {0xc?, 0xc001f97d08?, 0x2e131c0?})
	/opt/hostedtoolcache/go/1.21.11/x64/src/testing/testing.go:2052 +0x445
testing.(*M).Run(0xc00014c1e0)
	/opt/hostedtoolcache/go/1.21.11/x64/src/testing/testing.go:1925 +0x636
main.main()
	_testmain.go:99 +0x19c

goroutine 6 [chan receive]:
github.com/panjf2000/ants/v2.(*Pool).purgePeriodically(0xc0000d21c0)
	/home/runner/go/pkg/mod/github.com/panjf2000/ants/[email protected]/pool.go:69 +0x85
created by github.com/panjf2000/ants/v2.NewPool in goroutine 1
	/home/runner/go/pkg/mod/github.com/panjf2000/ants/[email protected]/pool.go:137 +0x325

goroutine 7 [chan receive]:
github.com/panjf2000/ants/v2.(*Pool).purgePeriodically(0xc0000d2230)
	/home/runner/go/pkg/mod/github.com/panjf2000/ants/[email protected]/pool.go:69 +0x85
created by github.com/panjf2000/ants/v2.NewPool in goroutine 1
	/home/runner/go/pkg/mod/github.com/panjf2000/ants/[email protected]/pool.go:137 +0x325

goroutine 10 [select, 60 minutes]:
github.com/dgraph-io/ristretto.(*defaultPolicy).processItems(0xc0001ce080)
	/home/runner/go/pkg/mod/github.com/dgraph-io/[email protected]/policy.go:102 +0x85
created by github.com/dgraph-io/ristretto.newDefaultPolicy in goroutine 1
	/home/runner/go/pkg/mod/github.com/dgraph-io/[email protected]/policy.go:86 +0x139

goroutine 11 [select]:
github.com/dgraph-io/ristretto.(*Cache).processItems(0xc0002ae000)
	/home/runner/go/pkg/mod/github.com/dgraph-io/[email protected]/cache.go:422 +0x149
created by github.com/dgraph-io/ristretto.NewCache in goroutine 1
	/home/runner/go/pkg/mod/github.com/dgraph-io/[email protected]/cache.go:204 +0x645

goroutine 29177 [select, 59 minutes]:
github.com/ethereum/go-ethereum/eth.(*handler).protoTracker(0xc001a86160)
	/home/runner/work/bsc/bsc/eth/handler.go:380 +0xe6
created by github.com/ethereum/go-ethereum/eth.(*handler).Start in goroutine 4351
	/home/runner/work/bsc/bsc/eth/handler.go:741 +0x465

goroutine 29202 [select, 59 minutes]:
github.com/ethereum/go-ethereum/eth/fetcher.(*BlockFetcher).loop(0xc005dff4a0)
	/home/runner/work/bsc/bsc/eth/fetcher/block_fetcher.go:399 +0x365
created by github.com/ethereum/go-ethereum/eth/fetcher.(*BlockFetcher).Start in goroutine 29176
	/home/runner/work/bsc/bsc/eth/fetcher/block_fetcher.go:250 +0x4f

goroutine 29185 [select, 59 minutes]:
github.com/ethereum/go-ethereum/eth/protocols/eth.(*Peer).dispatcher(0xc005dff680)
	/home/runner/work/bsc/bsc/eth/protocols/eth/dispatcher.go:192 +0x152
created by github.com/ethereum/go-ethereum/eth/protocols/eth.NewPeer in goroutine 4351
	/home/runner/work/bsc/bsc/eth/protocols/eth/peer.go:123 +0x5c5

goroutine 29065 [select, 59 minutes]:
github.com/ethereum/go-ethereum/eth.(*handler).protoTracker(0xc007b5adc0)
	/home/runner/work/bsc/bsc/eth/handler.go:380 +0xe6
created by github.com/ethereum/go-ethereum/eth.(*handler).Start in goroutine 4351
	/home/runner/work/bsc/bsc/eth/handler.go:741 +0x465

goroutine 29087 [select, 59 minutes]:
github.com/ethereum/go-ethereum/eth/fetcher.(*BlockFetcher).loop(0xc005dff0e0)
	/home/runner/work/bsc/bsc/eth/fetcher/block_fetcher.go:399 +0x365
created by github.com/ethereum/go-ethereum/eth/fetcher.(*BlockFetcher).Start in goroutine 29064
	/home/runner/work/bsc/bsc/eth/fetcher/block_fetcher.go:250 +0x4f

goroutine 29175 [select, 59 minutes]:
github.com/ethereum/go-ethereum/eth.(*handler).minedBroadcastLoop(0xc001a86160)
	/home/runner/work/bsc/bsc/eth/handler.go:944 +0xc6
created by github.com/ethereum/go-ethereum/eth.(*handler).Start in goroutine 4351
	/home/runner/work/bsc/bsc/eth/handler.go:733 +0x3bb

goroutine 4351 [chan receive, 59 minutes]:
github.com/ethereum/go-ethereum/eth/downloader.(*Downloader).spawnSync(0xc001766d00, {0xc017585080, 0x5, 0xa?})
	/home/runner/work/bsc/bsc/eth/downloader/downloader.go:623 +0x154
github.com/ethereum/go-ethereum/eth/downloader.(*Downloader).syncWithPeer(0xc001766d00, 0xc005e0b800, {0x94, 0xc5, 0x90, 0x52, 0xbb, 0x6e, 0x7d, 0xfe, ...}, ...)
	/home/runner/work/bsc/bsc/eth/downloader/downloader.go:602 +0x1074
github.com/ethereum/go-ethereum/eth/downloader.(*Downloader).synchronise(0x20?, {0xc0005fdc00, 0x40}, {0x94, 0xc5, 0x90, 0x52, 0xbb, 0x6e, 0x7d, ...}, ...)
	/home/runner/work/bsc/bsc/eth/downloader/downloader.go:442 +0x5b2
github.com/ethereum/go-ethereum/eth/downloader.(*Downloader).LegacySync(0xc001766d00, {0xc0005fdc00, 0x40}, {0x94, 0xc5, 0x90, 0x52, 0xbb, 0x6e, 0x7d, ...}, ...)
	/home/runner/work/bsc/bsc/eth/downloader/downloader.go:331 +0x53
github.com/ethereum/go-ethereum/eth.(*handler).doSync(0xc001a86160, 0x22d2a60a02c56937?)
	/home/runner/work/bsc/bsc/eth/sync.go:236 +0x79
github.com/ethereum/go-ethereum/eth.testChainSyncWithBlobs(0xc0015b6ea0, 0x1, 0x80, 0x49950f?)
	/home/runner/work/bsc/bsc/eth/sync_test.go:173 +0xc6f
github.com/ethereum/go-ethereum/eth.TestSnapSyncWithBlobs(0xc001e67fd0?)
	/home/runner/work/bsc/bsc/eth/sync_test.go:106 +0x25
testing.tRunner(0xc0015b6ea0, 0x1f3c748)
	/opt/hostedtoolcache/go/1.21.11/x64/src/testing/testing.go:1595 +0xff
created by testing.(*T).Run in goroutine 1
	/opt/hostedtoolcache/go/1.21.11/x64/src/testing/testing.go:1648 +0x3ad
FAIL	github.com/ethereum/go-ethereum/eth	3600.093s
ok  	github.com/ethereum/go-ethereum/eth/catalyst	1.049s

When submitting logs: please submit them as text and not screenshots.
@zzzckck zzzckck self-assigned this May 22, 2024
@zzzckck zzzckck mentioned this issue Jun 19, 2024
Merged
@zzzckck zzzckck closed this as completed Jun 20, 2024
@zzzckck
Copy link
Collaborator Author

zzzckck commented Jun 28, 2024

Seems it is not fixed yet:
https://github.com/bnb-chain/bsc/actions/runs/9690243822/job/26739746338?pr=2546


goroutine 4349 [chan receive, 59 minutes]:
github.com/ethereum/go-ethereum/eth/downloader.(*Downloader).spawnSync(0xc0017b3040, {0xc00195dbc0, 0x5, 0xa?})
	/home/runner/work/bsc/bsc/eth/downloader/downloader.go:623 +0x154
github.com/ethereum/go-ethereum/eth/downloader.(*Downloader).syncWithPeer(0xc0017b3040, 0xc00a233140, {0x94, 0xc5, 0x90, 0x52, 0xbb, 0x6e, 0x7d, 0xfe, ...}, ...)
	/home/runner/work/bsc/bsc/eth/downloader/downloader.go:602 +0x1074
github.com/ethereum/go-ethereum/eth/downloader.(*Downloader).synchronise(0x20?, {0xc000c2ac00, 0x40}, {0x94, 0xc5, 0x90, 0x52, 0xbb, 0x6e, 0x7d, ...}, ...)
	/home/runner/work/bsc/bsc/eth/downloader/downloader.go:442 +0x5b2
github.com/ethereum/go-ethereum/eth/downloader.(*Downloader).LegacySync(0xc0017b3040, {0xc000c2ac00, 0x40}, {0x94, 0xc5, 0x90, 0x52, 0xbb, 0x6e, 0x7d, ...}, ...)
	/home/runner/work/bsc/bsc/eth/downloader/downloader.go:331 +0x53
github.com/ethereum/go-ethereum/eth.(*handler).doSync(0xc0019151e0, 0x22d2a60a02c56937?)
	/home/runner/work/bsc/bsc/eth/sync.go:236 +0x79
github.com/ethereum/go-ethereum/eth.testChainSyncWithBlobs(0xc00160f1e0, 0x1, 0x80, 0x49950f?)
	/home/runner/work/bsc/bsc/eth/sync_test.go:175 +0xc8a
github.com/ethereum/go-ethereum/eth.TestSnapSyncWithBlobs(0xc001d997d0?)
	/home/runner/work/bsc/bsc/eth/sync_test.go:106 +0x25
testing.tRunner(0xc00160f1e0, 0x1f725a8)
	/opt/hostedtoolcache/go/1.21.11/x64/src/testing/testing.go:1595 +0xff
created by testing.(*T).Run in goroutine 1
	/opt/hostedtoolcache/go/1.21.11/x64/src/testing/testing.go:1648 +0x3ad

@zzzckck zzzckck reopened this Jun 28, 2024
@MatusKysel MatusKysel self-assigned this Jun 28, 2024
@MatusKysel
Copy link
Contributor

This should stabilise it #2561 for me the issue was that not all time handler setup was done under 250ms

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@MatusKysel MatusKysel

@zzzckck zzzckck

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@MatusKysel @zzzckck