Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

increase timeout threshold #267

Merged
merged 2 commits into from
Apr 5, 2021
Merged

increase timeout threshold #267

merged 2 commits into from
Apr 5, 2021

Conversation

liamsi
Copy link
Member

@liamsi liamsi commented Apr 5, 2021

migling with these timeouts to see if that fixes the CI fails #241 (comment)

@liamsi
Copy link
Member Author

liamsi commented Apr 5, 2021

Still happening:

--- FAIL: TestStateLockPOLUnlockOnUnknownBlock (0.27s)
panic: Timeout expired while waiting for NewTimeout event [recovered]
	panic: Timeout expired while waiting for NewTimeout event

goroutine 77108 [running]:
testing.tRunner.func1.1(0x1ea1400, 0xc003d93d20)
	/opt/hostedtoolcache/go/1.15.10/x64/src/testing/testing.go:1072 +0x46a
testing.tRunner.func1(0xc047aa9500)
	/opt/hostedtoolcache/go/1.15.10/x64/src/testing/testing.go:1075 +0x636
panic(0x1ea1400, 0xc003d93d20)
	/opt/hostedtoolcache/go/1.15.10/x64/src/runtime/panic.go:975 +0x47a
github.com/lazyledger/lazyledger-core/consensus.ensureNewEvent(0xc01c0fa720, 0x1, 0x0, 0x5f5e100, 0x22456c7, 0x32)
	/home/runner/work/lazyledger-core/lazyledger-core/consensus/common_test.go:486 +0x43f
github.com/lazyledger/lazyledger-core/consensus.ensureNewTimeout(...)
	/home/runner/work/lazyledger-core/lazyledger-core/consensus/common_test.go:524
github.com/lazyledger/lazyledger-core/consensus.TestStateLockPOLUnlockOnUnknownBlock(0xc047aa9500)
	/home/runner/work/lazyledger-core/lazyledger-core/consensus/state_test.go:830 +0xb25
testing.tRunner(0xc047aa9500, 0x23917d0)
	/opt/hostedtoolcache/go/1.15.10/x64/src/testing/testing.go:1123 +0x203
created by testing.(*T).Run
	/opt/hostedtoolcache/go/1.15.10/x64/src/testing/testing.go:1168 +0x5bc
FAIL	github.com/lazyledger/lazyledger-core/consensus	81.675s

@liamsi
Copy link
Member Author

liamsi commented Apr 5, 2021

Now TestBlockPoolBasic fails. No race, and the others in question just pass?

@liamsi
Copy link
Member Author

liamsi commented Apr 5, 2021

Now there is this:

long output 
SIGQUIT: quit
PC=0x47b0d9 m=5 sigcode=0

goroutine 0 [idle]:
runtime.gentraceback(0xffffffffffffffff, 0xffffffffffffffff, 0x0, 0xc002353800, 0x0, 0x0, 0x7fffffff, 0x7f91899eace0, 0x0, 0x0, ...)
	/opt/hostedtoolcache/go/1.15.10/x64/src/runtime/traceback.go:189 +0x2b9
runtime.scanstack(0xc002353800, 0xc000053e98)
	/opt/hostedtoolcache/go/1.15.10/x64/src/runtime/mgcmark.go:844 +0x17c
runtime.markroot.func1()
	/opt/hostedtoolcache/go/1.15.10/x64/src/runtime/mgcmark.go:245 +0xc6
runtime.markroot(0xc000053e98, 0xbfe)
	/opt/hostedtoolcache/go/1.15.10/x64/src/runtime/mgcmark.go:218 +0x310
runtime.gcDrain(0xc000053e98, 0xb)
	/opt/hostedtoolcache/go/1.15.10/x64/src/runtime/mgcmark.go:1109 +0x118
runtime.gcBgMarkWorker.func2()
	/opt/hostedtoolcache/go/1.15.10/x64/src/runtime/mgc.go:1979 +0x191
runtime.systemstack(0x7f9180000020)
	/opt/hostedtoolcache/go/1.15.10/x64/src/runtime/asm_amd64.s:370 +0x66
runtime.mstart()
	/opt/hostedtoolcache/go/1.15.10/x64/src/runtime/proc.go:1116

goroutine 20 [GC worker (idle)]:
runtime.systemstack_switch()
	/opt/hostedtoolcache/go/1.15.10/x64/src/runtime/asm_amd64.s:330 fp=0xc00005af60 sp=0xc00005af58 pc=0x48c800
runtime.gcBgMarkWorker(0xc000052800)
	/opt/hostedtoolcache/go/1.15.10/x64/src/runtime/mgc.go:1945 +0x1be fp=0xc00005afd8 sp=0xc00005af60 pc=0x4392de
runtime.goexit()
	/opt/hostedtoolcache/go/1.15.10/x64/src/runtime/asm_amd64.s:1374 +0x1 fp=0xc00005afe0 sp=0xc00005afd8 pc=0x48e5e1
created by runtime.gcBgMarkStartWorkers
	/opt/hostedtoolcache/go/1.15.10/x64/src/runtime/mgc.go:1839 +0x77

goroutine 1 [chan receive, 7 minutes]:
testing.(*T).Run(0xc000104f00, 0x220d1c7, 0x13, 0x2391790, 0x1)
	/opt/hostedtoolcache/go/1.15.10/x64/src/testing/testing.go:1169 +0x5f4
testing.runTests.func1(0xc000104f00)
	/opt/hostedtoolcache/go/1.15.10/x64/src/testing/testing.go:1439 +0xa7
testing.tRunner(0xc000104f00, 0xc0010ffba0)
	/opt/hostedtoolcache/go/1.15.10/x64/src/testing/testing.go:1123 +0x203
testing.runTests(0xc0002632a0, 0x30dbc80, 0x47, 0x47, 0xc012ee25a9989878, 0x6ff5b3a530, 0x32364e0, 0x0)
	/opt/hostedtoolcache/go/1.15.10/x64/src/testing/testing.go:1437 +0x613
testing.(*M).Run(0xc0000f0780, 0x0)
	/opt/hostedtoolcache/go/1.15.10/x64/src/testing/testing.go:1345 +0x3b4
github.com/lazyledger/lazyledger-core/consensus.TestMain(0xc0000f0780)
	/home/runner/work/lazyledger-core/lazyledger-core/consensus/replay_test.go:44 +0x177
main.main()
	_testmain.go:265 +0x372

goroutine 1574 [chan receive, 8 minutes]:
github.com/lazyledger/lazyledger-core/libs/pubsub.(*Server).loop(0xc0002367e0, 0xc001e05bc0, 0xc001e05bf0)
	/home/runner/work/lazyledger-core/lazyledger-core/libs/pubsub/pubsub.go:324 +0xb9
created by github.com/lazyledger/lazyledger-core/libs/pubsub.(*Server).OnStart
	/home/runner/work/lazyledger-core/lazyledger-core/libs/pubsub/pubsub.go:310 +0x73

goroutine 7 [select, 9 minutes]:
github.com/ipfs/go-log/writer.(*MirrorWriter).logRoutine(0xc00002f8c0)
	/home/runner/go/pkg/mod/github.com/ipfs/[email protected]/writer/writer.go:71 +0x1b6
created by github.com/ipfs/go-log/writer.NewMirrorWriter
	/home/runner/go/pkg/mod/github.com/ipfs/[email protected]/writer/writer.go:36 +0x13a

goroutine 31909 [runnable]:
github.com/dgraph-io/ristretto/z.(*AllocatorPool).freeupAllocators(0xc0234d1360)
	/home/runner/go/pkg/mod/github.com/dgraph-io/[email protected]/z/allocator.go:383 +0x265
created by github.com/dgraph-io/ristretto/z.NewAllocatorPool
	/home/runner/go/pkg/mod/github.com/dgraph-io/[email protected]/z/allocator.go:323 +0x139

goroutine 9 [runnable]:
go.opencensus.io/stats/view.(*worker).reportUsage(0xc0000f0200)
	/home/runner/go/pkg/mod/[email protected]/stats/view/worker.go:364 +0xfe
go.opencensus.io/stats/view.(*worker).start(0xc0000f0200)
	/home/runner/go/pkg/mod/[email protected]/stats/view/worker.go:280 +0x20b
created by go.opencensus.io/stats/view.init.0
	/home/runner/go/pkg/mod/[email protected]/stats/view/worker.go:34 +0xb2

goroutine 1572 [chan receive, 9 minutes]:
github.com/lazyledger/lazyledger-core/libs/pubsub.(*Server).loop(0xc0002362a0, 0xc001e04000, 0xc001e04030)
	/home/runner/work/lazyledger-core/lazyledger-core/libs/pubsub/pubsub.go:324 +0xb9
created by github.com/lazyledger/lazyledger-core/libs/pubsub.(*Server).OnStart
	/home/runner/work/lazyledger-core/lazyledger-core/libs/pubsub/pubsub.go:310 +0x73

goroutine 18386 [chan receive, 8 minutes]:
github.com/lazyledger/lazyledger-core/libs/pubsub.(*Server).loop(0xc00014ec40, 0xc00364b560, 0xc00364b590)
	/home/runner/work/lazyledger-core/lazyledger-core/libs/pubsub/pubsub.go:324 +0xb9
created by github.com/lazyledger/lazyledger-core/libs/pubsub.(*Server).OnStart
	/home/runner/work/lazyledger-core/lazyledger-core/libs/pubsub/pubsub.go:310 +0x73

goroutine 1571 [chan receive, 9 minutes]:
github.com/lazyledger/lazyledger-core/libs/pubsub.(*Server).loop(0xc000093c70, 0xc0025a5200, 0xc0025a5230)
	/home/runner/work/lazyledger-core/lazyledger-core/libs/pubsub/pubsub.go:324 +0xb9
created by github.com/lazyledger/lazyledger-core/libs/pubsub.(*Server).OnStart
	/home/runner/work/lazyledger-core/lazyledger-core/libs/pubsub/pubsub.go:310 +0x73

goroutine 1573 [chan receive, 9 minutes]:
github.com/lazyledger/lazyledger-core/libs/pubsub.(*Server).loop(0xc0002364d0, 0xc001e04de0, 0xc001e04e10)
	/home/runner/work/lazyledger-core/lazyledger-core/libs/pubsub/pubsub.go:324 +0xb9
created by github.com/lazyledger/lazyledger-core/libs/pubsub.(*Server).OnStart
	/home/runner/work/lazyledger-core/lazyledger-core/libs/pubsub/pubsub.go:310 +0x73

goroutine 34 [IO wait, 9 minutes]:
internal/poll.runtime_pollWait(0x7f91b1f48e88, 0x72, 0x72)
	/opt/hostedtoolcache/go/1.15.10/x64/src/runtime/netpoll.go:222 +0x55
internal/poll.(*pollDesc).wait(0xc0000f0a18, 0x72, 0x0, 0x0, 0x2135b1d)
	/opt/hostedtoolcache/go/1.15.10/x64/src/internal/poll/fd_poll_runtime.go:87 +0xe6
internal/poll.(*pollDesc).waitRead(...)
	/opt/hostedtoolcache/go/1.15.10/x64/src/internal/poll/fd_poll_runtime.go:92
internal/poll.(*FD).Accept(0xc0000f0a00, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0)
	/opt/hostedtoolcache/go/1.15.10/x64/src/internal/poll/fd_unix.go:394 +0x2fe
net.(*netFD).accept(0xc0000f0a00, 0x0, 0x0, 0x0)
	/opt/hostedtoolcache/go/1.15.10/x64/src/net/fd_unix.go:172 +0x58
net.(*TCPListener).accept(0xc00000ea40, 0x0, 0x0, 0x0)
	/opt/hostedtoolcache/go/1.15.10/x64/src/net/tcpsock_posix.go:139 +0x50
net.(*TCPListener).Accept(0xc00000ea40, 0x0, 0x0, 0x0, 0x0)
	/opt/hostedtoolcache/go/1.15.10/x64/src/net/tcpsock.go:261 +0x8e
github.com/lazyledger/lazyledger-core/p2p.(*MultiplexTransport).acceptPeers(0xc0003cd7a0)
	/home/runner/work/lazyledger-core/lazyledger-core/p2p/transport.go:266 +0x96
created by github.com/lazyledger/lazyledger-core/p2p.(*MultiplexTransport).Listen
	/home/runner/work/lazyledger-core/lazyledger-core/p2p/transport.go:259 +0x191

goroutine 39 [IO wait, 9 minutes]:
internal/poll.runtime_pollWait(0x7f91b1f48da0, 0x72, 0x72)
	/opt/hostedtoolcache/go/1.15.10/x64/src/runtime/netpoll.go:222 +0x55
internal/poll.(*pollDesc).wait(0xc0000f0d98, 0x72, 0x0, 0x0, 0x2135b1d)
	/opt/hostedtoolcache/go/1.15.10/x64/src/internal/poll/fd_poll_runtime.go:87 +0xe6
internal/poll.(*pollDesc).waitRead(...)
	/opt/hostedtoolcache/go/1.15.10/x64/src/internal/poll/fd_poll_runtime.go:92
internal/poll.(*FD).Accept(0xc0000f0d80, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0)
	/opt/hostedtoolcache/go/1.15.10/x64/src/internal/poll/fd_unix.go:394 +0x2fe
net.(*netFD).accept(0xc0000f0d80, 0x0, 0x8, 0xc00005b608)
	/opt/hostedtoolcache/go/1.15.10/x64/src/net/fd_unix.go:172 +0x58
net.(*TCPListener).accept(0xc00000f0e0, 0x211765e, 0xc001219e40, 0x213c224)
	/opt/hostedtoolcache/go/1.15.10/x64/src/net/tcpsock_posix.go:139 +0x50
net.(*TCPListener).Accept(0xc00000f0e0, 0x0, 0x0, 0x0, 0x0)
	/opt/hostedtoolcache/go/1.15.10/x64/src/net/tcpsock.go:261 +0x8e
*** Test killed: ran too long (9m0s).
FAIL	github.com/lazyledger/lazyledger-core/consensus	545.984s

@liamsi liamsi requested a review from tac0turtle April 5, 2021 17:15
@liamsi liamsi marked this pull request as ready for review April 5, 2021 17:15
@liamsi liamsi requested a review from evan-forbes as a code owner April 5, 2021 17:15
@liamsi
Copy link
Member Author

liamsi commented Apr 5, 2021

@marbar3778 feel free to merge that into your branch. It seems to fix some of the previously failing tests in the consensus package. But as I'm not really sure what is / was going (tests now seem to consistently pass on CI too), feel free to close and ignore it as well.

@tac0turtle tac0turtle merged commit d0f7fdb into marko/db_changes Apr 5, 2021
@tac0turtle tac0turtle deleted the ismail/marko/db_changes_increase_timeouts branch April 5, 2021 17:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@tac0turtle tac0turtle Awaiting requested review from tac0turtle

@evan-forbes evan-forbes Awaiting requested review from evan-forbes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@liamsi @tac0turtle