-
Notifications
You must be signed in to change notification settings - Fork 676
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
cosmovisor.service: Main process exited, code=exited, status=1/FAILURE #2928
Comments
Hi, this issue doesn't look like related to cosmovisor, as your node crashes, not cosmovisor. |
|
Can you post the full logs? |
Feb 05 13:03:35 interview-test3.us-central1-a.c.validator-379314.internal systemd[1]: Started cosmovisor.service - Cosmovisor service.
Feb 05 13:03:35 interview-test3.us-central1-a.c.validator-379314.internal cosmovisor[840810]: 1:03PM INF running app args=["start","--x-crisis-skip-assert-invariants","--home","/home/ubuntu/.gaia"] module=cosmovisor path=/home/ubuntu/.gaia/cosmovisor/genesis/bin/gaiad
Feb 05 13:03:35 interview-test3.us-central1-a.c.validator-379314.internal cosmovisor[840818]: 1:03PM INF starting node with ABCI Tendermint in-process
Feb 05 13:03:42 interview-test3.us-central1-a.c.validator-379314.internal cosmovisor[840818]: 1:03PM INF service start impl=multiAppConn module=proxy msg={}
Feb 05 13:03:42 interview-test3.us-central1-a.c.validator-379314.internal cosmovisor[840818]: 1:03PM INF service start connection=query impl=localClient module=abci-client msg={}
Feb 05 13:03:42 interview-test3.us-central1-a.c.validator-379314.internal cosmovisor[840818]: 1:03PM INF service start connection=snapshot impl=localClient module=abci-client msg={}
Feb 05 13:03:42 interview-test3.us-central1-a.c.validator-379314.internal cosmovisor[840818]: 1:03PM INF service start connection=mempool impl=localClient module=abci-client msg={}
Feb 05 13:03:42 interview-test3.us-central1-a.c.validator-379314.internal cosmovisor[840818]: 1:03PM INF service start connection=consensus impl=localClient module=abci-client msg={}
Feb 05 13:03:42 interview-test3.us-central1-a.c.validator-379314.internal cosmovisor[840818]: 1:03PM INF service start impl=EventBus module=events msg={}
Feb 05 13:03:42 interview-test3.us-central1-a.c.validator-379314.internal cosmovisor[840818]: 1:03PM INF service start impl=PubSub module=pubsub msg={}
Feb 05 13:03:42 interview-test3.us-central1-a.c.validator-379314.internal cosmovisor[840818]: 1:03PM INF service start impl=IndexerService module=txindex msg={}
Feb 05 13:03:42 interview-test3.us-central1-a.c.validator-379314.internal cosmovisor[840818]: 1:03PM INF Found local state with non-zero height, skipping state sync
Feb 05 13:03:42 interview-test3.us-central1-a.c.validator-379314.internal cosmovisor[840818]: 1:03PM INF ABCI Handshake App Info hash="���?\n�3E5����ڦ��n�\x02�(�8g�M���v�" height=20142000 module=consensus protocol-version=0 software-version=v13.0.0
Feb 05 13:03:42 interview-test3.us-central1-a.c.validator-379314.internal cosmovisor[840818]: 1:03PM INF ABCI Replay Blocks appHeight=20142000 module=consensus stateHeight=20142000 storeHeight=20142001
Feb 05 13:03:42 interview-test3.us-central1-a.c.validator-379314.internal cosmovisor[840818]: 1:03PM INF Replay last block using real app module=consensus
Feb 05 13:03:42 interview-test3.us-central1-a.c.validator-379314.internal cosmovisor[840818]: panic: runtime error: invalid memory address or nil pointer dereference
Feb 05 13:03:42 interview-test3.us-central1-a.c.validator-379314.internal cosmovisor[840818]: [signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x1838ed4]
Feb 05 13:03:42 interview-test3.us-central1-a.c.validator-379314.internal cosmovisor[840818]: goroutine 1 [running]:
Feb 05 13:03:42 interview-test3.us-central1-a.c.validator-379314.internal cosmovisor[840818]: github.com/cosmos/cosmos-sdk/x/upgrade.BeginBlocker({{0x7ffe8c908e2a, 0x12}, 0xc000d06570, {0x25cd140, 0xc000d2fd80}, {0x25ef510, 0xc000eed190}, 0xc000d07da0, {0x25c70a0, 0xc00007ee00}, ...}, ...)
Feb 05 13:03:42 interview-test3.us-central1-a.c.validator-379314.internal cosmovisor[840818]: github.com/cosmos/[email protected]/x/upgrade/abci.go:38 +0x1014
Feb 05 13:03:42 interview-test3.us-central1-a.c.validator-379314.internal cosmovisor[840818]: github.com/cosmos/cosmos-sdk/x/upgrade.AppModule.BeginBlock(...)
Feb 05 13:03:42 interview-test3.us-central1-a.c.validator-379314.internal cosmovisor[840818]: github.com/cosmos/[email protected]/x/upgrade/module.go:130
Feb 05 13:03:42 interview-test3.us-central1-a.c.validator-379314.internal cosmovisor[840818]: github.com/cosmos/cosmos-sdk/types/module.(*Manager).BeginBlock(_, {{0x25e1698, 0xc0001b0000}, {0x25ef0f0, 0xc000bf2340}, {{0xb, 0x0}, {0xc000d81b18, 0x11}, 0x13357b1, ...}, ...}, ...)
Feb 05 13:03:42 interview-test3.us-central1-a.c.validator-379314.internal cosmovisor[840818]: github.com/cosmos/[email protected]/types/module/module.go:491 +0x1db
Feb 05 13:03:42 interview-test3.us-central1-a.c.validator-379314.internal cosmovisor[840818]: github.com/cosmos/gaia/v13/app.(*GaiaApp).BeginBlocker(...)
Feb 05 13:03:42 interview-test3.us-central1-a.c.validator-379314.internal cosmovisor[840818]: github.com/cosmos/gaia/v13/app/app.go:235
Feb 05 13:03:42 interview-test3.us-central1-a.c.validator-379314.internal cosmovisor[840818]: github.com/cosmos/cosmos-sdk/baseapp.(*BaseApp).BeginBlock(_, {{0xc000be4100, 0x20, 0x20}, {{0xb, 0x0}, {0xc000d81b18, 0x11}, 0x13357b1, {0x2c5b33dd, ...}, ...}, ...})
Feb 05 13:03:42 interview-test3.us-central1-a.c.validator-379314.internal cosmovisor[840818]: github.com/cosmos/[email protected]/baseapp/abci.go:177 +0x97b
Feb 05 13:03:42 interview-test3.us-central1-a.c.validator-379314.internal cosmovisor[840818]: github.com/tendermint/tendermint/abci/client.(*localClient).BeginBlockSync(_, {{0xc000be4100, 0x20, 0x20}, {{0xb, 0x0}, {0xc000d81b18, 0x11}, 0x13357b1, {0x2c5b33dd, ...}, ...}, ...})
Feb 05 13:03:42 interview-test3.us-central1-a.c.validator-379314.internal cosmovisor[840818]: github.com/tendermint/[email protected]/abci/client/local_client.go:280 +0x118
Feb 05 13:03:42 interview-test3.us-central1-a.c.validator-379314.internal cosmovisor[840818]: github.com/tendermint/tendermint/proxy.(*appConnConsensus).BeginBlockSync(_, {{0xc000be4100, 0x20, 0x20}, {{0xb, 0x0}, {0xc000d81b18, 0x11}, 0x13357b1, {0x2c5b33dd, ...}, ...}, ...})
Feb 05 13:03:42 interview-test3.us-central1-a.c.validator-379314.internal cosmovisor[840818]: github.com/tendermint/[email protected]/proxy/app_conn.go:81 +0x55
Feb 05 13:03:42 interview-test3.us-central1-a.c.validator-379314.internal cosmovisor[840818]: github.com/tendermint/tendermint/state.execBlockOnProxyApp({0x25e2460?, 0xc000d7f3e0}, {0x25e9700, 0xc0001932e0}, 0xc000229a40, {0x25f0af8, 0xc000d6e498}, 0x13357b0?)
Feb 05 13:03:42 interview-test3.us-central1-a.c.validator-379314.internal cosmovisor[840818]: github.com/tendermint/[email protected]/state/execution.go:307 +0x51d
Feb 05 13:03:42 interview-test3.us-central1-a.c.validator-379314.internal cosmovisor[840818]: github.com/tendermint/tendermint/state.(*BlockExecutor).ApplyBlock(_, {{{0xb, 0x0}, {0xc000057380, 0x7}}, {0xc000d80e40, 0x11}, 0x89dbae, 0x13357b0, {{0xc000eeeb60, ...}, ...}, ...}, ...)
Feb 05 13:03:42 interview-test3.us-central1-a.c.validator-379314.internal cosmovisor[840818]: github.com/tendermint/[email protected]/state/execution.go:140 +0x171
Feb 05 13:03:42 interview-test3.us-central1-a.c.validator-379314.internal cosmovisor[840818]: github.com/tendermint/tendermint/consensus.(*Handshaker).replayBlock(_, {{{0xb, 0x0}, {0xc000057380, 0x7}}, {0xc000d80e40, 0x11}, 0x89dbae, 0x13357b0, {{0xc000eeeb60, ...}, ...}, ...}, ...)
Feb 05 13:03:42 interview-test3.us-central1-a.c.validator-379314.internal cosmovisor[840818]: github.com/tendermint/[email protected]/consensus/replay.go:503 +0x23c
Feb 05 13:03:42 interview-test3.us-central1-a.c.validator-379314.internal cosmovisor[840818]: github.com/tendermint/tendermint/consensus.(*Handshaker).ReplayBlocks(_, {{{0xb, 0x0}, {0xc000057380, 0x7}}, {0xc000d80e40, 0x11}, 0x89dbae, 0x13357b0, {{0xc000eeeb60, ...}, ...}, ...}, ...)
Feb 05 13:03:42 interview-test3.us-central1-a.c.validator-379314.internal cosmovisor[840818]: github.com/tendermint/[email protected]/consensus/replay.go:416 +0x7ae
Feb 05 13:03:42 interview-test3.us-central1-a.c.validator-379314.internal cosmovisor[840818]: github.com/tendermint/tendermint/consensus.(*Handshaker).Handshake(0xc000cd5c78, {0x25f1f40, 0xc0000b4000})
Feb 05 13:03:42 interview-test3.us-central1-a.c.validator-379314.internal cosmovisor[840818]: github.com/tendermint/[email protected]/consensus/replay.go:268 +0x3d4
Feb 05 13:03:42 interview-test3.us-central1-a.c.validator-379314.internal cosmovisor[840818]: github.com/tendermint/tendermint/node.doHandshake({_, _}, {{{0xb, 0x0}, {0xc000057380, 0x7}}, {0xc000d80e40, 0x11}, 0x89dbae, 0x13357b0, ...}, ...)
Feb 05 13:03:42 interview-test3.us-central1-a.c.validator-379314.internal cosmovisor[840818]: github.com/tendermint/[email protected]/node/node.go:329 +0x1b8
Feb 05 13:03:42 interview-test3.us-central1-a.c.validator-379314.internal cosmovisor[840818]: github.com/tendermint/tendermint/node.NewNode(0xc000ab2a00, {0x25de380, 0xc000bab720}, 0xc000d62d50, {0x25c8280, 0xc000d6fbf0}, 0x0?, 0x0?, 0xc000d62f70, {0x25e2460, ...}, ...)
Feb 05 13:03:42 interview-test3.us-central1-a.c.validator-379314.internal cosmovisor[840818]: github.com/tendermint/[email protected]/node/node.go:779 +0x597
Feb 05 13:03:42 interview-test3.us-central1-a.c.validator-379314.internal cosmovisor[840818]: github.com/cosmos/cosmos-sdk/server.startInProcess(_, {{0x0, 0x0, 0x0}, {0x25fe700, 0xc000aa4a50}, {0x0, 0x0}, {0x25e69b8, 0xc000eed190}, ...}, ...)
Feb 05 13:03:42 interview-test3.us-central1-a.c.validator-379314.internal cosmovisor[840818]: github.com/cosmos/[email protected]/server/start.go:280 +0x89b
Feb 05 13:03:42 interview-test3.us-central1-a.c.validator-379314.internal cosmovisor[840818]: github.com/cosmos/cosmos-sdk/server.StartCmd.func2(0xc000f4a600?, {0xc000aa4240?, 0x0?, 0x3?})
Feb 05 13:03:42 interview-test3.us-central1-a.c.validator-379314.internal cosmovisor[840818]: github.com/cosmos/[email protected]/server/start.go:128 +0x169
Feb 05 13:03:42 interview-test3.us-central1-a.c.validator-379314.internal cosmovisor[840818]: github.com/spf13/cobra.(*Command).execute(0xc000f4a600, {0xc000aa41b0, 0x3, 0x3})
Feb 05 13:03:42 interview-test3.us-central1-a.c.validator-379314.internal cosmovisor[840818]: github.com/spf13/[email protected]/command.go:940 +0x862
Feb 05 13:03:42 interview-test3.us-central1-a.c.validator-379314.internal cosmovisor[840818]: github.com/spf13/cobra.(*Command).ExecuteC(0xc000224f00)
Feb 05 13:03:42 interview-test3.us-central1-a.c.validator-379314.internal cosmovisor[840818]: github.com/spf13/[email protected]/command.go:1068 +0x3bd
Feb 05 13:03:42 interview-test3.us-central1-a.c.validator-379314.internal cosmovisor[840818]: github.com/spf13/cobra.(*Command).Execute(...)
Feb 05 13:03:42 interview-test3.us-central1-a.c.validator-379314.internal cosmovisor[840818]: github.com/spf13/[email protected]/command.go:992
Feb 05 13:03:42 interview-test3.us-central1-a.c.validator-379314.internal cosmovisor[840818]: github.com/spf13/cobra.(*Command).ExecuteContext(...)
Feb 05 13:03:42 interview-test3.us-central1-a.c.validator-379314.internal cosmovisor[840818]: github.com/spf13/[email protected]/command.go:985
Feb 05 13:03:42 interview-test3.us-central1-a.c.validator-379314.internal cosmovisor[840818]: github.com/cosmos/cosmos-sdk/server/cmd.Execute(0x0?, {0xc000eea730, 0xb})
Feb 05 13:03:42 interview-test3.us-central1-a.c.validator-379314.internal cosmovisor[840818]: github.com/cosmos/[email protected]/server/cmd/execute.go:36 +0x1eb
Feb 05 13:03:42 interview-test3.us-central1-a.c.validator-379314.internal cosmovisor[840818]: main.main()
Feb 05 13:03:42 interview-test3.us-central1-a.c.validator-379314.internal cosmovisor[840818]: github.com/cosmos/gaia/v13/cmd/gaiad/main.go:16 +0x2c
Feb 05 13:03:42 interview-test3.us-central1-a.c.validator-379314.internal cosmovisor[840810]: Error: exit status 2
Feb 05 13:03:42 interview-test3.us-central1-a.c.validator-379314.internal cosmovisor[840810]: 1:03PM ERR error="exit status 2" module=cosmovisor
Feb 05 13:03:42 interview-test3.us-central1-a.c.validator-379314.internal systemd[1]: cosmovisor.service: Main process exited, code=exited, status=1/FAILURE
Feb 05 13:03:42 interview-test3.us-central1-a.c.validator-379314.internal systemd[1]: cosmovisor.service: Failed with result 'exit-code'.
Feb 05 13:03:42 interview-test3.us-central1-a.c.validator-379314.internal systemd[1]: cosmovisor.service: Consumed 7.956s CPU time.
Feb 05 13:03:47 interview-test3.us-central1-a.c.validator-379314.internal systemd[1]: cosmovisor.service: Scheduled restart job, restart counter is at 3.
Feb 05 13:03:47 interview-test3.us-central1-a.c.validator-379314.internal systemd[1]: Stopped cosmovisor.service - Cosmovisor service.
Feb 05 13:03:47 interview-test3.us-central1-a.c.validator-379314.internal systemd[1]: cosmovisor.service: Consumed 7.956s CPU time. |
What do you get if you run your node without cosmovisor? It still looks like a node error to me and not an issue with cosmovisor. |
gaiad start --x-crisis-skip-assert-invariants --home /home/ubuntu/.gaia |
Make sure to run with the same user as the systemd service user, or use chown -R for the .gaia folder. |
it"s running on master |
I'm experiencing this same problem @julienrbrt Result of running
I get the same error when running gaiad with cosmovisor
|
Hi, this issue isn't cosmovisor related. Cosmovisor starts the app and then the app panics. Please refer to the gaia team. |
@umegbewe @Joshua1-Dev which Gaia version are you using and also which network are you pointing Gaia to? Can you also let us know the cosmovisor directory layout. |
Are you trying to catch up to the theta testnet? You needed to use To join at present time, you need to use |
Closing as there was no activity for 4 months. Replies above indicate users were attempting to run a non-supported version (either main/master) or some other version. The network does not use |
my node keeps crashing after a while with this errors
Feb 05 12:10:44 interview-test3.us-central1-a.c.validator-379314.internal systemd[1]: cosmovisor.service: Main process exited, code=exited, status=1/FAILURE
Feb 05 12:10:44 interview-test3.us-central1-a.c.validator-379314.internal systemd[1]: cosmovisor.service: Failed with result 'exit-code'.
this is my cosmovisor.service
[Unit]
Description=Cosmovisor service
After=network-online.target
[Service]
User=root
ExecStart=/home/ubuntu/go/bin/cosmovisor run start --x-crisis-skip-assert-invariants --home /home/ubuntu/.gaia
Restart=on-failure
RestartSec=5s
StartLimitBurst=5
StartLimitInterval=10m
LimitNOFILE=4096
Environment='DAEMON_NAME=gaiad'
Environment='DAEMON_HOME=/home/ubuntu/.gaia'
Environment='DAEMON_ALLOW_DOWNLOAD_BINARIES=true'
Environment='DAEMON_RESTART_AFTER_UPGRADE=true'
Environment='DAEMON_LOG_BUFFER_SIZE=512'
Environment='UNSAFE_SKIP_BACKUP=true'
[Install]
WantedBy=multi-user.target
what could be the issue?
The text was updated successfully, but these errors were encountered: