lightning service keeps restarting

[Freeben666]

Hi,

I have compiled from source CLN v24.11.1. It has been running flawlessly for nearly a month.

Yesterday I had to restart my node for update.

Ever since, the ligntning service keeps crashing and restarting. Here are the relevant logs:

Running on a Raspberry Pi 5 running Debian 12

2025-01-16T20:34:14.205Z INFO lightningd: v24.11.1
2025-01-16T20:34:15.216Z UNUSUAL plugin-bookkeeper: topic 'utxo_deposit' is not a known notification topic
2025-01-16T20:34:15.216Z UNUSUAL plugin-bookkeeper: topic 'utxo_spend' is not a known notification topic
2025-01-16T20:34:16.550Z INFO connectd: Static Tor service onion address: "XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX.onion:9735,127.0.0.1:9735" bound from extern port 9735
2025-01-16T20:34:16.577Z INFO plugin-bcli: bitcoin-cli initialized and connected to bitcoind.
2025-01-16T20:34:18.788Z UNUSUAL 03da1c27ca77872ac5b3e568af30673e599a47a5e4497f85c7b5da42048807b3ed-chan#2: gossipd lost track of announced channel: re-announcing!
2025-01-16T20:34:18.789Z UNUSUAL 033e9ce4e8f0e68f7db49ffb6b9eecc10605f3f3fcb3c630545887749ab515b9c7-chan#3: gossipd lost track of announced channel: re-announcing!
2025-01-16T20:34:18.854Z INFO plugin-clnrest: REST server running at https://127.0.0.1:3010
2025-01-16T20:34:19.091Z INFO lightningd: --------------------------------------------------
2025-01-16T20:34:19.091Z INFO lightningd: Server started with public key XXXXXXXXXXXXXXXX, alias XXX (color #ff0000) and lightningd v24.11.1
2025-01-16T20:36:30.211Z BROKEN gossipd: gossip_store: get delete entry offset 6266993/78571961 (version v24.11.1)
2025-01-16T20:36:30.211Z BROKEN gossipd: backtrace: common/daemon.c:38 (send_backtrace) 0x55564e063993
2025-01-16T20:36:30.211Z BROKEN gossipd: backtrace: common/status.c:206 (status_failed) 0x55564e06ffc7
2025-01-16T20:36:30.211Z BROKEN gossipd: backtrace: gossipd/gossip_store.c:466 (gossip_store_get_with_hdr) 0x55564e058e3f
2025-01-16T20:36:30.211Z BROKEN gossipd: backtrace: gossipd/gossip_store.c:491 (check_msg_type) 0x55564e058ecb
2025-01-16T20:36:30.211Z BROKEN gossipd: backtrace: gossipd/gossip_store.c:509 (gossip_store_set_flag) 0x55564e05909b
2025-01-16T20:36:30.211Z BROKEN gossipd: backtrace: gossipd/gossip_store.c:561 (gossip_store_del) 0x55564e05931b
2025-01-16T20:36:30.211Z BROKEN gossipd: backtrace: gossipd/gossmap_manage.c:793 (process_channel_update) 0x55564e05ac47
2025-01-16T20:36:30.211Z BROKEN gossipd: backtrace: gossipd/gossmap_manage.c:905 (gossmap_manage_channel_update) 0x55564e05b5d3
2025-01-16T20:36:30.211Z BROKEN gossipd: backtrace: gossipd/gossipd.c:210 (handle_recv_gossip) 0x55564e057c5f
2025-01-16T20:36:30.212Z BROKEN gossipd: backtrace: gossipd/gossipd.c:302 (connectd_req) 0x55564e057d5f
2025-01-16T20:36:30.212Z BROKEN gossipd: backtrace: common/daemon_conn.c:35 (handle_read) 0x55564e063cb7
2025-01-16T20:36:30.212Z BROKEN gossipd: backtrace: ccan/ccan/io/io.c:60 (next_plan) 0x55564e118657
2025-01-16T20:36:30.212Z BROKEN gossipd: backtrace: ccan/ccan/io/io.c:422 (do_plan) 0x55564e118c2f
2025-01-16T20:36:30.212Z BROKEN gossipd: backtrace: ccan/ccan/io/io.c:439 (io_ready) 0x55564e118d0b
2025-01-16T20:36:30.212Z BROKEN gossipd: backtrace: ccan/ccan/io/poll.c:455 (io_loop) 0x55564e11ac3f
2025-01-16T20:36:30.212Z BROKEN gossipd: backtrace: gossipd/gossipd.c:665 (main) 0x55564e058067
2025-01-16T20:36:30.212Z BROKEN gossipd: backtrace: ../sysdeps/nptl/libc_start_call_main.h:58 (__libc_start_call_main) 0x7ffeeda3773f
2025-01-16T20:36:30.212Z BROKEN gossipd: backtrace: ../csu/libc-start.c:360 (__libc_start_main_impl) 0x7ffeeda37817
2025-01-16T20:36:30.212Z BROKEN gossipd: backtrace: (null):0 ((null)) 0x55564e05466f
2025-01-16T20:36:30.212Z BROKEN gossipd: backtrace: (null):0 ((null)) 0xffffffffffffffff
2025-01-16T20:36:30.212Z BROKEN gossipd: STATUS_FAIL_INTERNAL_ERROR: gossip_store: get delete entry offset 6266993/78571961

Any idea what's going on?!

[Freeben666]

It seems removing (actually, renaming with .bak at the end) the gossip_store and gossip_store.corrupt solved the problem somehow. But was going on ? And if just restarting the gossip_store from scratch was the solution, why the crash?