Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

does not cleanly shut down on SIGTERM #646

Closed
dionysius opened this issue Mar 14, 2025 · 1 comment · Fixed by #647
Closed

does not cleanly shut down on SIGTERM #646

dionysius opened this issue Mar 14, 2025 · 1 comment · Fixed by #647

Comments

@dionysius
Copy link
Contributor

dionysius commented Mar 14, 2025 •
edited
Loading

What steps did you take and what happened:
Running zfs-localpv natively on a node without daemonset requires common service handling (systemd and alternatives). When stopping the service (SIGTERM) the process keeps running. Thus there must be a goroutine still running. A second stop (SIGTERM) stops the process finally. If it matters it was running successfully in agent mode. Running the binary manually yields the same issue, first Ctrl-C (SIGINT) does not exit the process but yields the output below, needs a second Ctrl-C (SIGINT) to finally exit.

Last few lines in the output:

I0314 03:28:09.527507   10320 volume.go:198] Shutting down ZV workers
I0314 03:28:09.527609   10320 restore.go:179] Shutting down Restore workers
I0314 03:28:09.527531   10320 zfsnode.go:214] Shutting down Node controller
I0314 03:28:09.527638   10320 snapshot.go:187] Shutting down Snap workers
I0314 03:28:09.527644   10320 backup.go:184] Shutting down Bkp workers

What did you expect to happen:
process cleanly stops on SIGTERM

Environment:

  • LocalPV-ZFS version: v2.7.1
  • Kubernetes version (use kubectl version): doesn't matter
  • Kubernetes installer & version: none
  • Cloud provider or hardware configuration: doesn't matter
  • OS (e.g. from /etc/os-release): Ubuntu jammy
@dionysius dionysius changed the title binary does not cleanly shut down on SIGTERM does not cleanly shut down on SIGTERM Mar 14, 2025
@dionysius
Copy link
Contributor Author

dionysius commented Mar 14, 2025 •
edited
Loading

Seems like grpc is still running and doesn't get the stopCh forwarded:

I0314 04:37:01.187532   32409 zfsnode.go:214] Shutting down Node controller
I0314 04:37:01.187552   32409 volume.go:198] Shutting down ZV workers
I0314 04:37:01.187526   32409 restore.go:179] Shutting down Restore workers
I0314 04:37:01.187590   32409 backup.go:184] Shutting down Bkp workers
I0314 04:37:01.187601   32409 snapshot.go:187] Shutting down Snap workers
I0314 04:41:10.040413   32409 grpc.go:72] GRPC call: /csi.v1.Identity/GetPluginInfo requests {}
I0314 04:41:10.041676   32409 grpc.go:81] GRPC response: {"name":"zfs.csi.openebs.io","vendor_version":"v2.7.1"}

@dionysius dionysius mentioned this issue Mar 14, 2025
Merged
7 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

fix(grpc): stop server on SIGTERM and SIGINT dionysius/zfs-localpv
1 participant
@dionysius