fix: send notifications when UK database is down (#6498) #6596
+504
−2
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Implement notification system that works even when database is unavailable. Problem: - When Uptime Kuma's database (external MariaDB/SQLite) becomes unavailable, UK stops functioning and cannot send notifications about its own failure. - Users are not alerted when UK itself is having database connectivity issues. Solution: - Added notification cache system that stores all active notification configs in memory when database is available - Implemented database error detection in both monitor.js and server.js error handlers to catch EHOSTUNREACH, ECONNREFUSED, and other DB errors - Added sendDatabaseDownNotification() method that uses cached notifications to send alerts when database connection fails - Cache automatically refreshes periodically (every 30 minutes) and when notifications are added/updated/deleted - Prevents duplicate notifications for the same database down event Changes: - server/notification.js: Added cache system and database down notification - server/server.js: Enhanced error handler to detect DB errors and trigger notifications, refresh cache on startup - server/model/monitor.js: Added DB error detection in safeBeat error handler - server/jobs.js: Added periodic cache refresh job (every 30 minutes) - test/backend-test/test-database-down-notification.js: Comprehensive test suite covering cache, notifications, error handling This ensures users are always notified when UK itself is having database connectivity issues, even if the database is completely unavailable.
Author
|
hi @CommanderStorm , could you review my pr? |
Author
|
all of the tests passed :) @CommanderStorm |
CommanderStorm
requested changes
Jan 6, 2026
Collaborator
CommanderStorm
left a comment
CommanderStorm
left a comment
There was a problem hiding this comment.
Architecturally, we need to move this to its own file.
Adding new, larger features to the notification.js or monitor.js are not the best of ideas. (the files for this are alreay over 1k lines, which is way too large).
The code is also a bit too spaced out and duplicated and I don't think it is currently quite to the point that it is production ready in terms of "this cannot cause very alarming false positives".
Could you give it another iteration?
CommanderStorm
added
the
pr:please address review comments
louislam
requested changes
Jan 6, 2026
Owner
louislam
left a comment
louislam
left a comment
There was a problem hiding this comment.
Need to think more.
- Will it send to all notification providers? If yes, I don't think it should.
- Need GUI to config which notification providers want to receive this.
- Turn off by default
- Will it send only one notification during a period, or it keep sending notifications as long as it hit the errors? It may be a problem that it can strike the notification providers (rate limit or sms pricing etc.)
- Add send_database_down column to notification table (default: false) - Only notifications with send_database_down enabled receive database down alerts - Add GUI checkbox in NotificationDialog to configure opt-in setting - Implement 24-hour cooldown period to prevent notification spam - Add database error detector to avoid false positives - Update tests to verify opt-in behavior Addresses owner feedback: - Not sending to all providers (opt-in only) - GUI configuration available - Turned off by default - Cooldown prevents spam/rate limiting issues
- Check for missing is_default column and add it if needed - Handle outdated template database in CI/CD environment - Ensures test works even if template DB is missing columns
…martDever02/uptime-kuma into fix/database-down-notification
github-actions
bot
added
pr:needs review
Collaborator
|
Honestly, I don't think this code or the way it is integrated would be maintainable long term. I am tempted to just close this feature and its feature request since this really would need some architectural changes to work well. |
1 similar comment
Collaborator
|
Honestly, I don't think this code or the way it is integrated would be maintainable long term. I am tempted to just close this feature and its feature request since this really would need some architectural changes to work well. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Implement notification system that works even when database is unavailable.
Problem:
Solution:
Changes:
This ensures users are always notified when UK itself is having database connectivity issues, even if the database is completely unavailable.
Closes #6498
Contribution by Gittensor, see my contribution statistics at https://gittensor.io/miners/details?githubId=102175066