How do you store measured values?

[ceskyDJ]

I'm considering using your product as it looks like a fantastic solution for my use case. I initially used Uptime Kuma for uptime monitoring, but it has its problems:

• missing support for distributive usage (I have a few servers in various isolated networks)—some agent for getting information from isolated networks,
• significant performance problems with many uptime sensors (80+ in my case),
• The details of the usage of HW resources are missing.

Checkmate looks like a solution for the first and the last problem when I use it with the Capture agent. This brings me to my question… Uptime Kuma stores measured values in a relational database (specifically SQLite, which isn't known for performance), and it wasn't a good deal for performance. I see you use MongoDB, which isn't optimized for time-series data. So, are you prepared for a potentially significant number of sensors (at least some hundreds)? How do you plan to achieve this goal? Uptime Kuma tries to figure it out via optimizations like shortening history and merging older values. I haven't tried it after these changes yet, but it didn't sound like a real solution to this problem, just a little incrementation of the highest number of usable sensors. I prefer the usage of some time-series database, as it's optimized for many writes and aggregation queries combined with some other database for other data (configuration, users, optimizations).

[ajhollid]

Hi @ceskyDJ,

Thanks for your interest in the project.

Data is indeed persisted with MongoDB, thus far it has been sufficient for our purposes.

I've personally run stress tests with 1000+ active monitors without any particular issues or performance bottlenecks. That said I only ran Checkmate with that many monitors for about a week, so I don't have extensive long term data.

We would need to see that longer term performance data to see where if any the bottlenecks in performance are. I don't want to commit to design or architecture decisions based purely on hypotheticals.

That said, the project has been written from the start with extensibility and modularity in mind, we are not tightly tied to MongoDB and can swap out databases with reasonable ease if it becomes clear that MognoDB performance is an issue.

Hope that helps answer your questions, let me know if you'd like to discuss further!

[ceskyDJ]

That said, the project has been written from the start with extensibility and modularity in mind, we are not tightly tied to MongoDB and can swap out databases with reasonable ease if it becomes clear that MognoDB performance is an issue.

Hmm, it sounds good to me. Uptime Kuma had to be rewritten, which was time-consuming and complicated (as far as I know). This approach seems to be way better.

I've personally run stress tests with 1000+ active monitors without any particular issues or performance bottlenecks. That said I only ran Checkmate with that many monitors for about a week, so I don't have extensive long term data.

We would need to see that longer term performance data to see where if any the bottlenecks in performance are. I don't want to commit to design or architecture decisions based purely on hypotheticals.

Ok, I think you're on the right way. The performance problem isn't just in the number of sensors. It worked just nicely for a few months. There is a long-term problem you need to consider. You can simulate long-term data, so you don't need to wait months for results. Less or more, you can duplicate your data and update timestamps to the past.

MongoDB may perform well; it's NoSQL DB, designed for big data. Anyway, I think it's better to know before the problem occurs. I agree hypotheses are reasonable but need to be proven empirically. As it should be rocket science, I'd recommend testing it to be sure. You can officially present your result if it will end well. Many people (as I've seen in Uptime Kuma's issues) need to be sure about performance as they have many targets to monitor and want to have long-term data for at least an annual analysis.

[gorkem-bwl]

Ok, I think you're on the right way. The performance problem isn't just in the number of sensors. It worked just nicely for a few months. There is a long-term problem you need to consider. You can simulate long-term data, so you don't need to wait months for results. Less or more, you can duplicate your data and update timestamps to the past.

That's a good point. I've worked with MongoDB before as well and it proved to be a very stable system for our needs (single server, a billion data points per month). While I personally don't think it would be an issue for MongoDB, we need to test it thoroughly for large systems as well IRL. I'll create an issue about this to see if anyone is interested as there are several contributors to the project despite its short life.

[gorkem-bwl]

Closing this but feel free to reopen (preferably in the Discussions area) if you would like to discuss further.

[ceskyDJ]

Oh, I'm sorry, it's definitely rather discussion than issue. You can convert the issue to discussion by following this guide: https://docs.github.com/en/discussions/managing-discussions-for-your-community/moderating-discussions#converting-an-issue-to-a-discussion.

I'd love to see some extensive performance benchmark for Checkmate as I want to miss the surprise after the UI is loading several tens of seconds for analytics and/or changing settings like it was with Uptime Kuma.

I don't have much time for setting up Checkmate these days, so I hope there will be some benchmark when I have time for looking on it. Maybe I could do it on my own (and probably need to, if you won't), but at least from my point of view it isn't just something only I need.

[gorkem-bwl]

Right, I fully understand your case. Currently we are focused on 2.0.1 and we can revisit this after the UI fixes here and there. :) Thanks again for bringing this up.