checking bot is alive via POST command

[sharuzzaman]

Hi,

I have an issue where the bot is running, but it is not responding.

So, if I use the command line tools to check the PID, it will return it correctly, but it does not mean the bot is responsive.

I'm thinking along the line of POST-ing to the bot with endpoint /health or /status, eg. http://1.2.3.4:8080/health with PING, and the bot will return with PONG, much like how Redis doing its health check. Reference: https://www.inovex.de/de/blog/redis-cli/

With that endpoint available, I can configure UptimeRobot to post the PING command and expect the PONG in return, and it will surely be able to monitor any downtime from that interaction.

Comments appreciated.

[J-Rios]

Hello,

So you mean that the Bot process is fine and running, but the Bot is not responding through Telegram? Then your point is to have some mechanism/interface to check if the Bot has some issue that doesn't allow to communicate through Telegram (i.e. Telegram Server is not available, or it is limiting communication due flood on a specific chat group, etc.)...

If so, I think creating an HTTP API-Rest Server to just offer this "health" check is somekind of over-engineering, it adds a full new component and interface to the project that then need to be maintaned and considered.

Only way were the implementation of that API-Rest make sense is when we want to really add that interface for further implementation and other usages, like request statistics, export some logs and extra maintenance tools, and this means a lot of work and implementations to support all these different things. But I really don't see any need for all of this...

What is your specific use case?
Is not enough to send some Telegram command to the Bot to check this?
Is not enough to check Bot "output.log" file to see if there is any issue in the Bot?

Regards :)

[sharuzzaman]

Hi,

I understand the complexity factor for adding the REST API interface.

The issue that I'm having quite frequently is the bot is running, the Linux OS was not hang, but the bot does not respond to any command sent to it via Telegram, eg. /version

And I have not able to monitor this until some bots start to sending spam to my group, then I became aware that my bot has stopped responding for quite some time.

The idea for the health check is to get some automatic monitoring tool to keep checking, and sending me email if the bot is not alive or not responding. While I can just open a chat window with my bot and send /version to it, as human sometimes I forgot or quite busy with work.

If there are monitoring in place, when the issue happen, eg. 3 AM in the morning, at least there was an email sent and when I check my mailbox in the morning, I would aware the bot is not responding.

I have checked the output.log and it does not show any error. Here is one example:

2025-01-03 13:25:14,788 - main - INFO - [-1001018484308] Captcha Fail - Ban - ‎@Nkskksj (7752204818)
2025-01-03 13:25:14,979 - main - INFO - [-1001018484308] Warning: The user ‎@Nkskksj tried and failed to resolve the captcha 1 times. The "user" was considered a Bot and
 banned. To let this "user" enter again, an Admin has to manually remove it restrictions from the group settings.
2025-01-03 13:25:15,176 - main - INFO - [-1001018484308] User ‎@Nkskksj (7752204818) left the group
2025-01-03 13:25:15,489 - main - INFO - [-1001018484308] Removing msgs from user ‎@Nkskksj...
2025-01-03 13:25:15,489 - main - WARNING - [-1001018484308] ‎@Nkskksj (7752204818) not in new_users list (already solve captcha)
2025-01-05 07:43:36,313 - telegram.ext._application - INFO - Application is stopping. This might take a moment.
2025-01-05 07:43:37,643 - apscheduler.scheduler - INFO - Scheduler has been shut down
2025-01-05 07:43:37,708 - telegram.ext._application - INFO - Application.stop() complete
2025-01-05 07:43:38,156 - main - INFO - Bot stopped. Releasing resources...
2025-01-05 07:43:38,156 - main - INFO - Waiting coroutine end: captcha_timeout()

As you can see in the log, there is a large gap from 3 Jan to 5 Jan, and nothing being logged during this time. This is the time when the bot is not responding.

I could create a cron job hourly if there is a new log appearing for the past hour, and if not, send me an email, but I believe that is more like a hack than an elegant solution.

Thank you.

[J-Rios]

Regarding the Issue

That log gap is a pretty weird and unexpected behaviour. I have a couple of Bot instances deployed (with webhook) and running for months without any restart, and I got no issue...

That feels like the thread for handling Telegram communication crashes silently (without an error, I don't think so), Telegram has banned your Bot instance for some reason, or Telegram Bot API Server is not reachable (networking issue), etc.

Some questions ideas to try identify the root cause of the issue:

• Which version of the Bot are you using (branch and version)?
• Which Python version are you using? and which library python-telegram-bot framework version (same as defined for the project, or you have update them to newer versions)?
• Have you modified the Bot code that could introduce some bug?
• Is your Bot configured to use Polling or Webhook?
• Is your Bot in multiple groups or just in a single one?
• Have you noticed some weird Bot usage behaviour taht could make Telegram to ban the Bot?
• Do you have enough RAM memory and disk space in the machine were the Bot is running?
• Have you noticed some network communication lost on that period of time on the machine?

Regarding the feature

I understand that could be nice to have an HTTP API-Rest that you can check to know if the Bot is running properly or has some issues, and is more near to other web development related apps, but sometimes simpler solutions is the best approach. Perhaps just having an external script that periodically checks that log file to identify issues from there and handle the notification system (email or whatever) is a more simpler and isolated approach to avoid adding complexity to the Bot project.