Why do agents disconnect and reconnect at regular intervals?

[kubamaruszczak]

I need to report each agent's connectivity status to an external system. Currently, I do this in meshcentral.js using the obj.SetConnectivityState and obj.ClearConnectivityState functions. However, I’ve noticed that agents disconnect at regular intervals and reconnect after 1 second. This disrupts my reporting system since two opposite states are sent quickly one after the other. In the screenshot below, you can see an analysis for one of the agents. The intervals are too consistent to be random.

My question is: why do agents disconnect at these intervals, and is it possible to prevent this?

I found the agentPong property and set it to 120 on my server. I'm currently testing whether this helps.

[si458]

they disconnect and reconnect because of a thing called the internet and the cables that connect them

the reason could be a number of things:
do u use a reverse proxy and the proxy disconnected them?
is ur server hosted in the cloud and the cloud has gone off for some reason?
did someone unplug the cables between the cloud and the remote machine ?
its hard to tell...
but generally the reason they disconnect and reconnect is because the websocket is dropping
generally setting agentPong to a LOW number helps as this sends a ping every X seconds to keep the socket open
for example the nginx docs dont set a time to live for websockets, so can disconnect them whenever they feel like
and meshcentral has no control over this
BUT setting agentPong: 20 tells it to keep the websocket open every 20 seconds which in turn keeps nginx open too!
so try setting the value LOW rather than high as 120 is 2 mintues for a timeout which isnt ideal
as for example most firewall have a timeout of 60 seconds for UDP traffic (just an example i no udp isnt tcp)

p.s sorry for the sarkyness ive just woke up and grumpy...
slow/tired morning for the jokes

p.ss: you shouldnt be editing meshcentral.js!!!
this is updated everytime we do updates,
you are better off writing a plugin and attaching to events - https://ylianst.github.io/MeshCentral/meshcentral/plugins/
we can add more events for plugins if you open an issue and let us know what events you are looking for and would like
and a donation would be greatful too! ❤️

[kubamaruszczak]

I keep track of my Meshcentral fork by doing regular updates. With each update, I merge my changes into the latest version of meshcentral.js and resolve any conflicts, if they exist. However, plugins seem to be an easier way of achieving the same result.

Based on what you said I assume that obj.SetConnectivityState and obj.ClearConnectivityState are the right places to monitor agent connectivity status. That's just the nature of the internet and it's more on my side to handle opposite connectivity states reports sent quickly one after the other. Am i right? If not, could you please suggest better place to do my connectivity reports?

I'm just trying to understand architecture and events a little more.

[si458]

yes you are correct, those are the best places, as they handle weather the agents/amt devices are connected or not

[kubamaruszczak]

Thanks for quick reply! Have a great day ahead!

[kubamaruszczak]

After some experimenting another related question arised. The obj.SetConnectivityState and obj.ClearConnectivityState seems to be the right places to monitor agent connectivity status. I experience agent disconnects and reconnects for most of the devices which happens because of web sockets nature. However, in my case for some devices after some time obj.ClearConnectivityState is called and obj.SetConnectivityState is never called again even if agent is connected. I know that agent is connected because I can connect to it via web view and when I list it from the db it has conn property set to 1. I experience it in peering environment. Why obj.SetConnectivityState isn't called if the agent is connected?

[si458]

The logic is correct

You should only need to call obj.SetConnectivityState when the device connects
and call obj.ClearConnectivityState when the device disconnects

Why would you want to call obj.SetConnectivityState over and over while the device is connected?

[kubamaruszczak]

I mean I wouldn't need to call obj.SetConnectivityState if obj.ClearConnectivityState wasn't called. My case is like below. I added logs in both functions to see whenever they are called for certain agent.

May 06 09:52:05 node[2767636]: SetConnectivityState for funky-vulture-supply
May 06 09:52:07 node[2767636]: ClearConnectivityState for funky-vulture-supply

So the SetConnectivityState wasn't called after ClearConnectivityState. I can still connected to the agent via web view, so I assume SetConnectivityState should be called at some point. Shouldn't it?

[si458]

I would have to look into more when I get chance as I can't remember what the functions do exactly as they do a mix of things

[kubamaruszczak]

Based on what I checked these are called from meshagent.js file in my case. As far as I know for each agent a meshagent object is created.