fix: implementing retrying_publish wrapper for the relay client

[geekbrother]

Description

This PR introduces retrying_publish function which wraps the current relay client .publish method and adds additional retrying and client reconnection logic to the current publishing mechanism:

• Checking if there is an error when publishing a message,
• Trying to reconnect with the 1-second retry interval in a loop,
• After successfully reconnection trying to re-publish the message,
• Give up when the publish errors after reconnecting.

The client.publish is replaced with the new function in websocket message handlers.

Resolves #74

How Has This Been Tested?

It should be tested by the current integration testing because the new function replaces the .publish which is indirectly used in the tests.

Due Diligence

• Breaking change
• Requires a documentation update
• Requires a e2e/integration test update

[geekbrother]

I don't like how connection is duplicated twice and this should be refactored according to DRY, but that can be a follow-up.
We already have a top-level reconnection logic, I'm sure it should be somewhere near it.

[chris13524]

Won't this conflict with the retry loop in websocket_service?

[chris13524]

I think we need to get the type of error. Not all errors are disconnect related

[geekbrother]

To handle properly with coming updates in #127 this implementation should be changed.
My thoughts about it and implementation should now look like below:

1. wsclient: add is_connected: bool and connected_notifier: Arc<Notify> for the state of the connection and notifying.
2. connect method spawning a detached (re)connection loop (in a Tokyo task) which changes the state of the connection when the connection is ready.
3. Publish method abstraction:
   1. If is_connected==false wait for the connected_notifier ,
   2. Publish a message,
   3. Trigger the connect method if the publish throws an error and wait for the connected_notifier before the re-publishing attempt.

What do you think, @chris13524 ?

[geekbrother]

Closing this PR in a favor of #128 128