Async Refactor WIP #75

fel115 · 2023-08-28T19:04:54Z

No description provided.

This keeps the old semantic with normal/senor mode but makes main more understandable. It's a good first step towards a refactor.

horenso · 2023-08-29T08:52:49Z

Seems like it only needs formatting.

horenso · 2023-08-29T15:24:02Z

We can merge this to have one consistent "old" version before the complete refactor.

markus2330 · 2023-08-30T07:38:48Z

We can merge this to have one consistent "old" version before the complete refactor.

If it is tested on hardware that it still works, we can do so, otherwise I don't see why we should do it. Master should be known to work. You can simply give v0.8.* tags on your branch when something is consistent. When we finish this refactoring, we can open v0.9.* for testing/fixing (then again on master) and finally we can release 1.0.0.

markus2330

Good to see progress, cannot wait to see more 🚀

files/opensesame.spec

src/main.rs

src/mod_ir_temp.rs

src/todo

horenso · 2023-08-31T18:46:21Z

As said, I would prefer if modules are self-contained, all logic/code related to a module should be within a module.

Yes, async and Traits can be tricky, we refactor it into stand-alone functions first and when it works, we see how to put it into Traits or whether we just put the functions into their respective module. The abstractions will come ;)

horenso · 2023-08-31T19:43:49Z

@markus2330 I have a few questions regarding config. Can we just have multiple copies of config cloating around, or should there only be one config object that is atomic? what about sync, is that done by libelectra?

horenso · 2023-08-31T19:43:51Z

@markus2330 I have a few questions regarding config. Can we just have multiple copies of config cloating around, or should there only be one config object that is atomic? what about sync, is that done by libelectra?

markus2330 · 2023-09-01T10:09:13Z

I have a few questions regarding config. Can we just have multiple copies of config cloating around,

Elektra's core definitely allows it but there are some plugins in Elektra that don't like concurrency. E.g. ElektraInitiative/libelektra#4466 was a problem for Opensesame but it should be fixed now. Furthermore, there might be issues if there are way too many open handles: ElektraInitiative/libelektra#3140

or should there only be one config object that is atomic?

Yeah, some mutex protection or similar is probably the simpler and safer way to use Elektra.

what about sync, is that done by libelectra?

Which sync do you mean? Sync on file system level is done by libelektra.

markus2330 · 2023-09-01T10:10:37Z

Elektra's core definitely allows it

To clarify: it is allowed (with potential limitations described above) to have separated instances of KDB, Key and KeySet per thread and use them concurrently.

…n CommandToButtons; and mod_ir_temps.rs and clima_sensor_us.rs remove timeouts

…into refactor_main

…r check if capacity changed; reorder interval.tick()

src/audio.rs

fel115 · 2023-09-29T09:47:05Z

I think the refactoring to async is done. Other changes can be done in a separate PR. I already tested Buttons, Bell, PWR, Environment, Battery, Watchdog, Audio Bell, Garage, Signal, and Sensors. Today I am going to test Weatherstation in combination with other modules.

Which version of opensesame should this PR be? v0.8.0?

@horenso @markus2330

markus2330 · 2023-09-29T10:03:12Z

I think the refactoring to async is done. Other changes can be done in a separate PR. I already tested Buttons, Bell, PWR, Environment, Battery, Watchdog, Audio Bell, Garage, Signal, and Sensors.

Great! 🚀

Is it already ready to review? How to start reviewing this big PR, can you write some intro to the module system and how everything works together?

Today I am going to test Weatherstation in combination with other modules.

Please watch out that the weatherstation keeps on publishing most of the time. (If you install a broken version, please replace it with a working version again afterwards.)

Which version of opensesame should this PR be? v0.8.0?

Yes, at least if already everything works. Maybe even 0.9.0 to indicate we will be soon at 1.0.0?

fel115 · 2023-09-29T11:04:34Z

Is it already ready to review? How to start reviewing this big PR, can you write some intro to the module system and how everything works together?

Yes, now it's ready for review. I hope my explanation below is helping.

OpenSesame

audio.rs

This module handles audio output for playing fire alarms and bell sounds. This module can receive commands from the Module Buttons (Bell), Environment (FireAlarm), Nextcloud (FireAlarm, Bell), and Signals (FireAlarm, Bell).

bat.rs

Checks the battery capacity every ten minutes and outputs to Nextcloud if it falls below 50%. If it goes below 50%, the threshold for the next Nextcloud message is set to 40%.

buttons.rs

This module implements the do_reset if an error occurs on the MOD-IO2 modules. In the async get_background_task, we implemented how the buttons were handled in the old main, including the validator and command receiver. The command receiver receives commands like opendoor (used by Garage, Nextcloud), ring_bell (used by Environment, Signals), switchlights (used by Garage, Nextcloud), and RingBellAlarm (used by Signals).

clima_sensor_us.rs

This module works independently, sending warnings to Nextcloud and publishing to opensensemap. We needed to implement Send to use libmodbus with async functions.

config.rs

No changes were made to this module.

environment.rs

Changed the functions rememberBaseline, restoreBaseline, and added a Muext of state. We moved handle_environment from the main into the get_background_task. This module receives commands (RestoreBaseline, RememberBaseline) from Signals.

garage.rs

This module is checked at intervals of 10 milliseconds, triggering Nextcloud Chat or Buttons commands if the button is pressed. Future changes will involve removing the interval and implementing trigger-oriented events because GPIO pins from the Olimex board are used, along with interrupts.

mod_ir_temp.rs

This module is triggered at given intervals and warns by sending Nextcloud messages.

nextcloud.rs

Implements two loops: one for sending (message_sender_loop) messages and status to Nextcloud, and the other for receiving (command_loop) messages/commands from Nextcloud. Commands can be sent via Nextcloud chat by typing "\opensesame" to open the door, or other commands like "\ring_bell", "\fire_alarm", "\status", and "\switchlights true true".

ping.rs

This module sends a ping message to Nextcloud if it receives the SendPing event. Other functions update Env, EnvStatus, EnvError, and BatCapacity, but these commands aren't used yet. They need to be triggered at the right spot in Environment and Battery to keep the ping up to date.

pwr.rs

No significant changes were made to this module.

sensors.rs

No major changes, only the get_background_task was added with a similar implementation as in the old version.

signals.rs

This module listens to system signals and executes the same events as in the old version.

ssh.rs

Changed to an async function.

types.rs

This module contains the error types defined so far.

validator.rs

No changes were made to this module.

watchdog.rs

This background task is triggered every few seconds and writes to the specified file. The test_get_background_task function is used to simulate the watchdog stopping writing to the watchdog file.

main.rs

In this loop, we initialize every module if it is enabled. Additionally, the MPSC (Multiple Producer, Single Consumer) channels are initialized here. To configure each module without encountering issues related to multiple access to one config object, we read the configuration in the main.

markus2330

Great progress, I like many of these changes. The documentation is a bit poor, please describe what works now for all the modules. There are a few German comments. Some TODOs are not clear. TODOs shouldn't be bugs in code (for that please create issues) but only things that could be improved.

doc/Testreport_Async_Refactor.md

doc/modules.md

emu_dev/emulate.sh

files/opensesame.spec

src/mod_ir_temp.rs

src/nextcloud.rs

src/signals.rs

Because it isn't working, and wasn't implemented in the old version of opensesame

markus2330

(only a few answers, did not review again)

src/sensors.rs

src/nextcloud.rs

src/signals.rs

src/main.rs

markus2330

A few last issues are to be created and a few TODOs should refer to these issues, then we can merge this. I could test this tomorrow if we manage to merge it till then.

doc/Testreport_Async_Refactor.md

doc/modules.md

src/buttons.rs

src/sensors.rs

fel115 · 2023-10-14T13:55:11Z

I merged this branch with the main, and it is now ready. I can't merge into this repo @markus2330.

markus2330 · 2023-10-14T14:21:13Z

Finally done 🚀 ❤️

horenso and others added 3 commits August 28, 2023 16:57

Refactor main: keep old semantic

0e7f8e1

This keeps the old semantic with normal/senor mode but makes main more understandable. It's a good first step towards a refactor.

Add scripts to emulate device.

3e50b6f

impl MOD-IR-TMP & Weatherstation

022fe0b

exec - cagro fmt

d9b14e5

horenso added 2 commits August 30, 2023 19:53

WIP

9f38a18

WIP: Async

84c29a0

markus2330 reviewed Aug 31, 2023

View reviewed changes

files/opensesame.spec Show resolved Hide resolved

src/main.rs Outdated Show resolved Hide resolved

src/mod_ir_temp.rs Outdated Show resolved Hide resolved

src/todo Outdated Show resolved Hide resolved

src/todo Outdated Show resolved Hide resolved

fel115 changed the title ~~Refactor main~~ Async Refactor WIP Aug 31, 2023

horenso added 2 commits August 31, 2023 22:35

More refactoring work: Split up functionality

814a60a

Update reqwest to 0.11

096c159

horenso and others added 12 commits September 1, 2023 12:32

Refactor nextcloud module.

04693c9

Changes in env_loop, weatherstation_loop, modir_loop, sensors_loop; i…

5bddb7c

…n CommandToButtons; and mod_ir_temps.rs and clima_sensor_us.rs remove timeouts

Merge branch 'refactor_main' of https://github.com/horenso/opensesame …

1aa694e

…into refactor_main

WIP move config out of loop

853e7c7

More async refactoring.

a1dbe33

compiling worked

35207ad

Add join_all in main such that main waits for async modules.

dbf0dd5

ssh async; ErrorType

6c3edbd

Buttons+Validator, Garage, Sensors, ModIR, Env, Nextcloud working

adca7d0

Merge branch 'refactor_main_update_async_loops' into refactor_main

d913bb6

Update gatage_loop after merge

a25a50e

Add serde_json for json deserialization.

6419f35

horenso and others added 2 commits September 27, 2023 15:38

Cancel previous audio signal when new Alarm/Bell signal comes in.

9cfd75a

Implementation of Issue ElektraInitiative#87

c269875

fel115 mentioned this pull request Sep 27, 2023

API: chat(kind, msg) and status(kind, msg) #87

Closed

fel115 added 4 commits September 27, 2023 19:40

implementation of bat.rs + cargo fmt

b8dc9bc

changes in button.rs to handle do_reset; update bat.rs to only trigge…

e423420

…r check if capacity changed; reorder interval.tick()

added test function for watchdog

77f004a

update specs; bat.rs reordering;

9cec7d5

markus2330 reviewed Sep 28, 2023

View reviewed changes

src/audio.rs Outdated Show resolved Hide resolved

fel115 marked this pull request as ready for review September 29, 2023 09:19

audio.rs logging to Ping Chat

4a8d992

markus2330 requested changes Sep 29, 2023

View reviewed changes

fel115 added 2 commits October 2, 2023 16:10

rm state.set;

4244064

Because it isn't working, and wasn't implemented in the old version of opensesame

some requested changes

e4cac3c

markus2330 reviewed Oct 7, 2023

View reviewed changes

src/sensors.rs Outdated Show resolved Hide resolved

src/nextcloud.rs Show resolved Hide resolved

src/signals.rs Show resolved Hide resolved

src/main.rs Show resolved Hide resolved

This was referenced Oct 12, 2023

Consistent use of config in main-loop #96

Open

Change Audio Module to explicit enable #97

Open

Refactoring of nextcloud code #98

Open

remove prints

4c7aed4

fel115 requested review from markus2330 and horenso October 13, 2023 18:25

markus2330 approved these changes Oct 14, 2023

View reviewed changes

fel115 mentioned this pull request Oct 14, 2023

First entering of pin caused time sequence timeout #102

Closed

fel115 added 2 commits October 14, 2023 15:11

changed requested changes

6e7599a

Merge branch 'master' into refactor_main

5619c48

markus2330 merged commit 88f9d37 into ElektraInitiative:master Oct 14, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Async Refactor WIP #75

Async Refactor WIP #75

fel115 commented Aug 28, 2023

horenso commented Aug 29, 2023

horenso commented Aug 29, 2023

markus2330 commented Aug 30, 2023 •

edited

Loading

markus2330 left a comment

horenso commented Aug 31, 2023 •

edited

Loading

horenso commented Aug 31, 2023

horenso commented Aug 31, 2023

markus2330 commented Sep 1, 2023

markus2330 commented Sep 1, 2023

fel115 commented Sep 29, 2023 •

edited

Loading

markus2330 commented Sep 29, 2023

fel115 commented Sep 29, 2023

markus2330 left a comment

markus2330 left a comment

markus2330 left a comment

fel115 commented Oct 14, 2023 •

edited

Loading

markus2330 commented Oct 14, 2023

Async Refactor WIP #75

Async Refactor WIP #75

Conversation

fel115 commented Aug 28, 2023

horenso commented Aug 29, 2023

horenso commented Aug 29, 2023

markus2330 commented Aug 30, 2023 • edited Loading

markus2330 left a comment

Choose a reason for hiding this comment

horenso commented Aug 31, 2023 • edited Loading

horenso commented Aug 31, 2023

horenso commented Aug 31, 2023

markus2330 commented Sep 1, 2023

markus2330 commented Sep 1, 2023

fel115 commented Sep 29, 2023 • edited Loading

markus2330 commented Sep 29, 2023

fel115 commented Sep 29, 2023

OpenSesame

audio.rs

bat.rs

buttons.rs

clima_sensor_us.rs

config.rs

environment.rs

garage.rs

mod_ir_temp.rs

nextcloud.rs

ping.rs

pwr.rs

sensors.rs

signals.rs

ssh.rs

types.rs

validator.rs

watchdog.rs

main.rs

markus2330 left a comment

Choose a reason for hiding this comment

markus2330 left a comment

Choose a reason for hiding this comment

markus2330 left a comment

Choose a reason for hiding this comment

fel115 commented Oct 14, 2023 • edited Loading

markus2330 commented Oct 14, 2023

markus2330 commented Aug 30, 2023 •

edited

Loading

horenso commented Aug 31, 2023 •

edited

Loading

fel115 commented Sep 29, 2023 •

edited

Loading

fel115 commented Oct 14, 2023 •

edited

Loading