Detect serial ports fails on OSX

[benediktarnold]

Hi!
I installed the device manager on OSX (python 3.12) and I'm able to launch it. If I try to detect the serial ports, I get the following error:

2024-04-08 17:24:34,959 ERROR root Was not able to detect serial ports.
Traceback (most recent call last):
  File "/usr/local/Caskroom/miniconda/base/envs/eltako/lib/python3.12/site-packages/eo_man/view/serial_communication_bar.py", line 206, in detect_serial_ports
    self.main.config(cursor="watch")    #set cursor for waiting
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/Caskroom/miniconda/base/envs/eltako/lib/python3.12/tkinter/__init__.py", line 1721, in configure
    return self._configure('configure', cnf, kw)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/Caskroom/miniconda/base/envs/eltako/lib/python3.12/tkinter/__init__.py", line 1711, in _configure
    self.tk.call(_flatten((self._w, cmd)) + self._options(cnf))
RuntimeError: main thread is not in main loop

I'm no python expert but from a quick research:
It seems that detect_serial_ports_command starts a new thread and this thread calls functions on the GUI. With TK this seems to be forbidden.

I'll try to debug more into it and open a PR if I'm successful finding a solution ;-)

[benediktarnold]

I got the serial ports detection to work and was able to discover my FAM14. Some small changes were required that I did on my fork https://github.com/benediktarnold/enocean-device-manager

But if I try to connect, basically everything falls apart. A lot of app_bus event handler try to update the UI, but the handler are not called from the main thread.

Traceback (most recent call last):
  File "/Users/ben/dev/projects/enocean-device-manager/eo_man/controller/serial_controller.py", line 384, in _get_fam14_device_on_bus
    await self.app_bus.async_fire_event(AppBusEventType.ASYNC_DEVICE_DETECTED, {'device': fam14, 'fam14': fam14, 'force_overwrite': force_overwrite})
  File "/Users/ben/dev/projects/enocean-device-manager/eo_man/controller/app_bus.py", line 51, in async_fire_event
    for h in self._controller_event_handlers[event].values(): await h(data)
                                                              ^^^^^^^^^^^^^
  File "/Users/ben/dev/projects/enocean-device-manager/eo_man/data/data_manager.py", line 182, in _async_device_detected_handler
    self.app_bus.fire_event(AppBusEventType.UPDATE_DEVICE_REPRESENTATION, bd)
  File "/Users/ben/dev/projects/enocean-device-manager/eo_man/controller/app_bus.py", line 47, in fire_event
    for h in self._controller_event_handlers[event].values(): h(data)
                                                              ^^^^^^^
  File "/Users/ben/dev/projects/enocean-device-manager/eo_man/view/serial_communication_bar.py", line 100, in update_cb_gateways_for_HA
    self.cb_gateways_for_HA['values'] = gateways
    ~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^
  File "/usr/local/Caskroom/miniconda/base/envs/eltako311/lib/python3.11/tkinter/__init__.py", line 1732, in __setitem__
    self.configure({key: value})
  File "/usr/local/Caskroom/miniconda/base/envs/eltako311/lib/python3.11/tkinter/__init__.py", line 1721, in configure
    return self._configure('configure', cnf, kw)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/Caskroom/miniconda/base/envs/eltako311/lib/python3.11/tkinter/__init__.py", line 1711, in _configure
    self.tk.call(_flatten((self._w, cmd)) + self._options(cnf))
RuntimeError: main thread is not in main loop
Exception in thread Thread-5 (run):
Traceback (most recent call last):
  File "/usr/local/Caskroom/miniconda/base/envs/eltako311/lib/python3.11/threading.py", line 1045, in _bootstrap_inner
    self.run()
  File "/usr/local/Caskroom/miniconda/base/envs/eltako311/lib/python3.11/threading.py", line 982, in run
    self._target(*self._args, **self._kwargs)
  File "/Users/ben/dev/projects/enocean-device-manager/eo_man/controller/serial_controller.py", line 220, in run
    asyncio.run( self._get_fam14_device_on_bus() )
  File "/usr/local/Caskroom/miniconda/base/envs/eltako311/lib/python3.11/asyncio/runners.py", line 190, in run
    return runner.run(main)
           ^^^^^^^^^^^^^^^^
  File "/usr/local/Caskroom/miniconda/base/envs/eltako311/lib/python3.11/asyncio/runners.py", line 118, in run
    return self._loop.run_until_complete(task)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/Caskroom/miniconda/base/envs/eltako311/lib/python3.11/asyncio/base_events.py", line 654, in run_until_complete
    return future.result()
diff --git a/eo_man/controller/serial_controller.py b/eo_man/controller/serial_controller.py
           ^^^^^^^^^^^^^^^
  File "/Users/ben/dev/projects/enocean-device-manager/eo_man/controller/serial_controller.py", line 390, in _get_fam14_device_on_bus
    raise e
  File "/Users/ben/dev/projects/enocean-device-manager/eo_man/controller/serial_controller.py", line 384, in _get_fam14_device_on_bus
    await self.app_bus.async_fire_event(AppBusEventType.ASYNC_DEVICE_DETECTED, {'device': fam14, 'fam14': fam14, 'force_overwrite': force_overwrite})
  File "/Users/ben/dev/projects/enocean-device-manager/eo_man/controller/app_bus.py", line 51, in async_fire_event
    for h in self._controller_event_handlers[event].values(): await h(data)
                                                              ^^^^^^^^^^^^^
  File "/Users/ben/dev/projects/enocean-device-manager/eo_man/data/data_manager.py", line 182, in _async_device_detected_handler
    self.app_bus.fire_event(AppBusEventType.UPDATE_DEVICE_REPRESENTATION, bd)
  File "/Users/ben/dev/projects/enocean-device-manager/eo_man/controller/app_bus.py", line 47, in fire_event
    for h in self._controller_event_handlers[event].values(): h(data)
                                                              ^^^^^^^
  File "/Users/ben/dev/projects/enocean-device-manager/eo_man/view/serial_communication_bar.py", line 100, in update_cb_gateways_for_HA
    self.cb_gateways_for_HA['values'] = gateways
    ~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^
  File "/usr/local/Caskroom/miniconda/base/envs/eltako311/lib/python3.11/tkinter/__init__.py", line 1732, in __setitem__
    self.configure({key: value})
  File "/usr/local/Caskroom/miniconda/base/envs/eltako311/lib/python3.11/tkinter/__init__.py", line 1721, in configure
    return self._configure('configure', cnf, kw)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/Caskroom/miniconda/base/envs/eltako311/lib/python3.11/tkinter/__init__.py", line 1711, in _configure
    self.tk.call(_flatten((self._w, cmd)) + self._options(cnf))
RuntimeError: main thread is not in main loop

[grimmpp]

There seems to be an odd issue with pyserial affecting version 0.1.17 changed it back to a more OS-independent hack in 0.1.18. Unfortunately, I do not have OSX for testing.

[schiller256]

The issue still exists in Version 0.1.20

2024-04-25 11:03:05,938 ERROR root Was not able to detect serial ports. Traceback (most recent call last): File "/Users/xxxxx/.venv/lib/python3.12/site-packages/eo_man/view/serial_communication_bar.py", line 206, in detect_serial_ports self.main.config(cursor="watch") #set cursor for waiting ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/homebrew/Cellar/python@3.12/3.12.3/Frameworks/Python.framework/Versions/3.12/lib/python3.12/tkinter/__init__.py", line 1721, in configure return self._configure('configure', cnf, kw) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/homebrew/Cellar/python@3.12/3.12.3/Frameworks/Python.framework/Versions/3.12/lib/python3.12/tkinter/__init__.py", line 1711, in _configure self.tk.call(_flatten((self._w, cmd)) + self._options(cnf)) RuntimeError: main thread is not in main loop

[grimmpp]

Hello @schiller256,

sorry, I dont have OSX to test with.

[malthee]

I got this to run on OSX by doing some fixes. If anyone also only has MacOS and you want a Home Assistant export you can do the following:

1. Clone my fork with macOS fixes (Checkbox errors, Init errors, Datatable not displayed, Threading issues)

   git clone https://github.com/malthee/enocean-device-manager
   cd enocean-device-manager

2. Install Python 3.12 via python.org
   It already includes Tk 8.6. Avoid using the Homebrew Python/Tk; Brew gave me Tk 9+, which broke the UI.

3. Create a venv with that exact Python and install deps

   /Library/Frameworks/Python.framework/Versions/3.12/bin/python3 -m venv .venv
   source .venv/bin/activate
   pip install -r requirements.txt

4. Plug in the USB into your Module of choice if you have not already and then list your USB serial port (macOS uses /dev/cu.*)

   python -m serial.tools.list_ports -v
   Example result to use later: /dev/cu.usbserial-xxxxxxxx

5. Run the application

   python -m eo_man

6. In eo_man: enter the Serial Port → Connect and wait

7. Press “Scan for devices” and wait for the scan to finish

8. Export for Home Assistant
   Click the Home Assistant button and save the YAML.

---

Notes

I don't know if this code is ready for a PR, it has some changes that Windows does not need and I did not really check code quality. But for all you MacOS users out there this should at least be helpful to get that initial export.
"Writing to devices" kind of works, but it didn't learn in my lights or covers correctly so I had to do it manually after in Home Assistant. My situation was everything could be read, but nothing was written - it always reverted back to the state it was in.

Some tips for manually learning in:

• Lights in F4SR14 or similar allow for easy debugging by going into TEST mode, selecting the channels which will either already be documented by the export (x/4) or they will also light up in Home Assistant when configuring. Select the right channel and light in HA, select the LRN mode and press the "Teach-In-Button". After the flashing from the F4SR will stop and it should work.
• Covers were more complicated, did some trial and error in configuring. I have two FSB controllers and two physical covers and did not find a way to know which one used which motor since I did not do the installation. You will most likely select 180 for motor 1 or 200 for motor 2 on the top dial and then LRN it in the same way as a light.

All in all took some time but more or less happy with the result and thank you lots for doing the work of initial setup grimmpp!