ADS integration Exec format error on latest docker

[jandeschuttere]

The problem

When starting the latest docker on a raspberry pi 4 the ADS integration seems to be broken, sounds like a build problem targeting the wrong architecture.

I've had a couple of issues (like MySql 8 suddenly requiring the mysql_native_password plugin instead of the previously already working caching_sha2_password) which I was able to work around but this one is blocking for me as it completely breaks the integration with my Beckhoff which is at the heart of my home control.

Labels provided through this image are:

"io.hass.arch": "armv7",
"io.hass.base.arch": "armv7",
"io.hass.base.image": "homeassistant/armv7-base:3.16",
"io.hass.base.name": "python",
"io.hass.base.version": "2022.06.0",
"io.hass.machine": "raspberrypi4",
"io.hass.type": "core",
"io.hass.version": "2022.7.0.dev20220627",
"org.opencontainers.image.authors": "The Home Assistant Authors",
"org.opencontainers.image.created": "2022-06-27 03:32:54+00:00",
"org.opencontainers.image.description": "Open-source home automation platform running on Python 3",
"org.opencontainers.image.documentation": "https://www.home-assistant.io/docs/",
"org.opencontainers.image.licenses": "Apache License 2.0",
"org.opencontainers.image.source": "https://github.com/home-assistant/core",
"org.opencontainers.image.title": "Home Assistant",
"org.opencontainers.image.url": "https://www.home-assistant.io/",
"org.opencontainers.image.version": "2022.7.0.dev20220627"

Labels of the image that is still working for me:

"io.hass.arch": "armv7",
"io.hass.base.arch": "armv7",
"io.hass.base.image": "homeassistant/armv7-base:3.14",
"io.hass.base.name": "python",
"io.hass.base.version": "2022.02.0",
"io.hass.machine": "raspberrypi4",
"io.hass.type": "core",
"io.hass.version": "2022.6.0.dev20220506",
"org.opencontainers.image.authors": "The Home Assistant Authors",
"org.opencontainers.image.created": "2022-05-06 03:33:40+00:00",
"org.opencontainers.image.description": "Open-source home automation platform running on Python 3",
"org.opencontainers.image.documentation": "https://www.home-assistant.io/docs/",
"org.opencontainers.image.licenses": "Apache License 2.0",
"org.opencontainers.image.source": "https://github.com/home-assistant/core",
"org.opencontainers.image.title": "Home Assistant",
"org.opencontainers.image.url": "https://www.home-assistant.io/",
"org.opencontainers.image.version": "2022.6.0.dev20220506"

What version of Home Assistant Core has the issue?

core-2022.7.0.dev20220627

What was the last working version of Home Assistant Core?

core-2022.6.0.dev20220506

What type of installation are you running?

Home Assistant Container

Integration causing the issue

ADS

Link to integration documentation on our website

https://www.home-assistant.io/integrations/ads/

Diagnostics information

No response

Example YAML snippet

No response

Anything in the logs that might be useful for us?

These is just one snippet of the many similar issues reported, figured this stacktrace is sufficient:

ERROR (MainThread) [homeassistant.config] Platform error: light
Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/loader.py", line 618, in get_platform
    cache[full_name] = self._import_platform(platform_name)
  File "/usr/src/homeassistant/homeassistant/loader.py", line 635, in _import_platform
    return importlib.import_module(f"{self.pkg_path}.{platform_name}")
  File "/usr/local/lib/python3.10/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 992, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 883, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/usr/src/homeassistant/homeassistant/components/ads/__init__.py", line 10, in <module>
    import pyads
  File "/usr/local/lib/python3.10/site-packages/pyads/__init__.py", line 12, in <module>
    from .ads import open_port, close_port, get_local_address, read_state, \
  File "/usr/local/lib/python3.10/site-packages/pyads/ads.py", line 18, in <module>
    from .pyads_ex import (
  File "/usr/local/lib/python3.10/site-packages/pyads/pyads_ex.py", line 82, in <module>
    _adsDLL = ctypes.CDLL(adslib)
  File "/usr/local/lib/python3.10/ctypes/__init__.py", line 374, in __init__
    self._handle = _dlopen(self._name, mode)
OSError: Error loading shared library /usr/local/lib/python3.10/site-packages/pyads/adslib.so: Exec format error

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/config.py", line 877, in async_process_component_config
    platform = p_integration.get_platform(domain)
  File "/usr/src/homeassistant/homeassistant/loader.py", line 627, in get_platform
    raise ImportError(
ImportError: Exception importing homeassistant.components.ads.light

### Additional information

_No response_

[probot-home-assistant]

ads documentation
ads source
_{^{(message by IssueLinks)}}

[jorenreynders]

Same issue on the raspberry pi 4 running Home Assistant OS.
Had to revert back to 2022.6.7

[jandeschuttere]

I have to still look further into the problem - time.. you precious thing - as I'm afraid this plugin has less usage and therefor less priority.

From what I gather it looks like the internals use https://github.com/stlehmann/pyads and the issues could be related to the python 3.10 upgrade

[jandeschuttere]

so looks like /usr/local/lib/python3.10/site-packages/pyads/adslib.so is indeed an x86-64 type, as identified via file

/usr/local/lib/python3.10/site-packages/pyads/adslib.so: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, with debug_info, not stripped

meaning this is incorrectly packaged with an arm container, can this be looked into?

[jandeschuttere]

For those looking for a (dirty) workaround the package from the working version is compatible it seems with the new release. Assuming you are using docker the following steps might help you out until this gets a fix:

Given: containerA is the name of your working container, containerB is the name of the new container with the faulty adslib.

docker cp containerA:/usr/local/lib/python3.9/site-packages/pyads /tmp
docker stop containerA
docker start containerB
docker cp /tmp/pyads containerB:/usr/local/lib/python3.10/site-packages/
docker restart containerB

If you are not using docker, the digest is basically.. take the pyads lib from the working version, copy it in the location that the new homeassistant build is using.

[jandeschuttere]

@frenck sorry to tag you personally but as you have ads integration tagged in recent PR #74522 I just wanted to check if above was solved through that PR? (the PR changes were simply too big for sieving through)

[reib3n]

For those looking for a (dirty) workaround the package from the working version is compatible it seems with the new release. Assuming you are using docker the following steps might help you out until this gets a fix:

Given: containerA is the name of your working container, containerB is the name of the new container with the faulty adslib.

docker cp containerA:/usr/local/lib/python3.9/site-packages/pyads /tmp
docker stop containerA
docker start containerB
docker cp /tmp/pyads containerB:/usr/local/lib/python3.10/site-packages/
docker restart containerB

If you are not using docker, the digest is basically.. take the pyads lib from the working version, copy it in the location that the new homeassistant build is using.

I tried to get the workaround working for me without docker, but it seems the directory is not there you described. There is no python3.9 folder. I am using core 2022.06.7 an got access with the Terminal & SSH Addon. May you give me a hint what I am doing wrong?

Thanks a lot.

[jandeschuttere]

I've not tried the core installation but according to the information it is using the python venv method. You could identify the locations python is using by doing python -m site or python -m site --user-site and go through the listed directories to find the pyads site-package. Once you've found this you could copy it (you could use scp or rsync for this if using a terminal and the installation is remote) to your local system and identify the location of the pyads site-package on the new installation and move them there.

[reib3n]

I've not tried the core installation but according to the information it is using the python venv method. You could identify the locations python is using by doing python -m site or python -m site --user-site and go through the listed directories to find the pyads site-package. Once you've found this you could copy it (you could use scp or rsync for this if using a terminal and the installation is remote) to your local system and identify the location of the pyads site-package on the new installation and move them there.

Thanks for the hints, but this doesn't worked for me either.

I got the workaround now working with the Add-on: SSH & Web Terminal.

With this Add-on I could connect to HassOS via SSH and use the "docker" workaround described above.

Thanks a lot guys! hopefully this will be fixed soon!

[zimbrich]

I don't have much experience with Python, but I have a ton of experience with ADS communications and all things TwinCAT-related. Let me know if I can be of any assistance in fixing this issue.

[jandeschuttere]

Thanks for reaching out! From what I gather this is a build issue on this project instead of having an actual issue with the library or the communication itself. Mainly because the library provided is bundled but is simply targeting the wrong cpu architecture and switching to the right one causes no issues.

I have scavanged a bit through the various repositories of this project to see if I can identify the build process itself to see if I can spot an issue there (f.i. a cache layer of some sorts during the build process) but have not yet found it.

[Jeroen45]

I am not able to find this location : /usr/local/lib/python3.9/site-packages/pyads
Can some one help me, Im running HassIO
I do have: /usr/local/lib but no python folder

[jandeschuttere]

#74067 (comment) might be able to help you locate the packages that your installation is using

it will show the base directory/directories in which python is running, one of them should have a subdirectory pyads

[reib3n]

I am not able to find this location : /usr/local/lib/python3.9/site-packages/pyads
Can some one help me, Im running HassIO
I do have: /usr/local/lib but no python folder

I got the same issue at first. I needed to use for the ssh connection the SSH & Web Terminal Addon with disabled protection mode. After that I could locate the directory

[Jeroen45]

I am not able to find this location : /usr/local/lib/python3.9/site-packages/pyads
Can some one help me, Im running HassIO
I do have: /usr/local/lib but no python folder

I got the same issue at first. I needed to use for the ssh connection the SSH & Web Terminal Addon with disabled protection mode. After that I could locate the directory

Unfortunately i dont have version 12.0.2. but 9.6.1 and i don't have the protection mode option

[reib3n]

I am not able to find this location : /usr/local/lib/python3.9/site-packages/pyads
Can some one help me, Im running HassIO
I do have: /usr/local/lib but no python folder

I got the same issue at first. I needed to use for the ssh connection the SSH & Web Terminal Addon with disabled protection mode. After that I could locate the directory

Unfortunately i dont have version 12.0.2. but 9.6.1 and i don't have the protection mode option

Its from the Community Addon Store. I couldnt use the "official" Addon and got the same issue as you with it.

[Jeroen45]

i have the addon installed, what next?
I still only see the /usr/local/lib and no more.
Do i need to login to a higher level, docker?

[jandeschuttere]

What is the output of python -m site and python -m site --user-site when you are logged in through ssh?

maybe a faster way is the next few lines, each line is after an enter. After the first one you will enter the interactive shell of python and each line will be prefixed with >>> so don't be alarmed

python
import pyads
print(pyads.__file__)
exit()

the print action will show you a location, in my current case this prints/usr/local/lib/python3.10/site-packages/pyads/__init__.py .. this could be different in your case.
More importantly, this tells you where the package is situated, in this case that means /usr/local/lib/python3.10/site-packages/pyads .. that is the directory you can use to move the working pyads into a home assistance version that has the corrupted pyads

[reib3n]

i have the addon installed, what next?
I still only see the /usr/local/lib and no more.
Do i need to login to a higher level, docker?

Do you have disabled the protection mode?
What core version are you running right now?

[Jeroen45]

Yes protection is disabled.
Home Assistant 2022.9.4
Supervisor 2022.08.6
Operating System 9.0
Frontend-versie: 20220907.2 - latest

[reib3n]

Yes protection is disabled.
Home Assistant 2022.9.4
Supervisor 2022.08.6
Operating System 9.0
Frontend-versie: 20220907.2 - latest

are you using the "docker" command to copy the file in the container (container name: homeassistant) like this?
docker cp /backup/pylib/pyads homeassistant:/usr/local/lib/python3.10/site-packages/

[Jeroen45]

When i try: python -m site
I get command not found

[Jeroen45]

What is the output of python -m site and python -m site --user-site when you are logged in through ssh?

I get : command not found
I am logged in thru the addon: ssh & WebTerminal

[reib3n]

@Jeroen45: have you tried to copy the files with the docker command within the homeassistant container I mentioned bevor?

[Jeroen45]

I am almost there to log in with HassOS SSH port 22222 Configurator to get to the root level

[alemonta]

@Jeroen45 I have the same issue, can't log with enough rights, so neither python commands nor python folder is available:
anyhow I fixed my setup with the docker commands just like @C0D3-IT was suggesting as originally posted by @jandeschuttere (by the way, thank you very much!)

[Jeroen45]

@Jeroen45 I have the same issue, can't log with enough rights, so neither python commands nor python folder is available: anyhow I fixed my setup with the docker commands just like @C0D3-IT was suggesting as originally posted by @jandeschuttere (by the way, thank you very much!)

I am not able to go outside the docker to the root, to do the docker commands just like @C0D3-IT and @jandeschuttere.
Still looking into this

[BartDurnez]

Can anyone else verify as well that I'm not having a fluke on my end?

Updatet my HA core to 2023 version and still not working.
Had to copy the old working folder to make everything working again, as described in this post

[gibii88]

Yes, i managed to kopie the files to a folder on my laptop, using Winscp. copied them back to the python3.10 folder. ADS is working now. Many many thanks to you guys...

@Jeroen45 could you explain how you connect via Winscp? I'm connected but i'm not able to find the python folder

[jandeschuttere]

Still doesn't work for me.

Thank you for checking, will check why mine isn't causing any issues and try to re-iterate on the possible solution that was identified. I'm not able to commit to this in the next few days however, I'll plan something in during next week 👍

[BeezLoisbert]

Hey guys,

are there any positive reports regarding the problem?

[alemonta]

Unfortunately, also with core 2023.3.0 still the same issue...

[BeezLoisbert]

Unfortunately i am not familiar with the development of home assistant.

Is there any information whether this problem has already been dealt with?

Unfortunately, 9 months have passed and the problem still exists :(

[alemonta]

Home Assistant 2023.4.5
Supervisor 2023.04.0
Operating System 10.0
Frontend 20230411.1 - latest

Still bugged.

[reib3n]

Hello guys,
as this Issue still persists, I now switched from ADS to Modbus communication with my CX8190.
It is ab bit more complex and slower as ADS but it works for my scenarios.

[jowies01]

Spinned up a new Home Assistant Yellow Deployment ( on version 2023.5.2 ) problem still exist and work-arround did not work because there was no version 2022.6.7 because the Yellow implementation only started on 2022.7.0

Any sugestion to a solution?

[jandeschuttere]

I expect all distribution types of Home Assistant are having pip as part of the command line (for those using docker, don't forget to first go into the running docker container). You could try pip install pyads, that should install it for you. After that was successful you can restart the Home Assistant instance and it should have registered.

I'm sorry to not be able to investigate the root problem further, time has not been, and is still not, on my side I'm afraid

[jowies01]

Pip is indeed installed but in the container there is no 'make' so installation of pyads was failing

In the mean time, I've installed an older version of homeassistant ( not yellow specific ) and did the quick & dirty trick... after restart it was working again.

[lgroen91]

Hi All,

Having the same issue while integrating my Beckhoff CX1010 into home assistant.
Finally i fixed this by putting the complete integration in Node Red.

I’ve used the node-red-contrib-ads-client for adding the ADS functionality to my Home Assistant.

Hopefully this will help some of you to get ADS running again.

Running:
Home Assistant 2023.6.3
Supervisor 2023.06.4
Operating System 10.3
RPI 3

[BeezLoisbert]

Hi All,

it's still not working ...

Unfortunately, I don't have enough programming skills to fix the problem at the source.

So I'll keep hoping for a miracle

Greets

[BeezLoisbert]

Hi All,

the problem still exists under Core Version 2023.9.1

Greets

[jarisinisalo]

For those looking for a (dirty) workaround the package from the working version is compatible it seems with the new release. Assuming you are using docker the following steps might help you out until this gets a fix:

Given: containerA is the name of your working container, containerB is the name of the new container with the faulty adslib.

docker cp containerA:/usr/local/lib/python3.9/site-packages/pyads /tmp
docker stop containerA
docker start containerB
docker cp /tmp/pyads containerB:/usr/local/lib/python3.10/site-packages/
docker restart containerB

If you are not using docker, the digest is basically.. take the pyads lib from the working version, copy it in the location that the new homeassistant build is using.

Is this workaround still working in python3.11? Have you guys tried? Can't seem to get it working anymore.

[SaWey]

Just making an update here to let you know I might have found the reason and solution.
See home-assistant/wheels#458 (comment) and stlehmann/pyads#363

The core ads library version will need to be bumped when the pull request has been merged.

[alemonta]

For those looking for a (dirty) workaround the package from the working version is compatible it seems with the new release. Assuming you are using docker the following steps might help you out until this gets a fix:
Given: containerA is the name of your working container, containerB is the name of the new container with the faulty adslib.

docker cp containerA:/usr/local/lib/python3.9/site-packages/pyads /tmp
docker stop containerA
docker start containerB
docker cp /tmp/pyads containerB:/usr/local/lib/python3.10/site-packages/
docker restart containerB

If you are not using docker, the digest is basically.. take the pyads lib from the working version, copy it in the location that the new homeassistant build is using.

Is this workaround still working in python3.11? Have you guys tried? Can't seem to get it working anymore.

Hi,
I confirm it still works with python 3.11, it's just matter of matching the destination folder accordingly:

docker cp /tmp/pyads containerB:/usr/local/lib/python3.11/site-packages/

A.

[BeezLoisbert]

Just making an update here to let you know I might have found the reason and solution. See home-assistant/wheels#458 (comment) and stlehmann/pyads#363

The core ads library version will need to be bumped when the pull request has been merged.

Hey,

that sounds really good!

Does anyone know how long it usually takes for the request to be implemented?

I wait happily :D

[gibii88]

Hi,
Is there any news about it? Do we know if it has already been released?

Thanks!

[alemonta]

Hi, Is there any news about it? Do we know if it has already been released?

Thanks!

at the moment nothing changed:
Core 2023.12.3
Supervisor 2023.12.0
Operating System 11.2 (RBPI4)
The issue is still there, the workaround described above (copying the old working pyads libs from python 3.9 into python 3.11 folders) is still valid.
🤷‍♂️

[jowies01]

Some news on the release?

[drindal82]

I really really need this to work. any news?

[KCU1976]

I really really need this to work. any news?

I "solved" the issue for me by switching to an Intel based pc (old NUC) so I don't have to patch every time. It has the added benefit of being a lot faster and not running out of ram when compiling esphome things .
Perhaps you can do the same?

[jandeschuttere]

SaWey is doing a great job at getting it through the process, the PR of the issue in the external package was merged. 2 days ago there was someone looking at trying to release it but was not yet able to do so, I would expect that a resolution is imminent. Great work @SaWey!

[drindal82]

SaWey is doing a great job at getting it through the process, the PR of the issue in the external package was merged. 2 days ago there was someone looking at trying to release it but was not yet able to do so, I would expect that a resolution is imminent. Great work @SaWey!

any news on a release date?

[gibii88]

I don't understand if the PR is approved or not..any update?

[kwiatkowskimarcinb]

It still doesn't work on the latest Home Assistant Core 2024.3.3 on Home Assistant Green Hardware.

@homeassistant - can we do something about it, as the root cause seems to be known:

HA has changed the way they manage dependencies, which caused the problem on most systems."

and ads is a core integration?

[alemonta]

News on this issue:
Allow for platform dependent builds
ADS Component error: Exception importing homeassistant.components.ads

[BartDurnez]

Issue fixed, this topic can be closed.
Working fine now with latest home assistant core build.

See Bump pyads to 3.4.0 #116934

Great work!