Could not find dependency: libpython3.8.so.1.0 #793

AnkS4 · 2022-07-18T13:12:40Z

AnkS4
Jul 18, 2022

Describe the bug
Could not find dependency: libpython3.8.so.1.0 while building the application

To Reproduce
Steps to reproduce the behavior:

$ briefcase build --no-docker

[speechtotext] Building AppImage...
linuxdeploy version 1-alpha (git commit ID 0ae13c0), GitHub actions build 95 built on 2022-07-18 10:27:47 UTC

-- Creating basic AppDir structure -- 
Creating directory /home/anks/beeware/speechtotext/linux/appimage/SpeechToText/SpeechToText.AppDir/usr/bin/ 
Creating directory /home/anks/beeware/speechtotext/linux/appimage/SpeechToText/SpeechToText.AppDir/usr/lib/ 
Creating directory /home/anks/beeware/speechtotext/linux/appimage/SpeechToText/SpeechToText.AppDir/usr/share/applications/ 
Creating directory /home/anks/beeware/speechtotext/linux/appimage/SpeechToText/SpeechToText.AppDir/usr/share/icons/hicolor/ 
Creating directory 
/home/anks/beeware/speechtotext/linux/appimage/SpeechToText/SpeechToText.AppDir/usr/share/icons/hicolor/16x16/apps/ 
Creating directory 
/home/anks/beeware/speechtotext/linux/appimage/SpeechToText/SpeechToText.AppDir/usr/share/icons/hicolor/32x32/apps/ 
Creating directory 
/home/anks/beeware/speechtotext/linux/appimage/SpeechToText/SpeechToText.AppDir/usr/share/icons/hicolor/64x64/apps/ 
Creating directory 
/home/anks/beeware/speechtotext/linux/appimage/SpeechToText/SpeechToText.AppDir/usr/share/icons/hicolor/128x128/apps/ 
Creating directory 
/home/anks/beeware/speechtotext/linux/appimage/SpeechToText/SpeechToText.AppDir/usr/share/icons/hicolor/256x256/apps/ 
Creating directory 
/home/anks/beeware/speechtotext/linux/appimage/SpeechToText/SpeechToText.AppDir/usr/share/icons/hicolor/scalable/apps/ 

-- Deploying dependencies for existing files in AppDir -- 
Deploying dependencies for ELF file 
/home/anks/beeware/speechtotext/linux/appimage/SpeechToText/SpeechToText.AppDir/usr/bin/python3.8 
ERROR: Could not find dependency: libpython3.8.so.1.0 
ERROR: Failed to deploy dependencies for existing files 
Building...

Error while building app speechtotext.

Expected behavior
AppImage creation without error

Environment:

Operating System: Manjaro, GNU/Linux
Python version: Python 3.8.5
Software versions:
- Briefcase: 0.3.8
- Toga:0.2.15

freakboy3742 · 2022-07-18T23:18:02Z

freakboy3742
Jul 18, 2022
Maintainer

(Converted to a discussion while we work the underlying cause of the problem).

The only reason that I can think this would happen would be an incomplete initial creation process - possibly due to a corrupted or interrupted briefcase create step. You may have received an error when that command ran. To confirm this:

Does your project contain a `linux/appimage/SpeechToText/SpeechToText.AppDir/usr/lib/python3.8 folder that contains the .py files and subdirectories associated with the Python standard library?
Does the file ~/.briefcase/support/Python-3.8-linux-x86_64-support.bN.tar.gz exist (for some value of N); and if so, is it a valid tarball (i.e., can you tar ztvf that file without error?)

If you delete the linux folder in your project, and re-run briefcase build --no-docker (note - don't run create first; build will cause the create step to be run as well), that should resolve the problem. If it doesn't, can you please provide the full .log file that was produced when briefcase build failed.

3 replies

AnkS4 Jul 19, 2022
Author

Does your project contain a `linux/appimage/SpeechToText/SpeechToText.AppDir/usr/lib/python3.8 folder that contains the .py files and subdirectories associated with the Python standard library?

Yes, it does.

Does the file ~/.briefcase/support/Python-3.8-linux-x86_64-support.bN.tar.gz exist (for some value of N); and if so, is it a valid tarball (i.e., can you tar ztvf that file without error?)

Yes, it can extract without errors.

Even after deleting linux folder & re-running briefcase build --no-docker, the same error persists.

I'm using conda as a virtual environment.

Here's full log:

https://gist.github.com/AnkS4/45e5af530183112deff796a8f2634bc3

freakboy3742 Jul 19, 2022
Maintainer

Ok - I can only assume that your support file is corrupted, but sufficiently complete that it can be unpacked. It should contain a usr/lib/libpython3.8.so.1.0

Could I get you to:

Provide the output of tar ztvf ~/.briefcase/support/Python-3.8-linux-x86_640-support.b6.tar.gz
Delete that tarball, and the linux folder in your project, and re-run briefcase build --no-docker again?

AnkS4 Jul 19, 2022
Author

https://gist.github.com/AnkS4/9897847a41e43a889234fe205b66b5f0

Same error.

Log:
https://gist.github.com/AnkS4/21233c22cfccda21dc799020f683a0e2

Edit:
There's Python-3.8-linux-x86_64-support.b5.tar.gz instead of Python-3.8-linux-x86_640-support.b6.tar.gz

rmartin16 · 2022-07-19T20:21:36Z

rmartin16
Jul 19, 2022
Collaborator

Repro'd on Ubuntu 18.04 with Python 3.10 (and Python 3.8 for that matter).

Possible workarounds at bottom of post.

Fresh `briefcase build -v --no-docker` (no `./linux` dir or briefcase dir):

Full Log: https://gist.github.com/rmartin16/52329d4c9a52cbab1c90a54ef3b174a4

Notably, the shared library is missing from the get-go:

❯ ldd ./linux/appimage/Hello\ World/Hello\ World.AppDir/usr/bin/python3.10
	linux-vdso.so.1 (0x00007fffecdf4000)
	libpython3.10.so.1.0 => not found
	libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f6bbf58a000)
	/lib64/ld-linux-x86-64.so.2 (0x00007f6bbfb7d000)

Python binary details for later:

❯ ls ./linux/appimage/Hello\ World/Hello\ World.AppDir/usr/bin/python3.10 
.rwxr-xr-x 26k user 12 Jun 20:55  ./linux/appimage/Hello World/Hello World.AppDir/usr/bin/python3.10

❯ ls ./linux/appimage/Hello\ World/Hello\ World.AppDir/usr/lib/libpython3.10.so.1.0 
.rwxr-xr-x 19M user 12 Jun 20:55  ./linux/appimage/Hello World/Hello World.AppDir/usr/lib/libpython3.10.so.1.0

Reverted to `Python-3.10-b1` (since `b2` is currently latest)

Full Log: https://gist.github.com/rmartin16/f388634d9f84d98e2fee4d02f5baa64c

This succeeds and produces the AppImage.

However, this Python binary is not linked to libpython.so:

❯ ldd ./linux/appimage/Hello\ World/Hello\ World.AppDir/usr/bin/python3.10 
	linux-vdso.so.1 (0x00007ffe629ec000)
	libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f239a0f1000)
	libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f2399eed000)
	libutil.so.1 => /lib/x86_64-linux-gnu/libutil.so.1 (0x00007f2399cea000)
	libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f239994c000)
	libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f239955b000)
	/lib64/ld-linux-x86-64.so.2 (0x00007f239b61f000)

And it is tremedously larger...as if it contains libpython.so:

❯ ls ./linux/appimage/Hello\ World/Hello\ World.AppDir/usr/bin/python3.10 
.rwxr-xr-x 20M user 19 Jul 15:31  ./linux/appimage/Hello World/Hello World.AppDir/usr/bin/python3.10

Running in Docker

Full Log: https://gist.github.com/rmartin16/ba92795dba32de7c1302ea3ba76b929d

The build appears to work in Docker with Python-3.10-b2 because linuxdeploy finds libpython.so from the python3.10-dev package that's installed when the image is built.
From log:

-- Deploying dependencies for existing files in AppDir --                                                                                               subprocess.py:406
Deploying dependencies for ELF file /app/appimage/Hello World/Hello World.AppDir/usr/bin/python3.10                                                     subprocess.py:406
Deploying shared library /usr/lib/x86_64-linux-gnu/libpython3.10.so.1.0                                                                                 subprocess.py:406
Deploying copyright files for file /usr/lib/x86_64-linux-gnu/libpython3.10.so.1.0                                                                       subprocess.py:406

Eventually, it actually copies this version of libpython.so in to our relevant usr/lib directory:

Copying file /usr/lib/x86_64-linux-gnu/libpython3.10.so.1.0 to /app/appimage/Hello World/Hello World.AppDir/usr/lib/libpython3.10.so.1.0                subprocess.py:406

Later on you see linuxdeploy patch usr/bin/python3.10 with usr/lib as a runtime path:

Setting rpath in ELF file /app/appimage/Hello World/Hello World.AppDir/usr/bin/python3.10 to $ORIGIN/../lib                                             subprocess.py:406

This seems like the reason this Python binary now works outside of Docker.

Available Workarounds I Could Think Of

Revert to previous Python support package

Add support_revision = "b4" (for Python 3.8) to pyproject.toml under [tool.briefcase.app.<appname>]

Include the location of the libpython.so in LD_LIBRARY_PATH; for example, a version of:

> export LD_LIBRARY_PATH="/path/to/linux/appimage/Hello World/Hello World.AppDir/usr/lib/:$LD_LIBRARY_PATH"

Install the relevant pythonX.Y-dev package

I didn't test this in my dev environment but it seems hopeful.

Hope this helps :)

10 replies

rmartin16 Jul 19, 2022
Collaborator

As a POC, these two patchelf statements got briefcase build --no-docker to work:

> ldd ./linux/appimage/Hello\ World/Hello\ World.AppDir/usr/bin/python3.10 
	linux-vdso.so.1 (0x00007ffd9ad7d000)
	libpython3.10.so.1.0 => not found
	libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fc65f7c0000)
	/lib64/ld-linux-x86-64.so.2 (0x00007fc65fdb3000)

> patchelf --set-rpath '$ORIGIN/../lib' ./linux/appimage/Hello\ World/Hello\ World.AppDir/usr/bin/python3.10 
warning: working around a Linux kernel bug by creating a hole of 2076672 bytes in ‘./linux/appimage/Hello World/Hello World.AppDir/usr/bin/python3.10’

> patchelf --set-rpath '$ORIGIN' ./linux/appimage/Hello\ World/Hello\ World.AppDir/usr/lib/libpython3.so

> ldd ./linux/appimage/Hello\ World/Hello\ World.AppDir/usr/bin/python3.10 
	linux-vdso.so.1 (0x00007ffe5c6a7000)
	libpython3.10.so.1.0 => /home/user/tmp/beeware/helloworld/./linux/appimage/Hello World/Hello World.AppDir/usr/bin/../lib/libpython3.10.so.1.0 (0x00007f1b86aba000)
	libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f1b866c9000)
	libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f1b864aa000)
	libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f1b862a6000)
	libutil.so.1 => /lib/x86_64-linux-gnu/libutil.so.1 (0x00007f1b860a3000)
	libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f1b85d05000)
	/lib64/ld-linux-x86-64.so.2 (0x00007f1b87271000)

I'm not necessarily advocating for this....just putting it out there in case keeping libpython.so linked is desired now or in the future.

prior art, fwiw.

freakboy3742 Jul 20, 2022
Maintainer

@rmartin16 Thanks for that investigation - it's really helpful. AFAICT, you're on the right track - the binaries are rpath-ed for a specific location, and as soon as the support package is relocated, that location is no longer accurate.

I've just pushed up beeware/Python-Linux-support#6 as a prospective fix; if people are happy with that, I'll push out new support packages.

AnkS4 Jul 20, 2022
Author

I did both:

Adding support_revision = "b4" (for Python 3.8) to pyproject.toml under [tool.briefcase.app.]
Including the location of the libpython.so in LD_LIBRARY_PATH; for example, a version of:

export LD_LIBRARY_PATH="/home/anks/beeware/speechtotext/linux/appimage/SpeechToText/SpeechToText.AppDir/usr/lib/:$LD_LIBRARY_PATH"

How do I install python dev package using conda?

I'm now getting:

ERROR: Could not find dependency: libffi.so.6 
ERROR: Failed to deploy dependencies for existing files 
>>> Return code: 1
Building...

Error while building app speechtotext.

I also tried combination of Python3.10.x & support_revision default & "b1".
On that, I'm getting:

ERROR: Could not find dependency: libgdbm.so.5 
ERROR: Failed to deploy dependencies for existing files

freakboy3742 Jul 20, 2022
Maintainer

@AnkS4 It's very difficult to follow the details you've provided, and what input is producing what results. However, if I'm understanding correctly, the errors you're seeing about libffi and libgdbm are caused by not having the dependencies in your Ubuntu 18.04 install that the support package requires. These packages are listed in the Dockerfile:

https://github.com/beeware/Python-Linux-support/blob/dev/Dockerfile#L10

This is the reason that we recommend using Docker to build Linux AppImages - we're able to guarantee the environment where the package is being built.

As for Conda - we don't currently support Conda in any form.

AnkS4 Jul 21, 2022
Author

I was able to make AppImage after using docker.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Could not find dependency: libpython3.8.so.1.0 #793

{{title}}

Replies: 2 comments 13 replies

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

{{title}}

{{title}}

Select a reply

Could not find dependency: libpython3.8.so.1.0 #793

AnkS4 Jul 18, 2022

Replies: 2 comments · 13 replies

freakboy3742 Jul 18, 2022 Maintainer

AnkS4 Jul 19, 2022 Author

freakboy3742 Jul 19, 2022 Maintainer

AnkS4 Jul 19, 2022 Author

rmartin16 Jul 19, 2022 Collaborator

Fresh briefcase build -v --no-docker (no ./linux dir or briefcase dir):

Reverted to Python-3.10-b1 (since b2 is currently latest)

Running in Docker

Available Workarounds I Could Think Of

rmartin16 Jul 19, 2022 Collaborator

freakboy3742 Jul 20, 2022 Maintainer

AnkS4 Jul 20, 2022 Author

freakboy3742 Jul 20, 2022 Maintainer

AnkS4 Jul 21, 2022 Author

AnkS4
Jul 18, 2022

Replies: 2 comments 13 replies

freakboy3742
Jul 18, 2022
Maintainer

AnkS4 Jul 19, 2022
Author

freakboy3742 Jul 19, 2022
Maintainer

AnkS4 Jul 19, 2022
Author

rmartin16
Jul 19, 2022
Collaborator

Fresh `briefcase build -v --no-docker` (no `./linux` dir or briefcase dir):

Reverted to `Python-3.10-b1` (since `b2` is currently latest)

rmartin16 Jul 19, 2022
Collaborator

freakboy3742 Jul 20, 2022
Maintainer

AnkS4 Jul 20, 2022
Author

freakboy3742 Jul 20, 2022
Maintainer

AnkS4 Jul 21, 2022
Author