[Bug]: Some packages cannot be built #20966

truboxl · 2024-07-28T05:21:46Z

Problem description

These packages are currently not buildable. Either because source URL is dead or some dependencies have since upgraded and not compatible or checksum has changed or some other build issues. Likely duplicate of existing issues.

Main packages

Root packages

avahi (3f7d10b)
frida (frida: revbump to rebuild #21236)
v4l-utils (367aa9d)

X11 packages

What steps will reproduce the bug?

./scripts/run-docker.sh ./build-package.sh -I -a aarch64 <package>

What is the expected behavior?

No response

System information

n/a

The text was updated successfully, but these errors were encountered:

Biswa96 · 2024-07-28T05:42:23Z

Are these related to the ndk r27 update? Or should these be fixed now?

truboxl · 2024-07-28T05:43:36Z

These issues found using NDK r26b

licy183 · 2024-07-28T06:46:08Z

lfortran needs llvm<=15. As it is statically linked against libllvm, so updating llvm doesn't break it.

As for pypy and pypy3, termux-docker image was updated to support DNS resolve, but I haven't updated the build script yet.

TomJo2000 · 2024-07-28T10:19:23Z

zrok is one of mine.
zls is a language server, which I sort of do the coordination for.
luvi is closely related enough to luv and neovim that I should probably take a look at it.

TomJo2000 · 2024-07-28T13:41:41Z

Finally got time to look into this.

zrok is fine, in fact it just autoupdated(6b6ce33)
I had to manually make a blocker issue for rust (Auto update failing for rust #20970),
which has been acting up due to running out of space, thus stopping anything in the auto update queue after it.
zls just seems to need to be updated, that is underway in (bump(main/zls): 0.13.0 #20973).
Since it has no maintainer currently I will also be taking on.
luvi I'll leave for @cattokomo to figure out.
From the build error it looks like a git submodule problem with zlib.

// [...]
Submodule path 'deps/pcre2': checked out '9a51f31da1c2d45e1f31d6a0d6b2f62975fed373'
remote: Total 0 (delta 0), reused 0 (delta 0), pack-reused 0
remote: Enumerating objects: 201, done.
remote: Counting objects: 100% (200/200), done.
remote: Compressing objects: 100% (93/93), done.
remote: Total 101 (delta 88), reused 14 (delta 5), pack-reused 0
Receiving objects: 100% (101/101), 17.81 KiB | 17.81 MiB/s, done.
Resolving deltas: 100% (88/88), completed with 83 local objects.
From https://github.com/madler/zlib
 * branch            51b7f2abdade71cd9bb0e7a373ef2610ec6f9daf -> FETCH_HEAD
Submodule path 'deps/zlib': checked out '51b7f2abdade71cd9bb0e7a373ef2610ec6f9daf'
~/.termux-build/luvi/cache
fatal: reference is not a tree: b85da58f2549a519486a7296f5b836a8f6a64880

Can probably be fixed by updating luvi to a newer commit.

TomJo2000 · 2024-07-28T13:56:51Z

Updated the tracker with links to each of the packages in question.

twaik · 2024-07-28T14:18:35Z

bionic-host package will be needed only in the case we use arm-hosted runners (which is pretty much hard because we should rebuild NDK for aarch64 host and build aarch64 docker image for builder) so we can safely disable it (#20958).

TomJo2000 · 2024-07-28T14:24:26Z

This might not be the place to kick off this discussion.
But regarding python2, what if we just, drop it.

It's been EOL since 2020.
I think 4 and a half years is a good point to finally drop it.
The only packages still depending on it are:

qt5-qtwebengine in a very transitive way.
e.g. only the host python2 package.

termux-packages/x11-packages/qt5-qtwebengine/build.sh

Line 20 in 806b424

    
           PATH="$PWD/fake-bin:$PATH" python2 $TERMUX_PKG_SRCDIR/src/3rdparty/chromium/third_party/ffmpeg/chromium/scripts/build_ffmpeg.py --config-only linux noasm-ia32

libduktape

termux-packages/packages/libduktape/build.sh

Lines 12 to 17 in 806b424

    
           termux_step_pre_configure() { 
        
           	# configure.py requires 'yaml' python2 module. 
        
           	if ! pip2 show pyyaml > /dev/null 2>&1; then 
        
           		pip2 install pyyaml 
        
           	fi 
        
           }

libglade

termux-packages/x11-packages/libglade/build.sh

Lines 12 to 13 in 806b424

# For libglade-convert.

TERMUX_PKG_SUGGESTS="python2"
pypy{3,}
Really not sure how strongly those depend on Python 2.

twaik · 2024-07-28T14:30:14Z

BTW, about termux-docker. We can use bionic-host to avoid using prebuilt binary blobs there. I mean we still use binary blobs, but in this case we will be able to rebuild them for fixing some docker-related stuff or to emulate some Android behaviour on non-Android OS.

TomJo2000 · 2024-07-28T14:33:18Z

I'm looking at bsd-finger right now.
I think we should just give it the netcat-openbsd treatment, and use the Debian Sid source mirror as the source URL.

TomJo2000 · 2024-07-28T15:23:23Z

Look like eltclsh is a late victim of #20867
The expected license file name changed from LICENSE to copyright,
so the license file wasn't getting picked up.

termux-packages/packages/eltclsh/build.sh

Line 18 in 191c936

    
           sed -ne '/Copyright/,/ADVISED OF THE POSSIBILITY OF SUCH DAMAGE./s%^# %%p' $TERMUX_PKG_SRCDIR/tcl/init.tcl > $TERMUX_PREFIX/share/doc/$TERMUX_PKG_NAME/LICENSE

Though I did run into a symbol error with it.

ERROR: ./lib/libeltclsh.so contains undefined symbols:
    28: 0000000000000000     0 NOTYPE  GLOBAL DEFAULT   UND mblen

TomJo2000 · 2024-07-29T13:37:09Z

lexter has been wiped off the face of the internet.
https://repology.org/project/lexter/information

We should probably retire the package.

licy183 · 2024-07-29T15:00:43Z

I can't reproduce the build failure in python2 and python-scipy.

TomJo2000 · 2024-07-29T15:15:44Z

I am getting a python-scipy build failure.

Traceback (most recent call last):
  File "/home/builder/.termux-build/python-crossenv-prefix-aarch64/build/lib/python3.11/site-packages/mesonbuild/mesonmain.py", line 188, in run
    return options.run_func(options)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/builder/.termux-build/python-crossenv-prefix-aarch64/build/lib/python3.11/site-packages/mesonbuild/msetup.py", line 364, in run
    app.generate()
  File "/home/builder/.termux-build/python-crossenv-prefix-aarch64/build/lib/python3.11/site-packages/mesonbuild/msetup.py", line 187, in generate
    return self._generate(env, capture, vslite_ctx)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/builder/.termux-build/python-crossenv-prefix-aarch64/build/lib/python3.11/site-packages/mesonbuild/msetup.py", line 252, in _generate
    captured_compile_args = intr.backend.generate(capture, vslite_ctx)
                            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/builder/.termux-build/python-crossenv-prefix-aarch64/build/lib/python3.11/site-packages/mesonbuild/backend/ninjabackend.py", line 649, in generate
    self.generate_target(t)
  File "/home/builder/.termux-build/python-crossenv-prefix-aarch64/build/lib/python3.11/site-packages/mesonbuild/backend/ninjabackend.py", line 1072, in generate_target
    self.generate_dependency_scan_target(target, compiled_sources, source2object, generated_source_files, fortran_order_deps)
  File "/home/builder/.termux-build/python-crossenv-prefix-aarch64/build/lib/python3.11/site-packages/mesonbuild/backend/ninjabackend.py", line 1125, in generate_dependency_scan_target
    write = old != scaninfo
            ^^^^^^^^^^^^^^^
  File "<string>", line 4, in __eq__
AttributeError: 'TargetDependencyScannerInfo' object has no attribute 'sources'

ERROR: Unhandled python exception

    This is a Meson bug and should be reported!

ERROR Backend subprocess exited when trying to invoke build_wheel

But python2 builds just fine.

TomJo2000 · 2024-07-29T15:18:07Z

Oh, nvm python2 gives me a symbol check failure on arm and i686

ERROR: ./lib/python2.7/lib-dynload/audioop.so contains undefined symbols:
    15: 00000000     0 NOTYPE  GLOBAL DEFAULT   UND floor

floor is probably in -lm

TomJo2000 · 2024-07-31T12:57:34Z

I think I have another package to add to the list.

libmdbx
Was looking at it in the process of working on fix(scripts/termux_step_install_license): fix logic error #21005.
But I can't even get it to clone the repo.
Al I get is:

fatal: unable to access 'https://gitflic.ru/project/erthink/libmdbx.git/': The requested URL returned error: 403

I have a suspicion the hosting service may be region locked to refuse IPs from NATO countries.
We may wanna consider dropping the package if it cannot be rebuilt or updated.
CC: @twaik you were the last person to work on it (6b6b5f8), got any insights?

The IP blocking theory is gaining ground.
I can successfully clone the repo if I set my VPN to Serbia.
But it consistenty denies me with a HTTP 403 code from my IP in Germany.

Biswa96 · 2024-08-01T17:21:06Z

Building with the r27 PR I have only run into issues for busybox + a lot of rust packages. Most rust packages fail because the dependency crate time cannot be built:

It has been fixed in upstream time-rs/time#681. The time crate need to be updated in the projects.

agnostic-apollo · 2024-08-02T02:22:02Z

Seems like sourceforge have broken their download API and links now do multiple redirections to a mirror. Redirections are not done by curl in termux_download() for obvious security reasons. Lot of packages are affected.

Currently, packages are using two formats:

https://sourceforge.net/projects/*: Mostly hangs indefinitely if --output is passed, even if -L is passed to curl. Their commandline download docs say url should end with /download, but that doesn't work either.
https://downloads.sourceforge.net/*: Works only if -L is passed to curl. (Used by fedora too)

A solution that doesn't use -L is to find the final url for the mirror in the format https://deac-fra.dl.sourceforge.net/project/<org_name>/<project_name>/<filename>?viasf=1. The ?viasf=1 url parameter is necessary as otherwise a final redirection will be done to it. The deac-fra is the DEAC (Frankfurt Am Main, Germany) mirror short name listed at https://sourceforge.net/p/forge/documentation/Mirrors/ and files may not be available on all mirrors. When you normally download a file from sourceforge, there is a button for Problem Downloading?, click that and it should show a list of mirrors, although sometimes not all our listed, even though files are available on the mirror. You can open Web Developer Tools in Firefox and in the Network tab, it should show all the redirections before file is downloaded.

For example:

For procps package https://sourceforge.net/projects/procps-ng/files/Production/procps-ng-$TERMUX_PKG_VERSION.tar.xz needs to be replaced with https://deac-fra.dl.sourceforge.net/project/procps-ng/Production/procps-ng-$TERMUX_PKG_VERSION.tar.xz?viasf=1
For nfs-utils package https://downloads.sourceforge.net/nfs/nfs-utils-${TERMUX_PKG_VERSION}.tar.bz2 needs to be replaced with https://deac-fra.dl.sourceforge.net/project/nfs/nfs-utils/2.6.4/nfs-utils-${TERMUX_PKG_VERSION}.tar.bz2?viasf=1

$ curl https://sourceforge.net/projects/procps-ng/files/Production/procps-ng-3.3.17.tar.xz
<html>
<head>
<title>301 Moved Permanently</title>
</head>
<body>
<h1>301 Moved Permanently</h1>
The resource has been moved to <a href="https://sourceforge.net/projects/procps-ng/files/Production/procps-ng-3.3.17.tar.xz/">https://sourceforge.net/projects/procps-ng/files/Production/procps-ng-3.3.17.tar.xz/</a>;
you should be redirected automatically.
</body>

$ curl https://downloads.sourceforge.net/nfs/nfs-utils-2.6.4.tar.bz2
<html>
 <head>
  <title>301 Moved Permanently</title>
 </head>
 <body>
  <h1>301 Moved Permanently</h1>
  The resource has been moved to <a href="https://downloads.sourceforge.net/project/nfs/nfs-utils/2.6.4/nfs-utils-2.6.4.tar.bz2">https://downloads.sourceforge.net/project/nfs/nfs-utils/2.6.4/nfs-utils-2.6.4.tar.bz2</a>;
you should be redirected automatically.
 </body>

$ curl https://sourceforge.net/projects/opencore-amr/files/opencore-amr/opencore-amr-0.1.6.tar.gz/download 
<html>
<head>
<title>302 Found</title>
</head>
<body>
<h1>302 Found</h1>
The resource was found at <a href="https://downloads.sourceforge.net/project/opencore-amr/opencore-amr/opencore-amr-0.1.6.tar.gz?ts=gAAAAABmrEGD1Ya2KP8BvDxNyHNBMS-yoZ8WNCauHXcFIrTTBmdCyokMsnLpBHyMchM4JiwClE9Mbkh2uufTUpxjIa0qE4OVZw%3D%3D&amp;use_mirror=cyfuture&amp;r=">https://downloads.sourceforge.net/project/opencore-amr/opencore-amr/opencore-amr-0.1.6.tar.gz?ts=gAAAAABmrEGD1Ya2KP8BvDxNyHNBMS-yoZ8WNCauHXcFIrTTBmdCyokMsnLpBHyMchM4JiwClE9Mbkh2uufTUpxjIa0qE4OVZw%3D%3D&amp;use_mirror=cyfuture&amp;r=</a>;
you should be redirected automatically.
</body>

Btw, procps has moved to gitlab at https://gitlab.com/procps-ng/procps/-/releases but only 3.3.16 and 4.0.0 are available, and not 3.3.17 currently being used.

Maybe sourceforge should be pinged before making changes. Seems like similar issues for currently used urls have been reported for like last 10 years.

agnostic-apollo · 2024-08-02T02:23:23Z

Additionally to us hosting a packages sources repo, there should be some biweekly/monthly workflow that downloads all sources of all packages too see if their urls and checksums are still valid.

twaik · 2024-08-12T18:24:45Z

Is there any reason I can not revbump pypy? It seems like builder tries to execute cross-compiled binary

[platform:execute] Cross Exec (termux-i686): /home/builder/.termux-build/pypy/src/usession-dir/usession-release-pypy2.7-v7.3.15-0/gcctest
Limited by the size of pthread_mutex_t, 32 bit bionic libc only accepts pid <= 65535, but current pid is 121353
libc: Limited by the size of pthread_mutex_t, 32 bit bionic libc only accepts pid <= 65535, but current pid is 121353

But I did not see it in earlier build attempts.
https://github.com/termux/termux-packages/actions/runs/10356856078/job/28667617880#step:8:19487

sylirre · 2024-08-12T22:11:37Z

Is there any reason I can not revbump pypy?

Intermittent issue with PID value. Re-run of the job may fix it.

TomJo2000 · 2024-08-12T22:50:17Z

@TomJo2000 https://github.com/pypy/pypy/blob/main/rpython/doc/faq.rst#why-does-the-translator-draw-a-mandelbrot-fractal-while-translating

Because it's fun.
You know what, fair enough.

truboxl · 2024-08-13T05:14:42Z

This is why I strongly oppose using bionic-host + qemu static binary because of Android libc limitation

sylirre · 2024-08-13T08:11:15Z

@truboxl Isn't particularly this limit (max pid) is common for all 32bit Linux systems, not just Android?

twaik · 2024-08-13T09:09:48Z

This is why I strongly oppose using bionic-host + qemu static binary because of Android libc limitation

This error is easy to fix with echo 32767 > /proc/sys/kernel/max_pid. I was wondering it uses bionic binaries during build.

TomJo2000 · 2024-08-13T09:20:19Z

I'm pretty sure some package builds churn through more than 2^15 PIDs.

twaik · 2024-08-13T10:35:25Z

When PID counter hits the limit it is being reset to the first unoccupied PID number above 1, and I am pretty much sure there will never be 32767 simultaneously running processes.

twaik · 2024-08-13T10:43:28Z

ARM revbump of gdbm's revdeps seems to be fine, but hit the CI time limit, probably because of running bionic executables with qemu (we already had this with glib-introspection). So I am reverting qemu integration.

twaik · 2024-08-14T07:46:29Z

It seems like Audacity is not in the list (related to #18790).

TomJo2000 · 2024-08-15T06:13:04Z

I'll quickly check what's already covered here so I can mark it as returning for the NDK 27 pass.

TomJo2000 · 2024-08-17T16:30:13Z

Could I get a spot check on txikijs?
It seems to be working again with NDK 27.

truboxl · 2024-09-09T06:46:37Z

Closing since most of the packages already can be rebuilt. Those that dont probably broken for a long time and will be tracked in #21130.

twaik · 2024-10-15T13:59:15Z

I think I have another package to add to the list.

libmdbx
Was looking at it in the process of working on fix(scripts/termux_step_install_license): fix logic error #21005.
But I can't even get it to clone the repo.
Al I get is:
fatal: unable to access 'https://gitflic.ru/project/erthink/libmdbx.git/': The requested URL returned error: 403
I have a suspicion the hosting service may be region locked to refuse IPs from NATO countries. We may wanna consider dropping the package if it cannot be rebuilt or updated. CC: @twaik you were the last person to work on it (6b6b5f8), got any insights?

The IP blocking theory is gaining ground. I can successfully clone the repo if I set my VPN to Serbia. But it consistenty denies me with a HTTP 403 code from my IP in Germany.

libmdbx is updated successfully in 3cf9b3c.

truboxl added bug report Something is not working properly untriaged labels Jul 28, 2024

licy183 added tracker and removed untriaged labels Jul 28, 2024

TomJo2000 mentioned this issue Jul 28, 2024

chore(main/bsd-finger): fix dead source and homepage links #20975

Merged

mbekkomo mentioned this issue Jul 29, 2024

fix(main/luvi): fix build #20990

Merged

This was referenced Jul 29, 2024

fix(main/icecast): explicitly include libxml2 #20991

Merged

fix(main/moria): specify license file #20992

Merged

fix(main/kibi): explicitly specify license file #20993

Merged

This was referenced Jul 29, 2024

fix(main/ol): specify license files #20994

Merged

fix(main/rc): rename from rcshell and replace dead source #20995

Merged

TomJo2000 mentioned this issue Jul 29, 2024

bump(main/simh): 3.12-5 #20996

Merged

TomJo2000 added packaging Issue related to building packages, not affecting end users directly help wanted Help is wanted in order to solve the issue labels Jul 30, 2024

licy183 mentioned this issue Aug 2, 2024

pypy: fix build #21026

Merged

This was referenced Aug 4, 2024

Auto update failing for taskwarrior #21039

Closed

fix(main/pypy3): Permission denied #21050

Closed

TomJo2000 pinned this issue Aug 10, 2024

This comment was marked as off-topic.

Sign in to view

TomJo2000 mentioned this issue Aug 13, 2024

[Bug]: Failed to build package 'coreutils' #21113

Closed

TomJo2000 mentioned this issue Aug 15, 2024

Some packages cannot be built 2: Electric boogaloo #21130

Open

truboxl added the has build issues Package compilation fails label Aug 20, 2024

truboxl closed this as completed Sep 9, 2024

truboxl unpinned this issue Sep 9, 2024

termux deleted a comment from FWNABRANW Sep 10, 2024

termux locked as resolved and limited conversation to collaborators Oct 14, 2024

[Bug]: Some packages cannot be built #20966

[Bug]: Some packages cannot be built #20966

Comments

truboxl commented Jul 28, 2024 • edited by TomJo2000 Loading

Problem description

What steps will reproduce the bug?

What is the expected behavior?

System information

Biswa96 commented Jul 28, 2024

truboxl commented Jul 28, 2024

licy183 commented Jul 28, 2024 • edited Loading

TomJo2000 commented Jul 28, 2024

TomJo2000 commented Jul 28, 2024

TomJo2000 commented Jul 28, 2024

twaik commented Jul 28, 2024

TomJo2000 commented Jul 28, 2024

twaik commented Jul 28, 2024

TomJo2000 commented Jul 28, 2024

TomJo2000 commented Jul 28, 2024

TomJo2000 commented Jul 29, 2024

licy183 commented Jul 29, 2024

TomJo2000 commented Jul 29, 2024

TomJo2000 commented Jul 29, 2024 • edited Loading

TomJo2000 commented Jul 31, 2024 • edited Loading

Biswa96 commented Aug 1, 2024

agnostic-apollo commented Aug 2, 2024

agnostic-apollo commented Aug 2, 2024

twaik commented Aug 12, 2024 • edited Loading

This comment was marked as off-topic.

This comment was marked as off-topic.

sylirre commented Aug 12, 2024

TomJo2000 commented Aug 12, 2024

truboxl commented Aug 13, 2024

sylirre commented Aug 13, 2024

twaik commented Aug 13, 2024 • edited Loading

TomJo2000 commented Aug 13, 2024

twaik commented Aug 13, 2024

twaik commented Aug 13, 2024

twaik commented Aug 14, 2024

TomJo2000 commented Aug 15, 2024

TomJo2000 commented Aug 17, 2024

truboxl commented Sep 9, 2024

twaik commented Oct 15, 2024

truboxl commented Jul 28, 2024 •

edited by TomJo2000

Loading

licy183 commented Jul 28, 2024 •

edited

Loading

TomJo2000 commented Jul 29, 2024 •

edited

Loading

TomJo2000 commented Jul 31, 2024 •

edited

Loading

twaik commented Aug 12, 2024 •

edited

Loading

twaik commented Aug 13, 2024 •

edited

Loading