pkg-config, installation script

[markuskimius]

Hi there,

I noticed this project has transitioned to using cargo instead of autoconf, and as a part of that transition it no longer produces the pkg-config file (*.pc), nor a script to install *.pc, *.h, and the libraries. Are these deprecated or will they be making a comeback?

We have a C-based project, Tux Paint, that uses libimagequant that expects it to work with pkg-config. I was wondering if we need to make changes to Tux Paint or we can just wait for the libimagequant project to make it available again.

Thank you

[kornelski]

I'm working on bringing these back.

Are you using pkg-config via a Linux distro or directly from the repo?

[markuskimius]

Sweet. Thank you.

I apologize. Upon review of the Tux Paint Makefile it appears we're not using pkg-config with libimagequant, but just linking with the -limagequant option. So just the install script would work for us. I'm not sure why we're not using pkg-config with libimagequant; we do with most other libraries.

To give you more information, we run on a few different OS's. I don't work on the Linux builds myself but I believe the two distributions we build for (Debian and RedHat) have libimagequant available as a distribution package so we just use those.

Our port to macOS builds libimagequant from the source and links with the -limagequant option (without using pkg-config, contrary to what I had thought in the earlier post). To confirm, the macOS port does not build using a Xcode project, but is built "like Linux" (make && make install). It gets any library it needs from the MacPorts project, but libimagequant isn't available from there so it is built from source and installed where MacPorts libraries are installed by default, /opt/local.

I believe our ports to Windows and Haiku also build libimagequant from the source. They link to it using the -limagequant option like the macOS port, at the very least. We also have a port to Android; I'm pretty sure it also builds libimagequant from the source but I'm not sure how it links to libimagequant since I'm not too familiar with the Android port.

I hope some of that information is useful if not confusing. Please let me know if I can provide more information. Thank you again!

[kornelski]

I've added docs, and brought the Makefile back. Does that work for you?

https://github.com/ImageOptim/libimagequant/tree/main#upgrading-instructions

[markuskimius]

Hi Kornel,

Unfortunately the issue appears to be bigger than I had thought. I had previously thought we just needed an install script, but there also appears to be issues with the symbols in the library. Additionally, a new issue has also been introduced since yesterday with the build process.

First, the new build process issue: cargo build --release is no longer building the static library. It was building without errors yesterday. It appears this change introduced the error. The error I get is:

$ cargo build --release
...
   Compiling imagequant v4.0.0-beta.4 (/Users/mark/src/libimagequant)
warning: field is never read: `pixels`
  --> src/rows.rs:10:48
   |
10 |     Pixels { rows: SeaCow<'rows, *const RGBA>, pixels: Option<SeaCow<'pixels, RGBA>> },
   |                                                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
   |
   = note: `#[warn(dead_code)]` on by default

warning: variant is never constructed: `Borrowed`
   --> src/seacow.rs:122:5
    |
122 |     Borrowed(&'a mut T),
    |     ^^^^^^^^^^^^^^^^^^^

warning: `imagequant` (lib) generated 2 warnings
    Finished release [optimized] target(s) in 1.82s
$

If I switch to the commit just prior then cargo build --release, the static library builds without errors.

When I copy the built files to /opt/local/{include,lib}, Tux Paint does see the files but the linking fails due to bad symbols:

$ make
...
Undefined symbols for architecture arm64:
  "_liq_attr_create", referenced from:
      _do_slideshow in tuxpaint.o
  "_liq_get_palette", referenced from:
      _do_slideshow in tuxpaint.o
  "_liq_image_create_rgba", referenced from:
      _do_slideshow in tuxpaint.o
  "_liq_image_quantize", referenced from:
      _do_slideshow in tuxpaint.o
  "_liq_set_dithering_level", referenced from:
      _do_slideshow in tuxpaint.o
  "_liq_set_max_colors", referenced from:
      _do_slideshow in tuxpaint.o
  "_liq_write_remapped_image", referenced from:
      _do_slideshow in tuxpaint.o
ld: symbol(s) not found for architecture arm64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make: *** [tuxpaint] Error 1
$ 

I get the same error with the latest commit on the main branch after building the files using the Makefile under imagequant-sys (make && sudo make PREFIX=/opt/local install)

It looks like the static library does have the symbols Tux Paint needs, but the library appears to be referencing some rust symbols that's missing. Would it be possible to link these symbols into the static library?

$ nm -gU /opt/local/lib/libimagequant.a | grep -w _liq_attr_create
000000000000250c T _liq_attr_create
/opt/local/lib/libimagequant.a:addr2line-6310e2c569be07d8.addr2line.61ef25ac-cgu.9.rcgu.o: no symbols
/opt/local/lib/libimagequant.a:rustc_demangle-e53e6633e50c3ad1.rustc_demangle.e68410d5-cgu.12.rcgu.o: no symbols
/opt/local/lib/libimagequant.a:rustc_std_workspace_alloc-5aa63f595a685d7d.rustc_std_workspace_alloc.c85b64ad-cgu.0.rcgu.o: no symbols
/opt/local/lib/libimagequant.a:cfg_if-54b116f513fe2e08.cfg_if.89dcec8f-cgu.0.rcgu.o: no symbols
/opt/local/lib/libimagequant.a:rustc_std_workspace_core-da8f7b2bb8776124.rustc_std_workspace_core.7ac3b843-cgu.0.rcgu.o: no symbols
$ 

Finally, attempting to build and install using cargo-c fails outright:

$ sudo cargo cinstall --prefix=/opt/local --destdir=.
Error: CliError { error: Some(Package `imagequant v4.0.0-beta.4 (/Users/mark/src/libimagequant)` does not have the feature `capi`), exit_code: 101 }
$

Thank you for looking into our errors. I really appreciate it!

Regards,
Mark

[kornelski]

Sorry for the confusion. I've moved the C library to a folder: https://github.com/ImageOptim/libimagequant/tree/main/imagequant-sys

[markuskimius]

Thank you. cargo build --release works in imagequant-sys!

The built library is till failing to link, however. The symbol issue appears to remain:

$ nm -gU target/release/libimagequant.a >/dev/null
target/release/libimagequant.a:addr2line-6310e2c569be07d8.addr2line.61ef25ac-cgu.9.rcgu.o: no symbols
target/release/libimagequant.a:rustc_demangle-e53e6633e50c3ad1.rustc_demangle.e68410d5-cgu.12.rcgu.o: no symbols
target/release/libimagequant.a:rustc_std_workspace_alloc-5aa63f595a685d7d.rustc_std_workspace_alloc.c85b64ad-cgu.0.rcgu.o: no symbols
target/release/libimagequant.a:cfg_if-54b116f513fe2e08.cfg_if.89dcec8f-cgu.0.rcgu.o: no symbols
target/release/libimagequant.a:rustc_std_workspace_core-da8f7b2bb8776124.rustc_std_workspace_core.7ac3b843-cgu.0.rcgu.o: no symbols
$

And Tux Paint continues to fail to build:

$ make
...

...Linking Tux Paint...
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang  -O2 -W -Wall -fno-common -ffloat-store -fvisibility=hidden -Wcast-align -Wredundant-decls -Wbad-function-cast -Wwrite-strings -Waggregate-return -Wstrict-prototypes -Wmissing-prototypes -Wstrict-aliasing=2 -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk -I/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include -I/opt/local/include -mmacosx-version-min=10.8 -arch arm64 -w -headerpad_max_install_names -DHAVE_STRCASESTR -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk -L/opt/local/lib -mmacosx-version-min=10.8 -arch arm64  -D_GNU_SOURCE=1 -D_THREAD_SAFE -I/opt/local/include/SDL -I/opt/local/include/fribidi -DVER_DATE=\"2022-01-09\" -DVER_VERSION=\"0.9.28\" -DDATA_PREFIX=\"Resources/share/tuxpaint/\" -DDOC_PREFIX=\"Resources/share/doc/tuxpaint-0.9.28/\" -DLOCALEDIR=\"Resources/share/locale/\" -DIMDIR=\"Resources/share/tuxpaint/im/\" -DCONFDIR=\"Resources/etc/tuxpaint/\" -DMAGIC_PREFIX=\"Resources/lib/tuxpaint/plugins/\"      \
		-o tuxpaint obj/tuxpaint.o obj/i18n.o obj/im.o obj/cursor.o obj/pixels.o obj/rgblinear.o obj/playsound.o obj/fonts.o obj/parse.o obj/fill.o obj/progressbar.o obj/dirwalk.o obj/get_fname.o obj/onscreen_keyboard.o obj/gifenc.o obj/sounds.o src/macos_print.m obj/macos.o \
		-L/opt/local/lib -lSDLmain -Wl,-framework,AppKit -lSDL -Wl,-framework,Cocoa -L/opt/local/lib -lSDL_image -lSDLmain -Wl,-framework,AppKit -lSDL -Wl,-framework,Cocoa -L/opt/local/lib -lSDL_ttf -lSDLmain -Wl,-framework,AppKit -lSDL -Wl,-framework,Cocoa -L/opt/local/lib -lSDL_gfx -lSDLmain -Wl,-framework,AppKit -lSDL -Wl,-framework,Cocoa -L/opt/local/lib -lz -L/opt/local/lib -lpng16 -L/opt/local/lib -lSDL_mixer -lSDLmain -Wl,-framework,AppKit -lSDL -Wl,-framework,Cocoa -L/opt/local/lib -lSDL_Pango -lpango-1.0 -lgobject-2.0 -lglib-2.0 -lintl -lharfbuzz -L/opt/local/lib -lrsvg-2 -lm -lgio-2.0 -lgdk_pixbuf-2.0 -lgobject-2.0 -lglib-2.0 -lintl -lcairo -L/opt/local/lib -lfribidi -limagequant
Undefined symbols for architecture arm64:
  "_liq_attr_create", referenced from:
      _do_slideshow in tuxpaint.o
  "_liq_get_palette", referenced from:
      _do_slideshow in tuxpaint.o
  "_liq_image_create_rgba", referenced from:
      _do_slideshow in tuxpaint.o
  "_liq_image_quantize", referenced from:
      _do_slideshow in tuxpaint.o
  "_liq_set_dithering_level", referenced from:
      _do_slideshow in tuxpaint.o
  "_liq_set_max_colors", referenced from:
      _do_slideshow in tuxpaint.o
  "_liq_write_remapped_image", referenced from:
      _do_slideshow in tuxpaint.o
ld: symbol(s) not found for architecture arm64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make: *** [tuxpaint] Error 1
$

[kornelski]

Are you specifying -L for the correct dir for the new build of libimagequant.a? In the linker invocation I see -limagequant, but not an -L for it.

[kornelski]

Don't worry about the "no symbols" error. That may be from Rust dependencies that don't have C symbols (are inline, generic, or macros only).

[markuskimius]

Yes, a proper -L option is being passed. libimagequant.a was copied to /opt/local/lib so -L/opt/local/lib should be picking up libimagequant.a:

$ ls -l /opt/local/lib/libimagequant.a
-rw-r--r-- 1 mark staff 12641360 Jan  9 18:37 /opt/local/lib/libimagequant.a
$ 

[kornelski]

I can't reproduce. I think you have copied wrong/old version of the file. The file size I get on M1 Mac is 17185824 bytes.

test.c:

#include <stdio.h>
#include "libimagequant/imagequant-sys/libimagequant.h"

int main(int argc, char const *argv[])
{
    printf("%d\n", liq_version());
}

git clone https://github.com/ImageOptim/libimagequant
( cd libimagequant/imagequant-sys/; cargo build --release )
# this works
gcc test.c libimagequant/imagequant-sys/target/release/libimagequant.a
# this works too
gcc test.c -Llibimagequant/imagequant-sys/target/release/ -limagequant

[markuskimius]

Thank you for that. Yes, your code works on my setup as well.

Apparently what's happening is gcc picks up the shared library instead of the static one if they are in the same directory, and it is shared library that is having the problem. make PREFIX=/opt/local install installs both files to the same directory, /opt/local/lib so Tux Paint couldn't compile by referencing /opt/local/lib. But libimagequant/imagequant-sys/target/release/ only has the static library so the test code runs successfully.

If I delete /opt/local/lib/libimagequant*dylib, Tux Paint compiles fine without errors.

It looks like libimagequant/imagequant-sys has both the static and the dynamic libraries after make runs so attempting to compile the test program by passing -L with a reference to that directory, instead of to libimagequant/imagequant-sys/target/release`, fails for me. Is this error reproducible on your end?

$ gcc test.c -L libimagequant/imagequant-sys -limagequant
Undefined symbols for architecture arm64:
  "_liq_version", referenced from:
      _main in test-b9be17.o
ld: symbol(s) not found for architecture arm64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
$ 

It appears the shared library does not have any symbols:

$ nm -gU libimagequant/imagequant-sys/libimagequant.dylib 
libimagequant/imagequant-sys/libimagequant.dylib: no symbols
$ 

[kornelski]

Thank you for diagnosing this. You're right — re-export of symbols for dylib doesn't work ;(

[kornelski]

So it seems the problem is not easy to solve. In that case I recommend:

1. use pkg-config to find an existing instance of the library, if there is any
2. otherwise always build and link static library (which cargo build --release does), and don't try building a dynamic library.

[markuskimius]

That works for us. Tux Paint can just use the static library.

Would it be possible to get the install target in Makefile to just install the static library (and the header and pkg-config file)? That target was very useful (though we can just copy manually if needed.)

[kornelski]

Yes, I can make it install a static lib.

[markuskimius]

Fantastic. Thank you!