jemalloc-sys error : could not find native static library jemalloc

[wangjia184]

Compile on Linux targeting windows.

cargo build --target x86_64-pc-windows-gnu --release --verbose

Running `rustc --crate-name jemalloc_sys /usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/jemalloc-sys-0.3.2/src/lib.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts --crate-type lib --emit=dep-info,metadata,link -C opt-level=3 --cfg 'feature="background_threads_runtime_support"' -C metadata=b6586f656434efdb -C extra-filename=-b6586f656434efdb --out-dir /builds/bonus/ubs.reverseproxy/target/x86_64-pc-windows-gnu/release/deps --target x86_64-pc-windows-gnu -L dependency=/builds/bonus/ubs.reverseproxy/target/x86_64-pc-windows-gnu/release/deps -L dependency=/builds/bonus/ubs.reverseproxy/target/release/deps --extern libc=/builds/bonus/ubs.reverseproxy/target/x86_64-pc-windows-gnu/release/deps/liblibc-4505f84397ff6c74.rmeta --cap-lints allow -L native=/builds/bonus/ubs.reverseproxy/target/x86_64-pc-windows-gnu/release/build/jemalloc-sys-111025ffa6cbfd48/out/build/lib --cfg prefixed -l static=jemalloc`
 error: could not find native static library `jemalloc`, perhaps an -L flag is missing?
 error: aborting due to previous error
 error: could not compile `jemalloc-sys`.
 Caused by:
   process didn't exit successfully: `rustc --crate-name jemalloc_sys /usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/jemalloc-sys-0.3.2/src/lib.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts --crate-type lib --emit=dep-info,metadata,link -C opt-level=3 --cfg 'feature="background_threads_runtime_support"' -C metadata=b6586f656434efdb -C extra-filename=-b6586f656434efdb --out-dir /builds/bonus/ubs.reverseproxy/target/x86_64-pc-windows-gnu/release/deps --target x86_64-pc-windows-gnu -L dependency=/builds/bonus/ubs.reverseproxy/target/x86_64-pc-windows-gnu/release/deps -L dependency=/builds/bonus/ubs.reverseproxy/target/release/deps --extern libc=/builds/bonus/ubs.reverseproxy/target/x86_64-pc-windows-gnu/release/deps/liblibc-4505f84397ff6c74.rmeta --cap-lints allow -L native=/builds/bonus/ubs.reverseproxy/target/x86_64-pc-windows-gnu/release/build/jemalloc-sys-111025ffa6cbfd48/out/build/lib --cfg prefixed -l static=jemalloc` (exit code: 1)
 warning: build failed, waiting for other jobs to finish...
 error: build failed
 ERROR: Job failed: exit code 1

Here is the building environment dockerfile.

FROM rust:slim-buster


RUN apt-get update; \
    apt-get install -y mingw-w64 build-essential ; \
    apt-get clean; 

RUN rustup target add x86_64-pc-windows-gnu


RUN mkdir ~/.cargo/ && \
    printf "\n[target.x86_64-pc-windows-gnu]" >> ~/.cargo/config  && \
    printf "\nlinker = \"/usr/bin/x86_64-w64-mingw32-gcc\"" >> ~/.cargo/config && \
    printf "\nar = \"/usr/bin/x86_64-w64-mingw32-gcc-ar\"" >> ~/.cargo/config && \
    printf "\nnm = \"/usr/bin/x86_64-w64-mingw32-gcc-nm\"" >> ~/.cargo/config && \
    printf "\nranlib = \"/usr/bin/x86_64-w64-mingw32-gcc-ranlib\"" >> ~/.cargo/config && \
    printf "\ncrt-static = true" >> ~/.cargo/config 

VOLUME ["/var/run/docker.sock"]

Same error occurs if compiling with cross

[wangjia184]

I have worked around this issue, the basic idea is to build jemalloc manually

./configure --host=x86_64-w64-mingw32 --build=x86_64-unknown_linux-gnu --disable-cxx --disable-initial-exec-tls --with-private-namespace=_rjem_ --with-jemalloc-prefix=;
make;

Then create the static library manually.

ar crus /usr/local/lib/libjemalloc.a src/jemalloc.o src/arena.o src/background_thread.o src/base.o \
    src/bin.o src/bitmap.o src/ckh.o src/ctl.o src/div.o src/extent.o src/extent_dss.o src/extent_mmap.o \
    src/hash.o src/hook.o src/large.o src/log.o src/malloc_io.o src/mutex.o src/mutex_pool.o src/nstime.o \
    src/pages.o src/prng.o src/prof.o src/rtree.o src/safety_check.o src/stats.o src/sc.o src/sz.o src/tcache.o \
    src/test_hooks.o src/ticker.o src/tsd.o src/witness.o;

Next, set JEMALLOC_OVERRIDE=/usr/local/lib/libjemalloc.a environment

In this way, we can have Linux-based docker image running in CI to build windows binaries.

Here is full dockerfile

FROM rust:slim-buster


RUN apt-get update; \
    apt-get install -y mingw-w64 build-essential curl; \
    apt-get clean; 

RUN rustup target add x86_64-pc-windows-gnu


RUN mkdir ~/.cargo/ && \
    printf "\n[target.x86_64-pc-windows-gnu]" >> ~/.cargo/config  && \
    printf "\nlinker = \"/usr/bin/x86_64-w64-mingw32-gcc\"" >> ~/.cargo/config && \
    printf "\nar = \"/usr/bin/x86_64-w64-mingw32-gcc-ar\"" >> ~/.cargo/config && \
    printf "\nnm = \"/usr/bin/x86_64-w64-mingw32-gcc-nm\"" >> ~/.cargo/config && \
    printf "\nranlib = \"/usr/bin/x86_64-w64-mingw32-gcc-ranlib\"" >> ~/.cargo/config && \
    printf "\ncrt-static = true" >> ~/.cargo/config 

RUN curl -L https://github.com/jemalloc/jemalloc/releases/download/5.2.1/jemalloc-5.2.1.tar.bz2 --output jemalloc.tar.bz2; \
    tar -xvjf ./jemalloc.tar.bz2; \
    rm -f  ./jemalloc.tar.bz2; \
    cd jemalloc-5.2.1; \
    ./configure --host=x86_64-w64-mingw32 --build=x86_64-unknown_linux-gnu --disable-cxx --disable-initial-exec-tls --with-private-namespace=_rjem_ --with-jemalloc-prefix=; \
    make; \
    ar crus /usr/local/lib/libjemalloc.a src/jemalloc.o src/arena.o src/background_thread.o src/base.o \
    src/bin.o src/bitmap.o src/ckh.o src/ctl.o src/div.o src/extent.o src/extent_dss.o src/extent_mmap.o \
    src/hash.o src/hook.o src/large.o src/log.o src/malloc_io.o src/mutex.o src/mutex_pool.o src/nstime.o \
    src/pages.o src/prng.o src/prof.o src/rtree.o src/safety_check.o src/stats.o src/sc.o src/sz.o src/tcache.o \
    src/test_hooks.o src/ticker.o src/tsd.o src/witness.o; \
    cd ..; \
    rm -rf ./jemalloc-5.2.1;


ENV JEMALLOC_OVERRIDE=/usr/local/lib/libjemalloc.a