Android glue

[dvc94ch]

The native-app-glue provided by the ndk which is used by cargo-apk is buggy. This PR implements the required parts in rust and rewrites cargo-apk, because it's overly complicated and hard to maintain.

• Ability to set crate-type depending on target rust-lang/cargo#4881
• Cargo does not always search for .cargo/config file in project root rust-lang/cargo#2930
• New behavior of --feature + --package combination rust-lang/cargo#5364

Closes #10

[dvc94ch]

cc @katyo

[dvc94ch]

Got another batch of refactorings. Want to make sure the API works for the flutter-rs multiapk setup I'm working on first. In terms naming, should we rename cargo-ndk to cargo-apk, as the name cargo-ndk is already taken and archive the android-rs-glue repo?

[katyo]

I agree, the cargo-ndk should be named cargo-apk (or cargo-android).

Also I would like to implement some additional features. For example, the ability to extend Native Activity needed for things missing in NDK like IME interfacing and etc.

[katyo]

I tested android-no-glue example on device with target sdk 18 and it fails with:

java.lang.IllegalArgumentException: Unable to load native library: /data/app-lib/rust.android_no_glue-2/libandroid_no_glue.so

Possibly this related to #12 but I'm not sure.
It seems the native activity does not give extra info about why loading fails.
There is known workaround to get this extra info.
But it requires the possibility to build some java code.

[goddessfreya]

Hmmm, so you folks want to name it cargo-apk? I'd have to bug @tomaka to get publishing rights for the crate, so that might take a while.

[dvc94ch]

We probably also need to bug @mb64 for publishing the other crates... Anyone know if he is still alive? :)

[dvc94ch]

There are two remaining issues. apksigner has a .bat and not a .exe extension on windows and if a command fails we continue as we don't check the exit status anywhere. Other than that are @goddessfreya @katyo ok to merge?

[katyo]

@dvc94ch I think it will be good to print some usage/help when cargo apk called without args and/or with '--help' arg.

Currently when I call cargo apk it prints "Error: InvalidArgs". When I call cargo apk --help the cargo help is printed.

[katyo]

@dvc94ch As I see @mb64 completely inactive on github under last three month. It seems it will be problematic to communicate with him.

[katyo]

May be it will the better to implement proc_macro instead.
So using attribute macros the usage may looks like the follow:

#[ndk::entry]
fn main() {
// ....
}

For example you can see cortex-m-rt entry macro implementation.

[dvc94ch]

that may be a good improvement, but I'm getting some PR fatigue, can we leave it for a follow up?

[katyo]

Of course, don't worry :)

[dvc94ch]

For some reason jni still doesn't work for me. @katyo can you provide an example that worked for you?

#![cfg(target_os = "android")]

use jni_android_sys::android::content::Context;

ndk_glue::ndk_glue!(main);

fn main() {
    let vm = unsafe { jni_glue::VM::from_jni_local(&*ndk_glue::native_activity().vm()) };

    let native_lib_dir = vm.with_env(|env| {
        let context = Context::new(env).unwrap();
        //let info = context.getApplicationInfo().unwrap().unwrap();
        //let lib_dir = info.nativeLibraryDir().unwrap();
        //lib_dir.to_string().unwrap()
        ""
    });

    println!("native lib dir is {:?}", native_lib_dir);
}

02-19 12:50:53.938 13658 13658 F DEBUG   : pid: 13609, tid: 13654, name: Thread-2  >>> rust.example.jni <<<
02-19 12:50:53.945 13658 13658 F DEBUG   :     #08 pc 000000000001d7e0  /data/app/rust.example.jni-4w_2VFpG0ATyZAaj3WU-YA==/lib/arm64/libjni.so (jni_glue::env::Env::new_object_a::h6744845ef91a6224+76)
02-19 12:50:53.945 13658 13658 F DEBUG   :     #09 pc 000000000001ddbc  /data/app/rust.example.jni-4w_2VFpG0ATyZAaj3WU-YA==/lib/arm64/libjni.so (jni_android_sys::android::content::Context::new::hcf0b513db4bff890+152)
02-19 12:50:53.946 13658 13658 F DEBUG   :     #10 pc 0000000000005308  /data/app/rust.example.jni-4w_2VFpG0ATyZAaj3WU-YA==/lib/arm64/libjni.so (jni::main::_$u7b$$u7b$closure$u7d$$u7d$::hd7969440a25f58b6+20)
02-19 12:50:53.946 13658 13658 F DEBUG   :     #11 pc 0000000000004b78  /data/app/rust.example.jni-4w_2VFpG0ATyZAaj3WU-YA==/lib/arm64/libjni.so (jni_glue::vm::VM::with_env::ha2bfdd8554ee29ca+416)
02-19 12:50:53.946 13658 13658 F DEBUG   :     #12 pc 0000000000004d7c  /data/app/rust.example.jni-4w_2VFpG0ATyZAaj3WU-YA==/lib/arm64/libjni.so (jni::main::hfe40f5c643af1c31+44)
02-19 12:50:53.946 13658 13658 F DEBUG   :     #13 pc 0000000000010878  /data/app/rust.example.jni-4w_2VFpG0ATyZAaj3WU-YA==/lib/arm64/libjni.so (ndk_glue::init::_$u7b$$u7b$closure$u7d$$u7d$::h1f9650c5afa39c72+152)
02-19 12:50:53.946 13658 13658 F DEBUG   :     #14 pc 000000000000c2b4  /data/app/rust.example.jni-4w_2VFpG0ATyZAaj3WU-YA==/lib/arm64/libjni.so (std::sys_common::backtrace::__rust_begin_short_backtrace::hf5d359fb8c6f708f+16)
02-19 12:50:53.946 13658 13658 F DEBUG   :     #15 pc 000000000000b894  /data/app/rust.example.jni-4w_2VFpG0ATyZAaj3WU-YA==/lib/arm64/libjni.so (std::thread::Builder::spawn_unchecked::_$u7b$$u7b$closure$u7d$$u7d$::_$u7b$$u7b$closure$u7d$$u7d$::hea75f786b66b8340+16)
02-19 12:50:53.946 13658 13658 F DEBUG   :     #16 pc 000000000001305c  /data/app/rust.example.jni-4w_2VFpG0ATyZAaj3WU-YA==/lib/arm64/libjni.so (_$LT$std..panic..AssertUnwindSafe$LT$F$GT$$u20$as$u20$core..ops..function..FnOnce$LT$$LP$$RP$$GT$$GT$::call_once::hd818eb5bdbde1faa+16)
02-19 12:50:53.946 13658 13658 F DEBUG   :     #17 pc 0000000000014598  /data/app/rust.example.jni-4w_2VFpG0ATyZAaj3WU-YA==/lib/arm64/libjni.so (std::panicking::try::do_call::hbc0d5782dfb4d564+44)
02-19 12:50:53.946 13658 13658 F DEBUG   :     #18 pc 0000000000032990  /data/app/rust.example.jni-4w_2VFpG0ATyZAaj3WU-YA==/lib/arm64/libjni.so (__rust_maybe_catch_panic+32)
02-19 12:50:53.946 13658 13658 F DEBUG   :     #19 pc 0000000000014440  /data/app/rust.example.jni-4w_2VFpG0ATyZAaj3WU-YA==/lib/arm64/libjni.so (std::panicking::try::hbfe660383477c598+80)
02-19 12:50:53.946 13658 13658 F DEBUG   :     #20 pc 00000000000130e4  /data/app/rust.example.jni-4w_2VFpG0ATyZAaj3WU-YA==/lib/arm64/libjni.so (std::panic::catch_unwind::hca591d4539c5427a+16)
02-19 12:50:53.946 13658 13658 F DEBUG   :     #21 pc 000000000000b6c8  /data/app/rust.example.jni-4w_2VFpG0ATyZAaj3WU-YA==/lib/arm64/libjni.so (std::thread::Builder::spawn_unchecked::_$u7b$$u7b$closure$u7d$$u7d$::hcc10361d60d81092+212)
02-19 12:50:53.946 13658 13658 F DEBUG   :     #22 pc 0000000000008f20  /data/app/rust.example.jni-4w_2VFpG0ATyZAaj3WU-YA==/lib/arm64/libjni.so (core::ops::function::FnOnce::call_once$u7b$$u7b$vtable.shim$u7d$$u7d$::h45da1a1034e1bb1b+16)
02-19 12:50:53.946 13658 13658 F DEBUG   :     #23 pc 000000000002b18c  /data/app/rust.example.jni-4w_2VFpG0ATyZAaj3WU-YA==/lib/arm64/libjni.so (_$LT$alloc..boxed..Box$LT$F$GT$$u20$as$u20$core..ops..function..FnOnce$LT$A$GT$$GT$::call_once::h72164f22cf22b003+72)
02-19 12:50:53.946 13658 13658 F DEBUG   :     #24 pc 00000000000324ac  /data/app/rust.example.jni-4w_2VFpG0ATyZAaj3WU-YA==/lib/arm64/libjni.so (std::sys::unix::thread::Thread::new::thread_start::h8fae6fef275a32d6+20)

[katyo]

I using jni in my oboe-rs to deal with android APIs related to audio. See https://github.com/katyo/oboe-rs/tree/master/oboe/src/java_interface, in particular utils.rs.
Jni works good.
There is the demo.

I have an issues related to missing features in jni-bindgen.
So my bindings still doesn't use it for now.

[katyo]

So this approach won't work, I opened rust-lang/cargo#7899 but I'm not sure it'll be fixed anytime soon if at all. Suggestions for alternative approaches? :)

Tried to fix it in rust-lang/cargo#7900, but don't understand the problem well enough.

If I understand right the problem in changing crate type to cdylib for android builds only, i.e when cargo apk is used to build app.
Because the PR for cargo stalled for now (and it seems for long time), we forced try another workarounds.

[katyo]

The rust-android-gradle plugin from mozilla team overrides cargo linker by wrapper script to add specific options to it when linking.
https://github.com/mozilla/rust-android-gradle/blob/master/plugin/src/main/kotlin/com/nishtahir/CargoBuildTask.kt#L180
https://github.com/mozilla/rust-android-gradle/blob/master/plugin/src/main/kotlin/com/nishtahir/CargoBuildTask.kt#L193
Of course this is dirty hack but it works.
Is there some aspects which I don't understand right or which I don't account?

[dvc94ch]

I think it still relies on cargo to create the cdylib? https://github.com/mozilla/rust-android-gradle/blob/master/samples/rust/Cargo.toml#L12 although I'm a little confused why the manifest contains dylib instead of cdylib, as I thought dylib is for macosx.

[katyo]

Actually them wrapper for linker overrides cargo's behavior.
You free to set any cargo-type this setting will be ignored because in current implementation wrapper overrides it.

[dvc94ch]

The only interesting thing the linker wrapper seems to do is add this "-Wl,-soname,lib${cargoExtension.libname!!}.so". I'll try adding those to RUSTFLAGS and see if I can get it to work.

This didn't work for me:

export RUSTFLAGS="-Clink-arg=-Wl,-soname,libhello_world.so"

The linker wrapper only seems to add these args to the linker invokation:
https://github.com/mozilla/rust-android-gradle/blob/master/plugin/src/main/resources/com/nishtahir/linker-wrapper.py

[katyo]

For some reasons that cannot be done using RUSTFLAGS, otherwise no wrapper would be needed.

[dvc94ch]

So what's the easiest way to test this?

[katyo]

I think as experiment we may try add wrapper for linker in NDK.

[dvc94ch]

@UndeadLeech let's start with this one! :)

cc @Osspial

[Osspial]

@dvc94ch Hi! Is there anything in particular you'd like me to help out with? I'm not seeing anything in particular in the past few messages and this is a pretty long thread.

[dvc94ch]

@Osspial looks like this is ready to merge

[Osspial]

Oh, neat. Will do.

@dvc94ch Do you want collaborator access on this repository? It looks like you've been doing good work here, and that would make it easier for you to continue improving things.

[dvc94ch]

That would be cool. Can you publish to crates.io?

[Osspial]

I'll get that done as soon as I figure out how to build this properly on my Windows machine.

[Osspial]

@dvc94ch I managed to run into a standard library bug that's preventing me from building this on Windows, so it'll be a bit before I can personally upload this to crates.io. Could you pull the latest changes I made to this repository, publish it yourself, then add github:rust-windowing:publishers to the owners list for each of the crates?

[Osspial]

Alright, I managed to work around the bug with a patch to cc-rs. The crates have been published!

[dvc94ch]

Great! Thanks. I'll backport the winit stuff, just haven't had time yet.