|
| 1 | +# MNIST number detector Android App |
| 2 | + |
| 3 | +This project is a sample Android application that demonstrates how to integrate a `Burn` into an android app using the JNI (Java Native Interface). |
| 4 | + |
| 5 | +## Table of Contents |
| 6 | + |
| 7 | +- [Workflow](#workflow) |
| 8 | +- [Prerequisites](#prerequisites) |
| 9 | +- [Setup](#setup) |
| 10 | +- [How To make your own](#how-to-make-your-own) |
| 11 | +- [License](#license) |
| 12 | + |
| 13 | +## Workflow |
| 14 | +1. **Image Input:** The user provides an image input through the app's interface. |
| 15 | +2. **Image Processing:** The image is converted to a grayscale `byteArray` in Kotlin. |
| 16 | +3. **JNI Bridge:** The grayscale `byteArray` is passed to a Rust function via JNI. |
| 17 | +4. **Rust Processing:** The Rust function calls the `forward` method from the `burn` library, using a pretrained MNIST ONNX model to perform inference. |
| 18 | +5. **Result Handling:** The result, an integer representing the predicted digit, is logged to the android console and returned from Rust to Kotlin. |
| 19 | +6. **Output Display:** The predicted digit is displayed on the screen |
| 20 | + |
| 21 | +## Prerequisites |
| 22 | +- Android Studio (latest version recommended) |
| 23 | +- Rust (installed and configured) |
| 24 | +- Android NDK (Native Development Kit) |
| 25 | + |
| 26 | +## Setup |
| 27 | +1. **Install Rust dependencies:** |
| 28 | + |
| 29 | + Ensure Rust is installed and the `cargo` command is available: |
| 30 | + |
| 31 | + ```bash |
| 32 | + rustup update |
| 33 | + ``` |
| 34 | + And that you have installed all the rustup toolchains required: |
| 35 | + ```bash |
| 36 | + rustup target add \ |
| 37 | + aarch64-linux-android \ |
| 38 | + armv7-linux-androideabi \ |
| 39 | + i686-linux-android \ |
| 40 | + x86_64-linux-android |
| 41 | + ``` |
| 42 | + |
| 43 | +2. **Configure the Android NDK:** |
| 44 | + |
| 45 | + Ensure that the Android NDK is installed. You can install it via Android Studio's SDK Manager. |
| 46 | +
|
| 47 | +3. **Build the android app:** |
| 48 | +
|
| 49 | + Running the android app should automatically build the rust libraries due to the gradle tasks configured at the app level. (More on that later) |
| 50 | +
|
| 51 | +
|
| 52 | +## How To make your own |
| 53 | +1. There are a few ways to compile a rust library for android - |
| 54 | + - Add targets in `.cargo/config.toml` and build with them. Then we can add the `.so` files generated to the jni directory in `app/src/main/jniLibs` |
| 55 | + - Add gradle plugins (like [rust-android-gradle](https://github.com/mozilla/rust-android-gradle) or [cargo-ndk-android](https://github.com/willir/cargo-ndk-android-gradle) using `rust-android-gradle` in this project) to do the work for you, so that the rust library is built on each app build. (Might want to change for expensive library builds) |
| 56 | +2. To interface with Kotlin(Java) you can either use an interface generator (like [flapigen-rs](https://github.com/Dushistov/flapigen-rs)) or make them by yourself. This sample function doesn't use flapigen. |
| 57 | +3. Now the function to be called from android (`infer()` here) needs to follow the [JNI naming conventions](https://docs.oracle.com/javase/1.5.0/docs/guide/jni/spec/design.html) (The correct name is also shown in the call error if it doesn't exist). |
| 58 | +4. **Important** The first 2 arguments of the jni interfacing function will be the `env` variable (for interface functions) and the `this` object. The data you pass will start from the 3rd argument. |
| 59 | +5. Next for converting the data from java to rust data types, there are multiple functions in the env variable passed to the function. Use as required... |
| 60 | +6. Then in the app's `build.gradle` we add the part to run the cargo build before building the app and the also the cargo build details: |
| 61 | +```kotlin |
| 62 | +// Cargo build details |
| 63 | +cargo { |
| 64 | + module = "./src/main/rust" // Or whatever directory contains your Cargo.toml |
| 65 | + libname = "mnist_inference_android" // Or whatever matches Cargo.toml's [package] name. |
| 66 | + targets = listOf( |
| 67 | + "arm", "arm64", |
| 68 | + "x86", |
| 69 | + "x86_64" |
| 70 | + ) |
| 71 | + prebuiltToolchains = true |
| 72 | +} |
| 73 | +
|
| 74 | +// Used to build cargo before the android build task is run |
| 75 | +// See more options here: https://github.com/mozilla/rust-android-gradle/issues/133 |
| 76 | +project.afterEvaluate { |
| 77 | + tasks.withType(com.nishtahir.CargoBuildTask::class) |
| 78 | + .forEach { buildTask -> |
| 79 | + tasks.withType(com.android.build.gradle.tasks.MergeSourceSetFolders::class) |
| 80 | + .configureEach { |
| 81 | + this.inputs.dir( |
| 82 | + layout.buildDirectory.dir("rustJniLibs" + File.separatorChar + buildTask.toolchain!!.folder) |
| 83 | + ) |
| 84 | + this.dependsOn(buildTask) |
| 85 | + } |
| 86 | + } |
| 87 | +} |
| 88 | +``` |
| 89 | +(In the example we have also added the target directory in `config.toml` since otherwise it will build into the workspace target, which we do not want) |
| 90 | +7. Here the library's name is `mnist-android` so we will initialize it in our app: |
| 91 | +```kotlin |
| 92 | +class MainActivity : ComponentActivity() { |
| 93 | + init { |
| 94 | + System.loadLibrary("mnist_android") // Note: '-' is changed to '_' |
| 95 | + } |
| 96 | + ... |
| 97 | +} |
| 98 | +``` |
| 99 | +8. Finally use it by declaring it as an external function first |
| 100 | +```kotlin |
| 101 | +external fun infer(inputImage: ByteArray): Int; |
| 102 | +
|
| 103 | +... |
| 104 | +infer(byteArray) |
| 105 | +``` |
0 commit comments