Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/ci-build.yml
Original file line number Diff line number Diff line change
Expand Up @@ -164,7 +164,7 @@ jobs:
@litert//litert/vendors/qualcomm/dispatch:dispatch_api_so \
-- \
-//python/... \
-//kotlin/java/com/google/ai/edge/litertlm/example/...
-//kotlin/kotlin/com/google/ai/edge/litertlm/example/...
env:
ANDROID_NDK_HOME: ${{ steps.setup-ndk.outputs.ndk-path }}

Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/ml-ci-build-android.yml
Original file line number Diff line number Diff line change
Expand Up @@ -225,7 +225,7 @@ jobs:
@litert//litert/vendors/qualcomm/dispatch:dispatch_api_so \
-- \
-//python/... \
-//kotlin/java/com/google/ai/edge/litertlm/example/... \
-//kotlin/kotlin/com/google/ai/edge/litertlm/example/... \
-//runtime/components/rust/...

- name: Update litert_lm_main prebuilt for Android if new version tag is pushed.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ integrate them. Agents MUST follow these steps when the user chooses the
`--config=android_x86` for `x86`, or `--config=android_x86_64` for
`x86_64`):
* **Kotlin Class JAR**: Compile the Kotlin bindings (e.g.,
`//kotlin/java/com/google/ai/edge/litertlm:litertlm-android`). Locate
`//kotlin/kotlin/com/google/ai/edge/litertlm:litertlm-android`). Locate
the resulting output `.jar` that contains the compiled `.class` files.
* **Native JNI JAR**: Compile the native JNI libraries. Zip all required
`.so` files into a directory structure `lib/<target_abi>/` inside a
Expand Down Expand Up @@ -65,7 +65,7 @@ prebuilt targets.
**Discovered Dependency Exports**: When configuring the `litertlm_kotlin`
target, you MUST inspect the original `BUILD` file of the bindings library
inside your cloned LiteRT-LM source tree (e.g. read the `deps` attribute of the
`//kotlin/java/com/google/ai/edge/litertlm:litertlm-android` build target).
`//kotlin/kotlin/com/google/ai/edge/litertlm:litertlm-android` build target).
Register all third-party external libraries listed there inside the `exports`
attribute of the prebuilt `java_import`. This transitively propagates compiling
and runtime classpaths to the sample app, maintaining clean modular
Expand Down
46 changes: 36 additions & 10 deletions docs/api/kotlin/getting_started.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,10 +31,10 @@ suspend fun main() {
![](demo.gif)

To try out the above sample, clone the repo and run with
[example/Main.kt](../../../kotlin/java/com/google/ai/edge/litertlm/example/Main.kt):
[example/Main.kt](../../../kotlin/kotlin/com/google/ai/edge/litertlm/example/Main.kt):

```bazel
bazel run -c opt //kotlin/java/com/google/ai/edge/litertlm/example:main -- <abs_model_path>
bazel run -c opt //kotlin/kotlin/com/google/ai/edge/litertlm/example:main -- <abs_model_path>
```

Available `.litertlm` models are on the
Expand All @@ -52,18 +52,46 @@ Gradle/Maven users.

### 1. Add the Gradle dependency

#### Standard

```
dependencies {
// For Android
// For Android source set
implementation("com.google.ai.edge.litertlm:litertlm-android:latest.release")

// For JVM (Linux, MacOS, Windows)
// OR

// For JVM (Linux, macOS, Windows) source set
implementation("com.google.ai.edge.litertlm:litertlm-jvm:latest.release")
}
```

#### Multiplatform

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I assume a -core package won't be published just yet, so this multiplatform section might not be necessary for now.


```
commonMain {
dependencies {
// For common source set
implementation("com.google.ai.edge.litertlm:litertlm-core:latest.release")
}
}
androidMain {
dependencies {
// For Android source set
implementation("com.google.ai.edge.litertlm:litertlm-android:latest.release")
}
}
jvmMain {
dependencies {
// For JVM (Linux, macOS, Windows) source set
implementation("com.google.ai.edge.litertlm:litertlm-jvm:latest.release")
}
}
```

You can find the available versions on Google Maven in
[litertlm-android](https://maven.google.com/web/index.html#com.google.ai.edge.litertlm:litertlm-android)
[litertlm-core](https://maven.google.com/web/index.html#com.google.ai.edge.litertlm:litertlm-core),
[litertlm-android](https://maven.google.com/web/index.html#com.google.ai.edge.litertlm:litertlm-android),
and
[litertlm-jvm](https://maven.google.com/web/index.html#com.google.ai.edge.litertlm:litertlm-jvm).

Expand Down Expand Up @@ -194,8 +222,6 @@ through a callback.
import com.google.ai.edge.litertlm.Content
import com.google.ai.edge.litertlm.Message
import com.google.ai.edge.litertlm.MessageCallback
import java.util.concurrent.CountDownLatch
import java.util.concurrent.TimeUnit

val callback = object : MessageCallback {
override fun onMessage(message: Message) {
Expand Down Expand Up @@ -450,10 +476,10 @@ if (responseMessage.toolCalls.isNotEmpty()) {
#### Example

To try out tool use, clone the repo and run with
[example/ToolMain.kt](../../../kotlin/java/com/google/ai/edge/litertlm/example/ToolMain.kt):
[example/ToolMain.kt](../../../kotlin/kotlin/com/google/ai/edge/litertlm/example/ToolMain.kt):

```bazel
bazel run -c opt //kotlin/java/com/google/ai/edge/litertlm/example:tool -- <abs_model_path>
bazel run -c opt //kotlin/kotlin/com/google/ai/edge/litertlm/example:tool -- <abs_model_path>
```

### 7. Extra Template Context Variables
Expand Down Expand Up @@ -485,7 +511,7 @@ These variables are used within the Jinja-style prompt templates, e.g.,

## Error Handling

API methods can throw `LiteRtLmJniException` for errors from the native layer or
API methods can throw `LiteRtLmNativeException` for errors from the native layer or
standard Kotlin exceptions like `IllegalStateException` for lifecycle issues.
Always wrap API calls in try-catch blocks. The `onError` callback in
`MessageCallback` will also report errors during asynchronous operations.
Original file line number Diff line number Diff line change
Expand Up @@ -13,26 +13,16 @@
# limitations under the License.

load("@rules_kotlin//kotlin:android.bzl", "kt_android_library")
load("@rules_kotlin//kotlin:jvm.bzl", "kt_jvm_library")

package(default_visibility = ["//visibility:public"])

kt_jvm_library(
name = "litertlm-jvm",
srcs = glob(["*.kt"]),
resources = ["//kotlin/java/com/google/ai/edge/litertlm/jni:litertlm_jni"],
resource_strip_prefix = "kotlin/java",
deps = [
"@rules_kotlin//kotlin/compiler:kotlin-reflect",
"@maven//:com_google_code_gson_gson",
"@maven//:org_jetbrains_kotlinx_kotlinx_coroutines_core_jvm"
],
)

# Note: To build with dynamic linking, use the following build flags:
kt_android_library(
name = "litertlm-android",
srcs = glob(["*.kt"]),
associates = [
"//kotlin/kotlin/com/google/ai/edge/litertlm/core:litertlm-core",
"//kotlin/kotlin/com/google/ai/edge/litertlm/jni:litertlm-jni",
],
deps = [
"@rules_kotlin//kotlin/compiler:kotlin-reflect",
"@maven//:com_google_code_gson_gson",
Expand Down
37 changes: 37 additions & 0 deletions kotlin/kotlin/com/google/ai/edge/litertlm/android/Engine.kt
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
/*
* Copyright 2025 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.google.ai.edge.litertlm

import com.google.gson.JsonArray

/**
* Manages the lifecycle of a LiteRT-LM engine, providing an interface for interacting with the
* underlying native library.
*
* Example usage:
* ```
* val config = EngineConfig(modelPath = "...")
* val engine = Engine(config)
* engine.initialize()
* ...
* engine.close()
* ```
*
* @param engineConfig The configuration for the engine.
*/
fun Engine(engineConfig: EngineConfig): Engine {
return EngineJni(engineConfig, NativeLibraryLoaderJni)
}
27 changes: 27 additions & 0 deletions kotlin/kotlin/com/google/ai/edge/litertlm/core/BUILD
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
# Copyright 2025 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

load("@rules_kotlin//kotlin:jvm.bzl", "kt_jvm_library")

package(default_visibility = ["//visibility:public"])

kt_jvm_library(
name = "litertlm-core",
srcs = glob(["*.kt"]),
deps = [
"@rules_kotlin//kotlin/compiler:kotlin-reflect",
"@maven//:com_google_code_gson_gson",
"@maven//:org_jetbrains_kotlinx_kotlinx_coroutines_core_jvm"
],
)
Original file line number Diff line number Diff line change
Expand Up @@ -66,7 +66,7 @@ fun benchmark(
prompt: String = "How are you",
): BenchmarkInfo {
val enginePointer =
LiteRtLmJni.nativeCreateBenchmark(
LiteRtLmNative.nativeCreateBenchmark(
modelPath,
backend.name,
prefillTokens,
Expand All @@ -78,7 +78,7 @@ fun benchmark(

try {
val conversationHandle =
LiteRtLmJni.nativeCreateConversation(
LiteRtLmNative.nativeCreateConversation(
enginePointer,
null, // SamplerConfig
"[]", // messagesJsonString
Expand All @@ -97,6 +97,6 @@ fun benchmark(
return conversation.getBenchmarkInfo()
}
} finally {
LiteRtLmJni.nativeDeleteEngine(enginePointer)
LiteRtLmNative.nativeDeleteEngine(enginePointer)
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@
*/
package com.google.ai.edge.litertlm

import kotlin.jvm.Volatile
import kotlin.concurrent.Volatile

/**
* Provides information about capabilities and features supported by a LiteRT-LM file.
Expand All @@ -31,9 +31,9 @@ class Capabilities(modelPath: String) : AutoCloseable {
@Volatile private var handle: Long? = null

init {
val ptr = LiteRtLmJni.nativeCreateCapabilities(modelPath)
val ptr = LiteRtLmNative.nativeCreateCapabilities(modelPath)
if (ptr == 0L) {
throw LiteRtLmJniException("Failed to load capabilities for model: $modelPath")
throw LiteRtLmNativeException("Failed to load capabilities for model: $modelPath")
}
handle = ptr
}
Expand All @@ -42,15 +42,15 @@ class Capabilities(modelPath: String) : AutoCloseable {
fun hasSpeculativeDecodingSupport(): Boolean {
synchronized(lock) {
checkInitialized()
return LiteRtLmJni.nativeHasSpeculativeDecodingSupport(handle!!)
return LiteRtLmNative.nativeHasSpeculativeDecodingSupport(handle!!)
}
}

/** Closes the loaded capabilities and releases underlying resources. */
override fun close() {
synchronized(lock) {
checkInitialized()
LiteRtLmJni.nativeDeleteCapabilities(handle!!)
LiteRtLmNative.nativeDeleteCapabilities(handle!!)
handle = null
}
}
Expand Down
Loading
Loading