Demo/voice to command benchmark

Examples/VoiceRecorderSample/LFM_CRASH_INVESTIGATION.md

@@ -0,0 +1,98 @@
+# LFM Crash Investigation
+
+Date: 2026-05-27
+
+Device: iPhone 16 Pro, iOS 26.3.1 (23D771330a)
+
+App: `dev.wildedge.sample.voice`, benchmark build
+
+LeapSDK: 0.10.6
+
+## Finding
+
+The current LFM crash is caused by LeapSDK aborting while loading `LFM2.5-Audio-1.5B` with `Q4_0`.
+
+Captured console output from the device:
+
+```text
+LeapSDK load started: LFM2.5-Audio-1.5B Q4_0
+Using lfm2 audio engine
+loaded meta data ... LFM2.5-Audio-1.5B-Q4_0.gguf
+general.architecture = lfm2
+lfm2.context_length = 128000
+llama_model_load: error loading model: error loading model hyperparameters: key not found in model: gpt-oss.context_length
+LeapModelLoadingException: Failed to load model: 5
+Uncaught Kotlin exception
+App terminated due to signal 6.
+```
+
+The crash reports match this:
+
+- `VoiceRecorderSample-2026-05-27-202205.ips`: `EXC_CRASH`, `SIGABRT`, `InferenceEngineModelRunner.Companion#loadModel`
+- `VoiceRecorderSample-2026-05-27-204247.ips`: `EXC_CRASH`, `SIGABRT`, `InferenceEngineModelRunner.Companion#loadModel`
+- `VoiceRecorderSample-2026-05-27-204640.ips`: `EXC_CRASH`, `SIGABRT`, `InferenceEngineModelRunner.Companion#loadModel`
+
+This is not a Swift error path. The SDK throws an uncaught Kotlin exception after the backend load failure, so a normal Swift `do/catch` around `Leap.shared.load` is not enough to keep the app alive.
+
+## Current Mitigation
+
+The app now blocks the known-bad load before calling LeapSDK:
+
+- `LeapSpeechTranscriber.prepareModel()` returns a controlled error while `LeapSpeechTranscriber.isLoadTemporarilyDisabled` is true.
+- Benchmark remote config validation marks direct `speech_to_tool` as unsupported while this SDK/model combination is blocked.
+- Benchmark remote config validation also rejects Leap audio for `speech_to_text` while this SDK/model combination is blocked.
+
+This keeps the app from repeatedly crashing when the fetched config asks for Leap audio.
+
+## Ranked Theses
+
+1. Confirmed: LeapSDK 0.10.6 has a metadata/backend mismatch for `LFM2.5-Audio-1.5B Q4_0`.
+
+Evidence: the loaded GGUF metadata exposes `lfm2.context_length`, but the backend asks for `gpt-oss.context_length`. The crash happens inside LeapSDK model loading before generation starts.
+
+Test to clear: update LeapSDK or the Leap-managed model package and verify the same direct `Leap.shared.load(model: "LFM2.5-Audio-1.5B", quantization: "Q4_0", options: nil)` no longer asks for `gpt-oss.context_length`.
+
+2. Very likely: the same LeapSDK metadata issue explains the text-only LFM failures.
+
+Evidence: `lfm2.5-350m` and `lfm2.5-1.2b-instruct` previously failed at load with the same missing `gpt-oss.context_length` metadata family. Those models are already disabled for `text_to_tool`.
+
+Test to clear: launch a small isolated app path that only calls `Leap.shared.load` for `LFM2.5-350M Q4_K_M` after the SDK/model update.
+
+3. Likely not a memory jetsam.
+
+Evidence: the device reports `EXC_CRASH`, `SIGABRT`, `Abort trap: 6`, and a LeapSDK/Kotlin unhandled exception. The console shows around 5460 MiB free on the Apple A18 Pro GPU before load. This is a backend abort, not an iOS memory kill.
+
+Test to clear: check for a matching `JetsamEvent` at the same timestamp. The copied VoiceRecorderSample reports are not jetsam reports.
+
+4. Possible but not primary: stale app container cache.
+
+Evidence: the GGUF path is in the app container under `Documents/leap_models`. Reinstalling the app does not necessarily clear this directory. A corrupt or outdated cached model could make repeat failures deterministic.
+
+Evidence against: the backend successfully reads a coherent GGUF V3 header and metadata, then fails on a specific missing metadata key. That points more to an SDK/model compatibility issue than a partial download.
+
+Test to clear: uninstall the app, reinstall, let LeapSDK redownload the model, and confirm whether the same metadata error appears. This test is useful but should not be expected to fix the key mismatch.
+
+5. Separate issue: llama.cpp text-to-tool can also abort.
+
+Evidence: `VoiceRecorderSample-2026-05-27-203432.ips` is a different crash family. It aborts in `ggml_abort`, `llama_decode`, and `LlamaCppBenchmarkTextToToolModel.decode`, not in LeapSDK. That report belongs to the TinyLlama/Qwen/FunctionGemma llama.cpp path, not the Leap LFM path.
+
+Test to clear: reproduce with a fixed text-only config and capture the llama.cpp log around `llama_decode`.
+
+6. Unlikely: prompt or audio fixture content.
+
+Evidence: the confirmed LFM crash happens in `loadModel` before `createConversation`, before the prompt, and before the audio file is sent to generation.
+
+Test to clear: call `prepareModel()` without any audio input. The current console run already failed during model preparation.
+
+7. Unlikely: our recent switch back to `options: nil`.
+
+Evidence: older commits used the same direct `Leap.shared.load(... options: nil ...)` shape for `LeapSpeechTranscriber`, and `project.yml` pinned LeapSDK 0.10.6 in the compared commits. The latest console failure is not caused by explicit manifest options.
+
+Test to clear: compare against a build with an older LeapSDK or a known-working Leap-managed model manifest rather than only comparing app code.
+
+## Next Decisive Actions
+
+1. Ask Liquid/LeapSDK for the SDK version or model package that supports LFM2 GGUF metadata using `lfm2.context_length`.
+2. After upgrading SDK/model package, temporarily set `LeapSpeechTranscriber.isLoadTemporarilyDisabled` to false and run only `prepareModel()` with the device console attached.
+3. If the SDK still fails, uninstall the app to clear `Documents/leap_models`, reinstall, and repeat the same single-load test.
+4. Only after `prepareModel()` succeeds should `speech_to_tool` benchmark runs be re-enabled.

Examples/VoiceRecorderSample/Scripts/prepare_leap_for_distribution.sh

@@ -0,0 +1,181 @@
+#!/bin/sh
+set -e
+
+APP_FRAMEWORKS_DIR="${TARGET_BUILD_DIR}/${FRAMEWORKS_FOLDER_PATH}"
+LEAP_FRAMEWORK="${APP_FRAMEWORKS_DIR}/LeapSDK.framework"
+LEAP_BINARY="${LEAP_FRAMEWORK}/LeapSDK"
+LEAP_NESTED_FRAMEWORKS="${LEAP_FRAMEWORK}/Frameworks"
+TARGET_MINIMUM_OS="${IPHONEOS_DEPLOYMENT_TARGET:-}"
+
+code_signing_enabled() {
+  [ -n "${EXPANDED_CODE_SIGN_IDENTITY:-}" ] &&
+    [ "${EXPANDED_CODE_SIGN_IDENTITY}" != "-" ] &&
+    [ "${CODE_SIGNING_ALLOWED:-YES}" != "NO" ] &&
+    [ "${CODE_SIGNING_REQUIRED:-YES}" != "NO" ]
+}
+
+framework_bundle_identifier() {
+  framework="$1"
+  /usr/libexec/PlistBuddy -c "Print :CFBundleIdentifier" "${framework}/Info.plist" 2>/dev/null || true
+}
+
+sign_framework() {
+  framework="$1"
+  identifier="${2:-}"
+
+  if [ ! -d "${framework}" ]; then
+    return 0
+  fi
+
+  if ! code_signing_enabled; then
+    echo "Skipping framework signing for ${framework}: code signing disabled."
+    return 0
+  fi
+
+  if [ -z "${identifier}" ]; then
+    identifier="$(framework_bundle_identifier "${framework}")"
+  fi
+
+  if [ -n "${identifier}" ]; then
+    /usr/bin/codesign --force --sign "${EXPANDED_CODE_SIGN_IDENTITY}" \
+      --identifier "${identifier}" \
+      --timestamp=none \
+      --generate-entitlement-der \
+      "${framework}"
+  else
+    /usr/bin/codesign --force --sign "${EXPANDED_CODE_SIGN_IDENTITY}" \
+      --timestamp=none \
+      --generate-entitlement-der \
+      "${framework}"
+  fi
+}
+
+set_framework_minimum_os() {
+  framework="$1"
+  info_plist="${framework}/Info.plist"
+
+  if [ -z "${TARGET_MINIMUM_OS}" ] || [ ! -f "${info_plist}" ]; then
+    return 0
+  fi
+
+  /usr/libexec/PlistBuddy -c "Set :MinimumOSVersion ${TARGET_MINIMUM_OS}" "${info_plist}" 2>/dev/null ||
+    /usr/libexec/PlistBuddy -c "Add :MinimumOSVersion string ${TARGET_MINIMUM_OS}" "${info_plist}"
+}
+
+set_framework_minimum_os "${APP_FRAMEWORKS_DIR}/onnxruntime.framework"
+sign_framework "${APP_FRAMEWORKS_DIR}/onnxruntime.framework"
+
+if [ ! -d "${LEAP_NESTED_FRAMEWORKS}" ]; then
+  exit 0
+fi
+
+patch_dependency() {
+  binary="$1"
+  old_path="$2"
+  new_path="$3"
+
+  if [ -f "${binary}" ]; then
+    /usr/bin/install_name_tool -change "${old_path}" "${new_path}" "${binary}" || true
+  fi
+}
+
+runtime_bundle_identifier() {
+  framework="$1"
+  binary_name="$(/usr/bin/basename "${framework}" .framework)"
+  bundle_suffix="$(printf "%s" "${binary_name}" | /usr/bin/tr "_" "-")"
+
+  printf "dev.wildedge.leap-runtime.%s" "${bundle_suffix}"
+}
+
+wrap_dylib_as_framework() {
+  dylib_name="$1"
+  binary_name="${dylib_name%.dylib}"
+  source_binary="${LEAP_NESTED_FRAMEWORKS}/${dylib_name}"
+  framework_dir="${APP_FRAMEWORKS_DIR}/${binary_name}.framework"
+  framework_binary="${framework_dir}/${binary_name}"
+
+  if [ ! -f "${source_binary}" ]; then
+    return 0
+  fi
+
+  /bin/rm -rf "${framework_dir}"
+  /bin/mkdir -p "${framework_dir}"
+  /bin/mv "${source_binary}" "${framework_binary}"
+  /bin/chmod 755 "${framework_binary}" || true
+
+  bundle_suffix="$(printf "%s" "${binary_name}" | /usr/bin/tr "_" "-")"
+  cat > "${framework_dir}/Info.plist" <<EOF
+<?xml version="1.0" encoding="UTF-8"?>
+<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
+<plist version="1.0">
+<dict>
+	<key>CFBundleDevelopmentRegion</key>
+	<string>en</string>
+	<key>CFBundleExecutable</key>
+	<string>${binary_name}</string>
+	<key>CFBundleIdentifier</key>
+	<string>dev.wildedge.leap-runtime.${bundle_suffix}</string>
+	<key>CFBundleInfoDictionaryVersion</key>
+	<string>6.0</string>
+	<key>CFBundleName</key>
+	<string>${binary_name}</string>
+	<key>CFBundlePackageType</key>
+	<string>FMWK</string>
+	<key>CFBundleShortVersionString</key>
+	<string>1.0</string>
+	<key>CFBundleSupportedPlatforms</key>
+	<array>
+		<string>iPhoneOS</string>
+	</array>
+	<key>CFBundleVersion</key>
+	<string>1</string>
+	<key>MinimumOSVersion</key>
+	<string>${IPHONEOS_DEPLOYMENT_TARGET:-17.0}</string>
+	<key>UIDeviceFamily</key>
+	<array>
+		<integer>1</integer>
+		<integer>2</integer>
+	</array>
+	<key>UIRequiredDeviceCapabilities</key>
+	<array>
+		<string>arm64</string>
+	</array>
+</dict>
+</plist>
+EOF
+
+  /usr/bin/install_name_tool -id "@rpath/${binary_name}.framework/${binary_name}" "${framework_binary}" || true
+}
+
+wrap_dylib_as_framework "libinference_engine_llamacpp_backend.dylib"
+wrap_dylib_as_framework "libinference_engine.dylib"
+wrap_dylib_as_framework "libie_zip.dylib"
+
+patch_dependency "${LEAP_BINARY}" "@rpath/libinference_engine.dylib" \
+  "@rpath/libinference_engine.framework/libinference_engine"
+patch_dependency "${LEAP_BINARY}" "@rpath/libinference_engine_llamacpp_backend.dylib" \
+  "@rpath/libinference_engine_llamacpp_backend.framework/libinference_engine_llamacpp_backend"
+patch_dependency "${LEAP_BINARY}" "@rpath/libie_zip.dylib" \
+  "@rpath/libie_zip.framework/libie_zip"
+
+patch_dependency "${APP_FRAMEWORKS_DIR}/libinference_engine.framework/libinference_engine" \
+  "@rpath/libinference_engine_llamacpp_backend.dylib" \
+  "@rpath/libinference_engine_llamacpp_backend.framework/libinference_engine_llamacpp_backend"
+patch_dependency "${APP_FRAMEWORKS_DIR}/libinference_engine.framework/libinference_engine" \
+  "@rpath/inference_engine_llamacpp_backend.framework/inference_engine_llamacpp_backend" \
+  "@rpath/libinference_engine_llamacpp_backend.framework/libinference_engine_llamacpp_backend"
+
+/bin/rm -rf "${LEAP_NESTED_FRAMEWORKS}"
+
+for framework in \
+  "${APP_FRAMEWORKS_DIR}/libinference_engine_llamacpp_backend.framework" \
+  "${APP_FRAMEWORKS_DIR}/libinference_engine.framework" \
+  "${APP_FRAMEWORKS_DIR}/libie_zip.framework"
+do
+  if [ -d "${framework}" ]; then
+    bundle_identifier="$(runtime_bundle_identifier "${framework}")"
+    sign_framework "${framework}" "${bundle_identifier}"
+  fi
+done
+
+sign_framework "${LEAP_FRAMEWORK}"

Examples/VoiceRecorderSample/Sources/Assets.xcassets/AppIcon.appiconset/Contents.json

@@ -0,0 +1,14 @@
+{
+  "images": [
+    {
+      "filename": "benchmark-icon.png",
+      "idiom": "universal",
+      "platform": "ios",
+      "size": "1024x1024"
+    }
+  ],
+  "info": {
+    "author": "xcode",
+    "version": 1
+  }
+}