Skip to content

Commit b12f0aa

Browse files
committed
fix(ci): merge main and fold mmproj sizing into memory-fit override gate
Signed-off-by: Logan Nguyen <lg.131.dev@gmail.com>
2 parents 2184a50 + b8b24c1 commit b12f0aa

13 files changed

Lines changed: 686 additions & 118 deletions

.release-please-manifest.json

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,3 @@
11
{
2-
".": "0.15.9"
2+
".": "0.16.0"
33
}

CHANGELOG.md

Lines changed: 27 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -22,6 +22,33 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
2222
- **BREAKING**: Renamed the `[model]` section in `config.toml` to `[inference]` and reshaped it from a single `ollama_url` string into the providers schema described above. There is no backward-compatibility shim for the section name: if you had a custom `[model]` section, rename it to `[inference]` after upgrading; a flat `ollama_url` inside `[inference]` is migrated automatically.
2323
- Active model selection is now strictly Option-typed end to end: when nothing is installed and nothing is persisted, Thuki refuses to dispatch requests and surfaces a "Pick a model" prompt instead of falling back to a hardcoded slug. The previous `DEFAULT_MODEL_NAME` constant has been removed.
2424

25+
## [0.16.0](https://github.com/quiet-node/thuki/compare/v0.15.9...v0.16.0) (2026-07-18)
26+
27+
28+
### Features
29+
30+
* built-in web search with auto-search, citation audit, and progressive trace ([#312](https://github.com/quiet-node/thuki/issues/312)) ([7972b31](https://github.com/quiet-node/thuki/commit/7972b31346f5dd55e4ca396026e26fc974282e1c))
31+
* **ci:** ship Thuki Nightly as side-by-side signed install ([#336](https://github.com/quiet-node/thuki/issues/336)) ([3810dd1](https://github.com/quiet-node/thuki/commit/3810dd15cdff0c22276cd3e71c4347b6394a2253))
32+
* conversation-wide search evidence reuse ([#330](https://github.com/quiet-node/thuki/issues/330)) ([1fe32a4](https://github.com/quiet-node/thuki/commit/1fe32a4769b73a1acae7409b69933c9787853f03))
33+
* cut ForceWeb search latency and harden answer quality ([#324](https://github.com/quiet-node/thuki/issues/324)) ([51d2c1f](https://github.com/quiet-node/thuki/commit/51d2c1f5c281fce61e6ed415f18c7924f034d49b))
34+
* **history:** auto-save chats with retention and free chats ([#331](https://github.com/quiet-node/thuki/issues/331)) ([057b089](https://github.com/quiet-node/thuki/commit/057b089db4df43bc7f4560d3348550812b890717))
35+
* move Diagnostics to Behavior tab with trace retention and folder actions ([#325](https://github.com/quiet-node/thuki/issues/325)) ([70d68ac](https://github.com/quiet-node/thuki/commit/70d68ac489515699c2a2b46d162ffbd64faa2b54))
36+
* search language parity across retrieval and answers ([#326](https://github.com/quiet-node/thuki/issues/326)) ([9005d44](https://github.com/quiet-node/thuki/commit/9005d44ac14538d8f8dffdf74de6ad4c187ad75f))
37+
* search resilience and citation a11y ([#327](https://github.com/quiet-node/thuki/issues/327)) ([e8e2fd7](https://github.com/quiet-node/thuki/commit/e8e2fd73b98ed8d1cea1172d4c33fe88f72c79ae))
38+
* search trust and lawfulness package ([#323](https://github.com/quiet-node/thuki/issues/323)) ([151adf6](https://github.com/quiet-node/thuki/commit/151adf6768b4760039397d51f296f68e9064c4e1))
39+
* **settings:** Changelog tab with full release history ([#328](https://github.com/quiet-node/thuki/issues/328)) ([fd909fe](https://github.com/quiet-node/thuki/commit/fd909feda2c748136591a4629ad6ff852ef71e42))
40+
41+
42+
### Bug Fixes
43+
44+
* **ci:** make engine-gate throughput report-only instead of a blocking floor ([#321](https://github.com/quiet-node/thuki/issues/321)) ([07f1ca8](https://github.com/quiet-node/thuki/commit/07f1ca8afc9f2317395528d6e23371ec0d3e8218))
45+
* disclose when a requested web search can't reach the web or finds nothing ([#314](https://github.com/quiet-node/thuki/issues/314)) ([2a4d749](https://github.com/quiet-node/thuki/commit/2a4d749eb519540573085ef7d035a1ac0e75cd6c))
46+
* **models:** gate browse installs to chat brains only ([07e0b35](https://github.com/quiet-node/thuki/commit/07e0b35eae8d363f3476c3f072e9494a8d397b72))
47+
* **models:** gate Browse installs to chat brains only ([#337](https://github.com/quiet-node/thuki/issues/337)) ([07e0b35](https://github.com/quiet-node/thuki/commit/07e0b35eae8d363f3476c3f072e9494a8d397b72))
48+
* **search:** disclose when a requested web search is unreachable or finds nothing ([2a4d749](https://github.com/quiet-node/thuki/commit/2a4d749eb519540573085ef7d035a1ac0e75cd6c))
49+
* surface the 80% memory headroom rule in the model-fit warning ([#322](https://github.com/quiet-node/thuki/issues/322)) ([a9d5bb4](https://github.com/quiet-node/thuki/commit/a9d5bb4a297ae38a2b0739e28ff5ef22e746e507))
50+
* wire Auto search learn URL to disclosure blog post ([#335](https://github.com/quiet-node/thuki/issues/335)) ([01a99e2](https://github.com/quiet-node/thuki/commit/01a99e28790f916f5ce6d850cb917aae56ef9df9))
51+
2552
## [0.15.9](https://github.com/quiet-node/thuki/compare/v0.15.8...v0.15.9) (2026-07-10)
2653

2754

docs/configurations.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -194,6 +194,7 @@ The table below also lists the baked-in safety limits that govern Thuki's commun
194194
| `MAX_GGUF_KV_COUNT` | `4096` | No | Defense-in-depth bound on a downloaded GGUF's metadata-key count. A corrupt or hostile `metadata_kv_count` could otherwise drive an unbounded scan; real models carry a few dozen entries, so 4096 never truncates legitimate metadata. || The most GGUF metadata key-value pairs the reasoning classifier scans when reading a downloaded model's chat template. Scanning stops at the cap. |
195195
| `MAX_GGUF_KEY_BYTES` | `1 KiB` | No | Defense-in-depth bound on a downloaded GGUF's metadata-key length. Keys are short dotted identifiers (`tokenizer.chat_template`); capping the length stops a corrupt length field from forcing a large allocation. || The longest GGUF metadata key the reasoning classifier will read. A longer key stops the scan. |
196196
| `MAX_GGUF_STRING_BYTES` | `4 MiB` | No | Defense-in-depth bound on a downloaded GGUF's string values. Real chat templates run a few KB to ~100 KB; 4 MiB never truncates one while bounding the memory a corrupt length field can demand. || The largest GGUF string value (the chat template or architecture) the reasoning classifier will materialize. A larger value stops the scan and the model relies on the runtime backstop instead. |
197+
| `DENIED_PRIMARY_ARCHES` | `bert, t5, clip, …` | No | Defense-in-depth bound on attacker-influenced GGUF headers, and pin-scoped: a GGUF's `general.architecture` is editable, so a known non-chat family is refused as a primary chat model rather than handed to the engine, and an engine bump may add families that should stay out of the primary load path. Missing architecture stays soft (filename path) so an incomplete header never bricks an install. || Architectures that may never load as a primary chat model. A downloaded or pasted GGUF whose header reports one of these non-chat families (embedding, encoder-only, audio tokenizer) is rejected at finalize/load with a clear "not a chat model" message; a chat arch not on the list still passes. |
197198
| `HF_API_TIMEOUT_SECS` | `15 s` | No | Protocol cap on a hung remote service so the download UI cannot stall on metadata resolution; 15 s is generous for a small metadata call over the internet. || How long Thuki waits for a Hugging Face API metadata call (repo file listing) to respond before giving up. Applies to resolving pasted repo ids and listing a repo's GGUF files, not to the model download itself. |
198199
| `HF_BASE_URL` | `https://huggingface.co` | No | Single origin for model metadata and downloads. Provenance comes from the pinned repo revisions in the curated starter registry, and those pins are only meaningful against the canonical Hub; an arbitrary mirror could serve different content under the same revision ids. || The Hugging Face origin Thuki uses for all model metadata calls and blob downloads. Every starter in the registry pins a repo at an exact revision and carries a compiled-in sha256 digest checked after download; the digest catches truncation, bit rot, and resume corruption, while the pinned revision on the canonical Hub is what fixes which content is fetched. |
199200
| `HF_SEARCH_LIMIT` | `30` | No | The per-page step for the in-app model browser. The "Load more" control raises the requested page size in multiples of this value, so it is a layout step rather than a user preference. || How many GGUF model repos the first page of an in-app Hugging Face search returns, most-downloaded first. |

docs/models-and-providers.md

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -128,7 +128,9 @@ In practice you rarely need to weigh any of this yourself: Discover's **Staff pi
128128

129129
### mmproj (vision)
130130

131-
A vision model needs a second file, the **multimodal projector** (`mmproj`), that turns an image into something the model can read. Thuki downloads it alongside the main model and passes it to the engine with `--mmproj`. Models with this companion show a **Vision** badge. Projector files and other non-chat helpers (draft / MTP / dspark) never appear as standalone chat downloads in Browse all: only primary text weights are listed, and a matching projector is attached automatically when you install a brain from the same repo.
131+
A vision model needs a second file, the **multimodal projector** (`mmproj`), that turns an image into something the model can read. Thuki downloads it alongside the main model and passes it to the engine with `--mmproj`. Models with this companion show a **Vision** badge. Projector files and other non-chat helpers (draft / MTP / dspark / LoRA / embedders) never appear as standalone chat downloads in Browse all: only primary text weights with complete LFS metadata are listed, and a matching projector is attached automatically when you install a brain from the same repo. When a repo ships several projectors, Thuki ranks them like llama.cpp (shared path with the brain, then closest quant bit-width).
132+
133+
At install and load time Thuki also re-reads the GGUF header: CLIP/mmproj roles, embedding architectures (bert, t5, …), and other non-chat helpers are rejected as primary chat models so they never become Ready. Missing architecture stays soft (filename heuristics). The denylist is pin-scoped: an engine bump may need a list update when new non-chat families appear.
132134

133135
### Capabilities
134136

@@ -164,7 +166,7 @@ First it estimates the model's **resident size** at runtime:
164166
estimate = model file size + ~2 GB overhead
165167
```
166168

167-
The ~2 GB is a baseline for the engine's runtime buffers and KV cache. (The KV cache also grows with your context window, covered separately in the context-window guide.) For a vision model, a curated Staff pick folds its projector (`mmproj`) into the size; a Browse-all or installed-Library row counts only the single GGUF file you see on the row.
169+
The ~2 GB is a baseline for the engine's runtime buffers and KV cache. (The KV cache also grows with your context window, covered separately in the context-window guide.) For a vision model, Thuki folds the projector (`mmproj`) into the size when known: Staff picks use the curated projector size, and installed Library / load-gate paths use the on-disk projector blob (or the curated size when the blob length is not yet available).
168170

169171
Then it compares that estimate against your Mac's **total physical memory**, read straight from the system (`hw.memsize`). Apple Silicon uses unified memory shared between the CPU and GPU, so a model competes for RAM with everything else you are running:
170172

package.json

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
{
22
"name": "thuki",
3-
"version": "0.15.9",
3+
"version": "0.16.0",
44
"description": "A floating AI secretary for macOS with its own local AI engine",
55
"license": "Apache-2.0",
66
"repository": {

src-tauri/Cargo.lock

Lines changed: 1 addition & 1 deletion
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

src-tauri/Cargo.toml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
[package]
22
name = "thuki"
3-
version = "0.15.9"
3+
version = "0.16.0"
44
description = "Thuki: The context-aware floating secretary"
55
authors = ["Logan Nguyen"]
66
license = "Apache-2.0"

src-tauri/src/commands.rs

Lines changed: 91 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -258,15 +258,18 @@ pub fn builtin_target(
258258
.to_string(),
259259
});
260260
};
261-
// Refuse projector/helper rows left over from older installs or bypasses.
262-
if let Err(message) = crate::models::validate_primary_install(&model.file_name, None) {
261+
// Refuse projector/helper/denied-arch rows: re-read the local blob header
262+
// so a renamed companion or embedding GGUF cannot load as chat primary.
263+
let weights_blob = store.blob_path(&model.sha256);
264+
let meta = crate::models::gguf::read_gguf_metadata_from_file(&weights_blob);
265+
if let Err(message) = crate::models::validate_primary_install(&model.file_name, meta.as_ref()) {
263266
return Err(EngineError {
264267
kind: EngineErrorKind::ModelUnsupported,
265268
message,
266269
});
267270
}
268271
let model_path = if model.parts.is_empty() {
269-
store.blob_path(&model.sha256)
272+
weights_blob
270273
} else {
271274
store
272275
.materialize_split_shim(&model.parts)
@@ -292,7 +295,7 @@ pub fn builtin_target(
292295
/// pure [`crate::models::memory::decide_load_gate`] needs and delegates every
293296
/// decision to it (the single source of the block decision, shared with the
294297
/// frontend fit affordance so the two can never drift, issue #296):
295-
/// - the target's weights bytes from the manifest,
298+
/// - the target's weights bytes from the manifest (plus mmproj when known),
296299
/// - live available memory from the mach VM statistics,
297300
/// - the currently-resident model's path (so a same-model reload is a no-op and
298301
/// a different resident model's footprint is credited back, since the engine
@@ -327,13 +330,18 @@ pub(crate) fn preflight_memory_gate(
327330
// Read the live engine status once: it feeds both the already-loading bypass
328331
// and the resident-credit path, both applied inside `decide_load_gate`.
329332
let status = engine.current_status();
330-
// Cannot size the target -> do not block on a database hiccup. The same row
331-
// carries the weights SHA-256 that keys the remembered-override list.
333+
// Cannot size the target -> do not block on a database hiccup. Fold mmproj
334+
// blob length (or registry hint) when present so vision loads estimate true
335+
// footprint. The same row carries the weights SHA-256 that keys the
336+
// remembered-override list, so no extra lookup is needed.
332337
let (target_weights, dismissed) = match crate::models::manifest::get(conn, model_id) {
333-
Ok(Some(row)) => (
334-
memory::model_weights_bytes(&row),
335-
memory::is_model_remembered(&row.sha256, dismissed_shas),
336-
),
338+
Ok(Some(row)) => {
339+
let mm = crate::models::memory::mmproj_bytes_for_model(store, &row);
340+
(
341+
memory::model_load_bytes(&row, mm),
342+
memory::is_model_remembered(&row.sha256, dismissed_shas),
343+
)
344+
}
337345
_ => return memory::MemoryGate::Proceed,
338346
};
339347
// Map every installed row to (weights_bytes, weights blob path) so a
@@ -342,8 +350,9 @@ pub(crate) fn preflight_memory_gate(
342350
.unwrap_or_default()
343351
.into_iter()
344352
.map(|row| {
353+
let mm = crate::models::memory::mmproj_bytes_for_model(store, &row);
345354
(
346-
memory::model_weights_bytes(&row),
355+
memory::model_load_bytes(&row, mm),
347356
store.blob_path(&row.sha256),
348357
)
349358
})
@@ -479,6 +488,12 @@ pub fn engine_start_error(detail: &str) -> EngineError {
479488
kind: EngineErrorKind::ModelUnsupported,
480489
message: "This file is a vision projector, not a chat model.\nDownload a text model GGUF; Thuki attaches the projector automatically when the repo includes one.".to_string(),
481490
}
491+
} else if is_mmproj_mismatch(&lower) {
492+
// Wrong projector paired with text weights (n_embd / dim mismatch).
493+
EngineError {
494+
kind: EngineErrorKind::EngineStartFailed,
495+
message: "Vision projector mismatch\nThis model's projector does not match the text weights. Re-download the model so Thuki can pick a matching mmproj, or choose another vision model.".to_string(),
496+
}
482497
} else if lower.contains("unknown model architecture")
483498
|| lower.contains("unknown architecture")
484499
|| lower.contains("unsupported model architecture")
@@ -501,6 +516,29 @@ pub fn engine_start_error(detail: &str) -> EngineError {
501516
}
502517
}
503518

519+
/// True when llama.cpp stderr indicates a vision projector mismatch.
520+
///
521+
/// Requires the failure to actually name a projector: `mmproj`, `projector`, or
522+
/// `clip` appear in the engine's stderr only when a vision projector (`--mmproj`)
523+
/// is being loaded. Gating on that keeps a text-only load's metadata dump (which
524+
/// always prints an `n_embd = N` line) or a dyld / OS start failure from being
525+
/// mislabeled a projector mismatch and shadowing the unknown-architecture and OS
526+
/// arms below. Within a projector load, only an explicit incompatibility (never
527+
/// the bare `n_embd` dump line, never a lone `wrong`) marks a genuine mismatch;
528+
/// any other projector failure falls through to the raw engine reason.
529+
fn is_mmproj_mismatch(lower: &str) -> bool {
530+
let mentions_projector =
531+
lower.contains("mmproj") || lower.contains("projector") || lower.contains("clip");
532+
if !mentions_projector {
533+
return false;
534+
}
535+
lower.contains("failed to load mmproj")
536+
|| lower.contains("mismatch")
537+
|| lower.contains("does not match")
538+
|| lower.contains("incompatible")
539+
|| lower.contains("not compatible")
540+
}
541+
504542
/// Runs the built-in-engine stage of a chat turn: mark activity, ensure the
505543
/// engine serves `target`, then stream via the `/v1` client at the engine's
506544
/// port. An engine activity guard is held for the whole turn (ensure,
@@ -3231,6 +3269,48 @@ mod tests {
32313269
);
32323270
}
32333271

3272+
#[test]
3273+
fn engine_start_error_projector_mismatch_positive() {
3274+
// Each projector-token arm (mmproj / projector / clip) paired with each
3275+
// incompatibility verb maps to the fixable projector-mismatch copy.
3276+
for detail in [
3277+
"failed to load mmproj: bad projector file",
3278+
"clip: n_embd (2048) does not match model n_embd (4096)",
3279+
"projector tensors are incompatible with the model",
3280+
"mmproj embedding mismatch detected",
3281+
"vision projector is not compatible with these weights",
3282+
] {
3283+
let err = engine_start_error(detail);
3284+
assert_eq!(err.kind, EngineErrorKind::EngineStartFailed, "{detail}");
3285+
assert!(
3286+
err.message.contains("Vision projector mismatch"),
3287+
"detail={detail} got={}",
3288+
err.message
3289+
);
3290+
}
3291+
}
3292+
3293+
#[test]
3294+
fn engine_start_error_projector_mismatch_needs_projector_and_verb() {
3295+
// A text-only failure never names a projector, so its ubiquitous
3296+
// `n_embd`/`mismatch` metadata dump must not be mislabeled a projector
3297+
// problem; and a projector load that fails for another reason (out of
3298+
// memory) or with only the dropped bare `wrong` token also falls through
3299+
// to the raw engine reason rather than the projector copy.
3300+
for detail in [
3301+
"n_embd values mismatch between tensors",
3302+
"clip_model_load: loaded; ggml backend: out of memory",
3303+
"clip loaded; wrong number of tensors",
3304+
] {
3305+
let err = engine_start_error(detail);
3306+
assert!(
3307+
!err.message.contains("Vision projector mismatch"),
3308+
"detail={detail} got={}",
3309+
err.message
3310+
);
3311+
}
3312+
}
3313+
32343314
#[test]
32353315
fn engine_start_error_unknown_architecture_is_model_unsupported() {
32363316
let err = engine_start_error(

0 commit comments

Comments
 (0)