Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions docs/models-and-providers.md
Original file line number Diff line number Diff line change
Expand Up @@ -128,7 +128,7 @@ In practice you rarely need to weigh any of this yourself: Discover's **Staff pi

### mmproj (vision)

A vision model needs a second file, the **multimodal projector** (`mmproj`), that turns an image into something the model can read. Thuki downloads it alongside the main model and passes it to the engine with `--mmproj`. Models with this companion show a **Vision** badge.
A vision model needs a second file, the **multimodal projector** (`mmproj`), that turns an image into something the model can read. Thuki downloads it alongside the main model and passes it to the engine with `--mmproj`. Models with this companion show a **Vision** badge. Projector files and other non-chat helpers (draft / MTP / dspark) never appear as standalone chat downloads in Browse all: only primary text weights are listed, and a matching projector is attached automatically when you install a brain from the same repo.

### Capabilities

Expand All @@ -147,7 +147,7 @@ For a **Staff pick** the answer is baked in: the curated catalog records each mo
For **Browse all** there is no curated answer, so Thuki derives the badges live from what Hugging Face returns for each search result, the GGUF metadata block and the repo's file list, with no extra downloads:

- **Text**: every row you see is already a chat model. Browse all only lists repos whose Hugging Face task tag is a chat-style one (`text-generation` or `image-text-to-text`); image generators, embedders, and the like are filtered out before they reach you. That filter is what makes the Text badge a safe constant.
- **Vision**: Thuki scans the repo's file list for a multimodal projector (a `mmproj*.gguf` file). If one is present the model can read images, so it earns the Vision badge.
- **Vision**: Thuki scans the repo's file list for a multimodal projector (names like `mmproj-….gguf` or `…-mmproj-….gguf`). If one is present the model can read images, so it earns the Vision badge.
- **Reasoning**: this is read from the model's **chat template** (the embedded recipe that formats your messages), the only reliable signal for _how_ a model reasons. Thuki runs the template through a small classifier that recognizes the reasoning families: a structural reasoning channel (gpt-oss / Harmony), an `enable_thinking` / `thinking` switch (Qwen3, GLM, Granite), or always-on `<think>` / `<thought>` tags with no off switch (DeepSeek-R1, QwQ, Phi-4-reasoning). Match any of those and the model gets the Reasoning badge.

A repo's **name** is deliberately not trusted on its own. Plenty of models put "Thinking" or "Reasoning" in their title as marketing while shipping an ordinary chat template with no reasoning machinery at all, and badging those off the name would be a false promise. The name is consulted only as a last-resort fallback for the rare repo that ships no chat template for Thuki to read.
Expand Down
54 changes: 53 additions & 1 deletion src-tauri/src/commands.rs
Original file line number Diff line number Diff line change
Expand Up @@ -258,6 +258,13 @@ pub fn builtin_target(
.to_string(),
});
};
// Refuse projector/helper rows left over from older installs or bypasses.
if let Err(message) = crate::models::validate_primary_install(&model.file_name, None) {
return Err(EngineError {
kind: EngineErrorKind::ModelUnsupported,
message,
});
}
let model_path = if model.parts.is_empty() {
store.blob_path(&model.sha256)
} else {
Expand Down Expand Up @@ -446,7 +453,20 @@ fn is_os_incompatible(lower_detail: &str) -> bool {
/// runtime. Shared by `stream_builtin_chat` and `resolve_llm_transport`.
pub fn engine_start_error(detail: &str) -> EngineError {
let lower = detail.to_ascii_lowercase();
if lower.contains("unknown model architecture") || lower.contains("unknown architecture") {
// Projector GGUF loaded as `-m` (CLIP): clear product copy, not a vague start fail.
if lower.contains("architecture: 'clip'")
|| lower.contains("architecture: \"clip\"")
|| lower.contains("architecture: clip")
{
EngineError {
kind: EngineErrorKind::ModelUnsupported,
message: "This file is a vision projector, not a chat model.\nDownload a text model GGUF; Thuki attaches the projector automatically when the repo includes one.".to_string(),
}
} else if lower.contains("unknown model architecture")
|| lower.contains("unknown architecture")
|| lower.contains("unsupported model architecture")
|| lower.contains("unsupported architecture")
{
EngineError {
kind: EngineErrorKind::ModelUnsupported,
message: "Unsupported model\nThuki's engine doesn't support this model's architecture yet. Try another model; support expands as the engine updates.".to_string(),
Expand Down Expand Up @@ -3173,6 +3193,19 @@ mod tests {
)
}

#[test]
fn engine_start_error_clip_architecture_is_projector_copy() {
let err = engine_start_error(
"0.00.050.233 E llama_model_load: error loading model: unsupported model architecture: 'clip'",
);
assert_eq!(err.kind, EngineErrorKind::ModelUnsupported);
assert!(
err.message.contains("vision projector"),
"got: {}",
err.message
);
}

#[test]
fn engine_start_error_unknown_architecture_is_model_unsupported() {
let err = engine_start_error(
Expand Down Expand Up @@ -5298,6 +5331,25 @@ mod tests {
assert!(err.message.contains("Settings"));
}

/// A leftover projector install cannot be resolved as a chat target.
#[test]
fn builtin_target_rejects_projector_file_name() {
let conn = crate::database::open_in_memory().unwrap();
let dir = tempfile::tempdir().unwrap();
let store = crate::models::storage::ModelStore::new(dir.path().to_path_buf()).unwrap();
let mut row = installed_model("org/repo:Bonsai-27B-mmproj-Q8_0.gguf", "sha_p", None);
row.file_name = "Bonsai-27B-mmproj-Q8_0.gguf".to_string();
crate::models::manifest::insert(&conn, &row).unwrap();
let err = builtin_target(&conn, &store, "org/repo:Bonsai-27B-mmproj-Q8_0.gguf", 4096)
.unwrap_err();
assert_eq!(err.kind, EngineErrorKind::ModelUnsupported);
assert!(
err.message.contains("vision projector"),
"got: {}",
err.message
);
}

#[test]
fn builtin_target_manifest_read_error_is_other() {
// A bare connection without the schema makes `manifest::get` fail.
Expand Down
74 changes: 66 additions & 8 deletions src-tauri/src/models/gguf.rs
Original file line number Diff line number Diff line change
Expand Up @@ -32,21 +32,24 @@ const GGUF_TYPE_STRING: u32 = 8;
/// GGUF value type tag for an array (`elem_type(u32) | count(u64) | elements`).
const GGUF_TYPE_ARRAY: u32 = 9;

/// Metadata extracted from a GGUF header. Either field is `None` when the
/// Metadata extracted from a GGUF header. Each field is `None` when the
/// model does not carry it (or the reader stopped before reaching it).
#[derive(Debug, Default, Clone, PartialEq, Eq)]
pub struct GgufMetadata {
/// The embedded Jinja chat template (`tokenizer.chat_template`).
pub chat_template: Option<String>,
/// The model architecture (`general.architecture`, e.g. `qwen3`, `gpt-oss`).
pub architecture: Option<String>,
/// File role hint (`general.type`, e.g. `model`, `mmproj`, `adapter`).
pub general_type: Option<String>,
}

/// Reads `general.architecture` and `tokenizer.chat_template` from a GGUF
/// stream. Returns `None` only when the stream is not a GGUF the reader
/// understands (bad magic, unsupported version, or a header too short to carry
/// the counts); a stream that is a valid GGUF but is truncated or malformed
/// partway through returns `Some` with whatever was decoded before the fault.
/// Reads `general.architecture`, `general.type`, and `tokenizer.chat_template`
/// from a GGUF stream. Returns `None` only when the stream is not a GGUF the
/// reader understands (bad magic, unsupported version, or a header too short to
/// carry the counts); a stream that is a valid GGUF but is truncated or
/// malformed partway through returns `Some` with whatever was decoded before
/// the fault.
///
/// Generic over [`Read`] + [`Seek`] so it is driven by an in-memory
/// [`std::io::Cursor`] in tests and a [`std::io::BufReader`] over the blob
Expand Down Expand Up @@ -96,12 +99,26 @@ pub fn read_gguf_metadata<R: Read + Seek>(r: &mut R) -> Option<GgufMetadata> {
Some(s) => meta.architecture = Some(s),
None => break,
}
} else if value_type == GGUF_TYPE_STRING && key == b"general.type" {
match read_string_value(r) {
Some(s) => meta.general_type = Some(s),
None => break,
}
} else if skip_value(r, value_type).is_none() {
break;
}

// Both targets found: no reason to walk the rest of the header.
if meta.chat_template.is_some() && meta.architecture.is_some() {
// Role-critical fields plus template: stop walking once we have enough
// for both reasoning classification and primary-vs-projector gating.
if meta.chat_template.is_some()
&& meta.architecture.is_some()
&& meta.general_type.is_some()
{
break;
}
// Projectors often lack a chat template; architecture alone (clip) or
// architecture + type is enough to stop after a reasonable scan.
if meta.architecture.as_deref() == Some("clip") && meta.general_type.is_some() {
break;
}
}
Expand Down Expand Up @@ -356,6 +373,47 @@ mod tests {
assert_eq!(meta.chat_template, None);
}

#[test]
fn extracts_general_type() {
let bytes = build_gguf(
3,
&[
kv_string("general.architecture", b"clip"),
kv_string("general.type", b"mmproj"),
],
);
let meta = read(&bytes).unwrap();
assert_eq!(meta.architecture.as_deref(), Some("clip"));
assert_eq!(meta.general_type.as_deref(), Some("mmproj"));
}

#[test]
fn stops_after_template_arch_and_type_ignoring_trailing_malformed() {
let bad_nested = kv_array("trailing.bad", GGUF_TYPE_ARRAY, 1, &[]);
let bytes = build_gguf(
3,
&[
kv_string("general.architecture", b"qwen3"),
kv_string("general.type", b"model"),
kv_string("tokenizer.chat_template", b"<think>"),
bad_nested,
],
);
let meta = read(&bytes).unwrap();
assert_eq!(meta.architecture.as_deref(), Some("qwen3"));
assert_eq!(meta.general_type.as_deref(), Some("model"));
assert_eq!(meta.chat_template.as_deref(), Some("<think>"));
}

#[test]
fn general_type_string_too_large_stops_scan() {
let mut kv = enc_string(b"general.type");
kv.extend_from_slice(&GGUF_TYPE_STRING.to_le_bytes());
kv.extend_from_slice(&(MAX_GGUF_STRING_BYTES + 1).to_le_bytes());
let bytes = build_gguf(3, &[kv]);
assert_eq!(read(&bytes), Some(GgufMetadata::default()));
}

#[test]
fn stops_after_both_found_ignoring_trailing_malformed_kv() {
// A nested-array KV (unsupported) AFTER both targets must not matter:
Expand Down
Loading