Daemon rejects a byte-identical second copy of its own binary (image_unverifiable); client only sees an indefinite "pending"

[OmerZeyveli]

Version: eval-97ce23f (v0.9.0+338) · Platform: Linux 6.x, single user account

What happens

Two separate HOMEs, each with its own codebase-memory-mcp install, same version, byte-identical binaries (identical sha256, separate files on disk). With a daemon already warm from the first install, a client launched from the second never connects — it reports status: "pending" indefinitely and offers zero tools.

The reason appears only in the daemon log:

level=error msg=daemon.client_image_rejected reason=image_unverifiable

Two adjacent constraints compound the problem:

• the daemon socket lives in /tmp/cbm-daemon-<uid> — keyed by uid, so HOME isolation does not isolate the daemon;
• only one CBM_CACHE_DIR may be active per user at a time. A second is refused with "the active account daemon uses a different cache directory (active cache …; requested cache …). Close all CBM sessions and commands, then retry with one consistent CBM_CACHE_DIR."

That second message is clear and actionable. The image_unverifiable path is not: the client sees only pending, which is indistinguishable from a slow cold start.

Why it matters

Anyone running two workspaces side by side, or a build agent running two checkouts under one service account, hits this. It also makes A/B comparison of two configurations of the same version impossible without a workaround.

Workaround

Hard-linking both installs' binaries to a single inode, plus sharing one CBM_CACHE_DIR, makes both clients connect to one daemon. That works but is not something a user would discover from the symptom.

Suggestion

• If the check is a supply-chain guard, comparing binary content (hash) rather than image identity would admit a genuinely identical binary while keeping the guarantee.
• If single-instance enforcement is intentional, surfacing the reason to the client — instead of an indefinite pending — would save considerable diagnosis time.

[DeusData]

Thank you for isolating this with byte-identical binaries. Current daemon admission can classify a peer without a verifiable image identity as image_unverifiable, while the client-facing path can remain in a generic pending state. That combination explains the opaque hang surface even though it does not yet prove which identity check fails. I have routed this as a high-priority 0.9.1-rc daemon stability bug; the red case should require a deterministic accept or actionable rejection within a bounded deadline.

[DeusData]

Thank you for the exceptionally clear report — the two-HOMEs / byte-identical setup and the hard-link workaround made the diagnosis much faster.

Two updates:

1. The indefinite "pending" is fixed (#1439, on main shortly): the image-verification rejection was the only admission-failure path that never answered the peer. The client now receives an explicit rejection carrying the reason (fingerprint_mismatch / image_unverifiable) and an actionable message, in the same style as the cache-dir conflict message you rightly called clear and actionable.

2. Your suggestion about content-based verification is already the implemented model — and that's what makes your case interesting. Admission falls back to hashing the peer's executable via /proc/<pid>/exe and admits it when the fingerprint matches the daemon's, so two byte-identical binaries at different paths should connect to one daemon. image_unverifiable means that fingerprinting step failed outright on your machine, before any comparison. Could you share:

• kernel hardening state: cat /proc/sys/kernel/yama/ptrace_scope, and whether the box runs a hardened/grsec-style kernel or an LSM (SELinux/AppArmor) profile that restricts /proc/<pid>/exe access;
• where the second install's binary lives (local fs vs overlayfs/NFS/FUSE), and whether it was replaced/updated while the client was running;
• if easy: with the client stuck (pre-fix binary), does cat /proc/<client-pid>/exe >/dev/null succeed from the same user?

With the fix in place the client will at least tell you it was rejected and why — but I'd like to close the real gap so your A/B setup just works.