Skip to content

README.md

Latest commit

 

History

History
139 lines (101 loc) · 5.94 KB

README.md

File metadata and controls

139 lines (101 loc) · 5.94 KB
Raw

🎯 model-consistency

Make your AI agent behave consistently across any LLM model.

A portable skill for multi-model compatibility. Works with Claude, GPT, Gemini, DeepSeek, Grok, Kimi, Minimax, GLM, and any other LLM that reads system prompts from workspace files.

The Problem

You built your agent on Claude Opus. It works perfectly. You switch to Sonnet, Haiku, or a cheaper model to save cost — and suddenly:

  • It forgets its name
  • It announces tool calls instead of executing them
  • It skips verification steps
  • It leaks private data
  • It responds in the wrong language
  • Responses feel erratic and inconsistent

Why: Models vary in instruction-following depth. Placing critical rules early in the file improves compliance across models. Long instruction files suffer from positional bias — rules buried deep are more likely to be skipped, especially by smaller models.

The Solution

Restructure your agent's instructions into two tiers:

Tier 1: CORE (top of file, keep compact) — High-priority for ALL models

A compact profile at the TOP of AGENTS.md containing ONLY the essentials:

  • Identity (name, user, language)
  • 8 core rules (always follow)
  • Script/tool cheatsheet
  • Current projects
  • Response style

Tier 2: Full Rules (below CORE) — Extended guidance

Your existing detailed rules, workflows, philosophy — everything you had before. Strong models read it all. Weaker models benefit from CORE already covering the essentials.

Installation

For OpenClaw users

# Clone into your workspace skills folder
cd ~/.openclaw/workspace/skills
git clone https://github.com/[YOUR_USER]/model-consistency.git

For other agent frameworks

The core concept (CORE section in AGENTS.md + standalone CORE.md mirror) works with any framework that uses file-based system prompts. Just copy SKILL.md and the scripts/ folder.

Quick Start

1. Generate your CORE section

python3 scripts/generate_core.py ~/.openclaw/workspace

This reads your existing workspace files and generates a compact CORE section. It now also mines MEMORY.md, detects real scripts from scripts/, and pulls current projects from workspace context. Review the output, then paste it at the TOP of your AGENTS.md.

2. Create a standalone CORE.md

Copy the same content into a CORE.md file at workspace root. Cron jobs and isolated sessions that don't get AGENTS.md injected can read this one instead.

3. Verify compatibility

python3 scripts/core_lint.py ~/.openclaw/workspace

This checks:

  • CORE section exists and is near the top
  • CORE is compact (≤60 lines)
  • All required components present with real content, not placeholder junk (Identity, Rules, Scripts, Projects, Style)
  • Key rules are included
  • CORE.md standalone exists and is in sync with AGENTS.md

Score 15/15 means your structure is solid.

4. Test with your models

Send the 8 prompts from references/TEST_PROMPTS.md to your agent after switching to a new model. Score each as PASS/FAIL.

What's Included

model-consistency/
├── SKILL.md                        # Full skill instructions for agents
├── README.md                       # This file (human-readable)
├── scripts/
│   ├── generate_core.py            # Auto-generate CORE from existing files
│   └── core_lint.py                # Static lint check of AGENTS.md structure
└── references/
    ├── CORE_TEMPLATE.md            # Blank CORE template to customize
    └── TEST_PROMPTS.md             # 10 live-test prompts for any model

The 8 Core Rules

These rules address the most common cross-model failure modes:

  1. ACT IMMEDIATELY — Act on requests by calling tools immediately. Never narrate intent.
  2. VERIFY RESULTS — After completing a task, check using a DIFFERENT method.
  3. RETRY ON FAILURE — Try a different approach on failure. Up to 3 attempts before asking for help.
  4. NO PRIVATE DATA LEAKS — No API keys, passwords, or personal info in group chats.
  5. NO_REPLY = entire message — When nothing to say, reply ONLY: NO_REPLY
  6. HEARTBEAT_OK = entire message — For empty heartbeats, reply ONLY: HEARTBEAT_OK
  7. CITE SOURCES — Include source URLs when reporting facts from the web.
  8. STOP WHEN DONE — Stop when the task is complete. Don't add unsolicited follow-ups.

Rules are designed to be tight and universal. Add domain-specific rules in the Full Rules section.

Limitations

This skill improves cross-model consistency but does not guarantee it. Be aware of:

  • Framework prompt assembly order — Your AGENTS.md may not be at the top of what the model sees. The framework's runtime preamble, tool specs, and skill manifests are injected before or around it.
  • Context window overflow — On small-context models (8k-32k), CORE + conversation + tool output may already overflow, and no amount of reordering helps.
  • Model-specific refusals — Some models have RLHF that overrides system prompt rules (Grok's edgy tone, GPT's safety refusals). CORE rules compete with base model behavior.
  • Tool adapter compatibility — "USE TOOLS" as a rule only works if the framework correctly adapts each model's tool-calling format. The model wanting to call a tool ≠ the tool actually being called.
  • Models that ignore system prompts — Some models (e.g., certain DeepSeek versions, some Mistral variants) heavily weight user turns over system prompts. For these, re-inject CORE into the first user turn.
  • Tokenizer differences — Line counts are an approximation. The same 60 lines tokenize differently across models (Claude BPE ≠ Gemini SentencePiece ≠ GPT cl100k).

Credits & Inspiration

  • OWL Framework (camel-ai) — for the "Never say I will, just DO it" insight
  • OpenClaw — for the workspace file architecture this builds on
  • Real pain — built after watching Sonnet and Gemini forget everything when switched mid-session

License

MIT — use it, fork it, ship it.